Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
We Distribute
  1. Home
  2. Technical Discussion
  3. Alright it's late and i need to go to bed, but here's a draft FEP to do full account migration with posts and whatever other kinda objects you want to bring with you.

Alright it's late and i need to go to bed, but here's a draft FEP to do full account migration with posts and whatever other kinda objects you want to bring with you.

Scheduled Pinned Locked Moved Technical Discussion
moveallpostsfedidevfepfep1580fullmigration
53 Posts 11 Posters 6 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • jonny@neuromatch.socialJ jonny@neuromatch.social

    and no it doesn't require emitting and processing 100,000 Move activities to 10,000 instances at once

    jonny@neuromatch.socialJ This user is from outside of this forum
    jonny@neuromatch.socialJ This user is from outside of this forum
    jonny@neuromatch.social
    wrote last edited by
    #3

    i think i'm supposed to direct discussion here, but i also linked back here from there, so idk in the spirit of everything being everywhere as far as i'm concerned go hogwild and yell at me online wherever you want

    Link Preview Image
    FEP-1580: Move Actor Objects with a `migration` Collection

    PR: #692 - WIP: FEP-1580: Move Actor Objects with a `migration` Collection - fediverse/fep - Codeberg.org Fedi thread: jonny (good kind): "Alright it's late and i need to go to bed, but he…" - neurospace.live Hey every…

    favicon

    SocialHub (socialhub.activitypub.rocks)

    jonny@neuromatch.socialJ julian@activitypub.spaceJ 2 Replies Last reply
    0
    • jonny@neuromatch.socialJ jonny@neuromatch.social

      i think i'm supposed to direct discussion here, but i also linked back here from there, so idk in the spirit of everything being everywhere as far as i'm concerned go hogwild and yell at me online wherever you want

      Link Preview Image
      FEP-1580: Move Actor Objects with a `migration` Collection

      PR: #692 - WIP: FEP-1580: Move Actor Objects with a `migration` Collection - fediverse/fep - Codeberg.org Fedi thread: jonny (good kind): "Alright it's late and i need to go to bed, but he…" - neurospace.live Hey every…

      favicon

      SocialHub (socialhub.activitypub.rocks)

      jonny@neuromatch.socialJ This user is from outside of this forum
      jonny@neuromatch.socialJ This user is from outside of this forum
      jonny@neuromatch.social
      wrote last edited by
      #4

      i'm almost disappointed bc it's so simple and obvious. there are a lot of caveats and requirements there because i was trying to make it clear enough to be implementable, but really it's as simple as doing this: https://neuromatch.social/@jonny/115338330944599542

      gatesvp@mstdn.caG 1 Reply Last reply
      0
      • jonny@neuromatch.socialJ jonny@neuromatch.social

        Alright it's late and i need to go to bed, but here's a draft FEP to do full account migration with posts and whatever other kinda objects you want to bring with you. It's a trivial expansion of existing ActivityPub/streams systems and supports gradual migration as it's implemented and after an account migration. It should be possible to migrate pretty much everything this way, both private and public objects.

        criticism, feedback, revisions, etc. welcome - i don't think this is a "final version" and there are certainly things i overlooked.

        Link Preview Image
        fep/fep/1580/fep-1580.md at e6f7b7ce32aa6f84dcfa7bfdc10fd65119d75984

        fep - Fediverse Enhancement Proposals

        favicon

        Codeberg.org (codeberg.org)

        Link Preview Image
        WIP: FEP-1580: Move Actor Objects with a `migration` Collection

        fep - Fediverse Enhancement Proposals

        favicon

        Codeberg.org (codeberg.org)

        #MoveAllPosts #FediDev #FEP #FEP_1580 #FullMigration #AccountMigration

        mariusor@metalhead.clubM This user is from outside of this forum
        mariusor@metalhead.clubM This user is from outside of this forum
        mariusor@metalhead.club
        wrote last edited by
        #5

        > The mapping from old to new URIs is not knowable in advance, as the local IDs used by one instance software need not map onto the IDs used by another[...]

        @jonny frankly you're making it more difficult for yourself here. There's no actual reason why regenerating all IDs should be necessary.

        You're forcing the new server to do processing for every moved object, which is very inefficient.

        A good proposal should try to make the whole move in one batch. Keep the existing ID paths and mitigate against collisions by having them namespaced properly in the first place (which I think most servers kinda do already).

        jonny@neuromatch.socialJ kopper@not-brain.d.on-t.workK 2 Replies Last reply
        0
        • mariusor@metalhead.clubM mariusor@metalhead.club

          > The mapping from old to new URIs is not knowable in advance, as the local IDs used by one instance software need not map onto the IDs used by another[...]

          @jonny frankly you're making it more difficult for yourself here. There's no actual reason why regenerating all IDs should be necessary.

          You're forcing the new server to do processing for every moved object, which is very inefficient.

          A good proposal should try to make the whole move in one batch. Keep the existing ID paths and mitigate against collisions by having them namespaced properly in the first place (which I think most servers kinda do already).

          jonny@neuromatch.socialJ This user is from outside of this forum
          jonny@neuromatch.socialJ This user is from outside of this forum
          jonny@neuromatch.social
          wrote last edited by
          #6

          @mariusor cool well i figured that the URI mapping step (which is cheap) was a good compromise vs "modifying the URI structure in every existing fediverse app to support proper namespacing" especially since you do need to hit each object to update the proofs anyway, but note taken.

          1 Reply Last reply
          0
          • mariusor@metalhead.clubM mariusor@metalhead.club

            > The mapping from old to new URIs is not knowable in advance, as the local IDs used by one instance software need not map onto the IDs used by another[...]

            @jonny frankly you're making it more difficult for yourself here. There's no actual reason why regenerating all IDs should be necessary.

            You're forcing the new server to do processing for every moved object, which is very inefficient.

            A good proposal should try to make the whole move in one batch. Keep the existing ID paths and mitigate against collisions by having them namespaced properly in the first place (which I think most servers kinda do already).

            kopper@not-brain.d.on-t.workK This user is from outside of this forum
            kopper@not-brain.d.on-t.workK This user is from outside of this forum
            kopper@not-brain.d.on-t.work
            wrote last edited by
            #7
            @mariusor @jonny another problem requiring namespacing would've massively helped. and yet we must keep bikeshedding because the spec people say so
            mariusor@metalhead.clubM 1 Reply Last reply
            1
            • jonny@neuromatch.socialJ jonny@neuromatch.social

              Alright it's late and i need to go to bed, but here's a draft FEP to do full account migration with posts and whatever other kinda objects you want to bring with you. It's a trivial expansion of existing ActivityPub/streams systems and supports gradual migration as it's implemented and after an account migration. It should be possible to migrate pretty much everything this way, both private and public objects.

              criticism, feedback, revisions, etc. welcome - i don't think this is a "final version" and there are certainly things i overlooked.

              Link Preview Image
              fep/fep/1580/fep-1580.md at e6f7b7ce32aa6f84dcfa7bfdc10fd65119d75984

              fep - Fediverse Enhancement Proposals

              favicon

              Codeberg.org (codeberg.org)

              Link Preview Image
              WIP: FEP-1580: Move Actor Objects with a `migration` Collection

              fep - Fediverse Enhancement Proposals

              favicon

              Codeberg.org (codeberg.org)

              #MoveAllPosts #FediDev #FEP #FEP_1580 #FullMigration #AccountMigration

              kopper@not-brain.d.on-t.workK This user is from outside of this forum
              kopper@not-brain.d.on-t.workK This user is from outside of this forum
              kopper@not-brain.d.on-t.work
              wrote last edited by
              #8
              @jonny still reading so not everything but:
              Supporting instances MUST indicate their support of this FEP by including its namespace in the @context of affected Actor objects.
              the recent archive.org downtime made all objects with contexts containing purl.org URLs completely unfederatable for any json-ld handling software. i really recommend not MUST-ing this
              jonny@neuromatch.socialJ 1 Reply Last reply
              0
              • kopper@not-brain.d.on-t.workK kopper@not-brain.d.on-t.work
                @jonny still reading so not everything but:
                Supporting instances MUST indicate their support of this FEP by including its namespace in the @context of affected Actor objects.
                the recent archive.org downtime made all objects with contexts containing purl.org URLs completely unfederatable for any json-ld handling software. i really recommend not MUST-ing this
                jonny@neuromatch.socialJ This user is from outside of this forum
                jonny@neuromatch.socialJ This user is from outside of this forum
                jonny@neuromatch.social
                wrote last edited by
                #9

                @kopper i was looking for an instance metadata item that was just some list of tokens that was "the list of things that the instance supports," and i could have sworn it existed, but i couldn't find it when i looked. most fedi apps (and even most LD apps) don't actually dereference the URIs in a context and treat them as tokens anyway, but yes failure to resolve terms is a big problem and am not a fan of DNS-based linked data.

                kopper@not-brain.d.on-t.workK julian@activitypub.spaceJ 2 Replies Last reply
                0
                • kopper@not-brain.d.on-t.workK kopper@not-brain.d.on-t.work
                  @mariusor @jonny another problem requiring namespacing would've massively helped. and yet we must keep bikeshedding because the spec people say so
                  mariusor@metalhead.clubM This user is from outside of this forum
                  mariusor@metalhead.clubM This user is from outside of this forum
                  mariusor@metalhead.club
                  wrote last edited by
                  #10

                  @kopper I don't understand your complaint here, nor the "subtle" dig at the spec writers.

                  @jonny

                  jonny@neuromatch.socialJ 1 Reply Last reply
                  0
                  • mariusor@metalhead.clubM mariusor@metalhead.club

                    @kopper I don't understand your complaint here, nor the "subtle" dig at the spec writers.

                    @jonny

                    jonny@neuromatch.socialJ This user is from outside of this forum
                    jonny@neuromatch.socialJ This user is from outside of this forum
                    jonny@neuromatch.social
                    wrote last edited by
                    #11

                    @mariusor not speaking for @kopper , but there's a longstanding problem that linked data doesn't do namespacing and doesn't handle relative locations/identities at all, and that bleeds into activitypub and fedi app design. so like fully agreed that it would be way more convenient to just be able to pick up a whole subtree that includes all my posts and move it over there because all the relationships between things are relative to a single root, but it don't work that way atm unfortunately

                    kopper@not-brain.d.on-t.workK 1 Reply Last reply
                    0
                    • jonny@neuromatch.socialJ jonny@neuromatch.social

                      @kopper i was looking for an instance metadata item that was just some list of tokens that was "the list of things that the instance supports," and i could have sworn it existed, but i couldn't find it when i looked. most fedi apps (and even most LD apps) don't actually dereference the URIs in a context and treat them as tokens anyway, but yes failure to resolve terms is a big problem and am not a fan of DNS-based linked data.

                      kopper@not-brain.d.on-t.workK This user is from outside of this forum
                      kopper@not-brain.d.on-t.workK This user is from outside of this forum
                      kopper@not-brain.d.on-t.work
                      wrote last edited by
                      #12
                      @jonny feature-detection based on the moves and migration collections' presence would be enough for this purpose i'd think?
                      jonny@neuromatch.socialJ 1 Reply Last reply
                      0
                      • kopper@not-brain.d.on-t.workK kopper@not-brain.d.on-t.work
                        @jonny feature-detection based on the moves and migration collections' presence would be enough for this purpose i'd think?
                        jonny@neuromatch.socialJ This user is from outside of this forum
                        jonny@neuromatch.socialJ This user is from outside of this forum
                        jonny@neuromatch.social
                        wrote last edited by
                        #13

                        @kopper ya that's what i landed on - it's effectively the same as requiring the vocabulary to be declared in the context, since the properties moves and migration would otherwise be undefined in the json-ld document. i still would just like to have a standard way of declaring instance capabilities (and i could have sworn there already was one) so there's an unambiguous, easily resolved way of checking "does the instance know about this" without having to happen upon it by refreshing a migrated account

                        1 Reply Last reply
                        0
                        • jonny@neuromatch.socialJ jonny@neuromatch.social

                          @mariusor not speaking for @kopper , but there's a longstanding problem that linked data doesn't do namespacing and doesn't handle relative locations/identities at all, and that bleeds into activitypub and fedi app design. so like fully agreed that it would be way more convenient to just be able to pick up a whole subtree that includes all my posts and move it over there because all the relationships between things are relative to a single root, but it don't work that way atm unfortunately

                          kopper@not-brain.d.on-t.workK This user is from outside of this forum
                          kopper@not-brain.d.on-t.workK This user is from outside of this forum
                          kopper@not-brain.d.on-t.work
                          wrote last edited by
                          #14
                          @jonny @mariusor from what i can tell the only reason the origin based security model and it's problems are a thing at all is because there is no mechanism to see who "owns" an object from a reference alone, which namespacing would solve

                          otherwise you have to fetch every single object ever referenced just to check for a backlink. not only is fetching over AP is already inefficient as it is with no batching and surprisingly slow crypto,
                          nobody serves backlinks of any kind
                          jonny@neuromatch.socialJ mariusor@metalhead.clubM trwnh@mastodon.socialT 3 Replies Last reply
                          0
                          • kopper@not-brain.d.on-t.workK kopper@not-brain.d.on-t.work
                            @jonny @mariusor from what i can tell the only reason the origin based security model and it's problems are a thing at all is because there is no mechanism to see who "owns" an object from a reference alone, which namespacing would solve

                            otherwise you have to fetch every single object ever referenced just to check for a backlink. not only is fetching over AP is already inefficient as it is with no batching and surprisingly slow crypto,
                            nobody serves backlinks of any kind
                            jonny@neuromatch.socialJ This user is from outside of this forum
                            jonny@neuromatch.socialJ This user is from outside of this forum
                            jonny@neuromatch.social
                            wrote last edited by
                            #15

                            @kopper @mariusor in my fediverse, objects are given absolute addresses like @{actorPublicKey}/whatever/i/want and displayed like {petName}/whatever/i/want

                            mariusor@metalhead.clubM trwnh@mastodon.socialT 2 Replies Last reply
                            0
                            • jonny@neuromatch.socialJ jonny@neuromatch.social

                              @kopper @mariusor in my fediverse, objects are given absolute addresses like @{actorPublicKey}/whatever/i/want and displayed like {petName}/whatever/i/want

                              mariusor@metalhead.clubM This user is from outside of this forum
                              mariusor@metalhead.clubM This user is from outside of this forum
                              mariusor@metalhead.club
                              wrote last edited by
                              #16

                              @jonny that's how I do it in ONI. But because it's a service that supports client to server the ID of the object can be generated in the client (or by the user) when they submit the Create activity.

                              See here an example: https://marius.federated.id/uploads/big-air-package (you can CURL that as ActivityPub IRI, or in a browser as HTML or directly access the image with the Accept: image/jpeg header)

                              @kopper

                              1 Reply Last reply
                              0
                              • kopper@not-brain.d.on-t.workK kopper@not-brain.d.on-t.work
                                @jonny @mariusor from what i can tell the only reason the origin based security model and it's problems are a thing at all is because there is no mechanism to see who "owns" an object from a reference alone, which namespacing would solve

                                otherwise you have to fetch every single object ever referenced just to check for a backlink. not only is fetching over AP is already inefficient as it is with no batching and surprisingly slow crypto,
                                nobody serves backlinks of any kind
                                mariusor@metalhead.clubM This user is from outside of this forum
                                mariusor@metalhead.clubM This user is from outside of this forum
                                mariusor@metalhead.club
                                wrote last edited by
                                #17

                                @kopper TY. 🙂

                                And fetching every single object in order to check who owns it should be the norm when that's relevant.

                                And quoting form your post on the activitypub.rocks forum:

                                > Therefore, if a user can create an actor-like object with private keys in their control, they can impersonate any other actor on their instance.

                                I don't follow your explanation. Maybe it's because I never plan to support request time fully dynamic JSON-LD contexts, because for the type of API I want to use for GoActivityPub, the best I'll be able to do is compilation time dynamic.

                                kopper@not-brain.d.on-t.workK 1 Reply Last reply
                                0
                                • mariusor@metalhead.clubM mariusor@metalhead.club

                                  @kopper TY. 🙂

                                  And fetching every single object in order to check who owns it should be the norm when that's relevant.

                                  And quoting form your post on the activitypub.rocks forum:

                                  > Therefore, if a user can create an actor-like object with private keys in their control, they can impersonate any other actor on their instance.

                                  I don't follow your explanation. Maybe it's because I never plan to support request time fully dynamic JSON-LD contexts, because for the type of API I want to use for GoActivityPub, the best I'll be able to do is compilation time dynamic.

                                  kopper@not-brain.d.on-t.workK This user is from outside of this forum
                                  kopper@not-brain.d.on-t.workK This user is from outside of this forum
                                  kopper@not-brain.d.on-t.work
                                  wrote last edited by
                                  #18
                                  @mariusor this only becomes an issue for software which aims to be completely generic, with little to no knowledge on the objects being stored in them. if that is not a goal then you don't need to worry about this attack on your own actors
                                  mariusor@metalhead.clubM 1 Reply Last reply
                                  0
                                  • kopper@not-brain.d.on-t.workK kopper@not-brain.d.on-t.work
                                    @mariusor this only becomes an issue for software which aims to be completely generic, with little to no knowledge on the objects being stored in them. if that is not a goal then you don't need to worry about this attack on your own actors
                                    mariusor@metalhead.clubM This user is from outside of this forum
                                    mariusor@metalhead.clubM This user is from outside of this forum
                                    mariusor@metalhead.club
                                    wrote last edited by
                                    #19

                                    @kopper I saw that the code you posted as an example for the issue was C#.

                                    Do you maybe have a solution on how to make the dynamic nature of JSON-LD documents play nice with static typing and low overhead data types for the Activity Vocabulary?

                                    For GoActivityPub I want an object in the vocabulary to correspond to an identically structured data type.

                                    So far I can only think of compile time metaprogramming to generate these types based on a known list of JSON-LD contexts, but maybe I'm not seeing the whole picture.

                                    1 Reply Last reply
                                    0
                                    • jonny@neuromatch.socialJ jonny@neuromatch.social

                                      i think i'm supposed to direct discussion here, but i also linked back here from there, so idk in the spirit of everything being everywhere as far as i'm concerned go hogwild and yell at me online wherever you want

                                      Link Preview Image
                                      FEP-1580: Move Actor Objects with a `migration` Collection

                                      PR: #692 - WIP: FEP-1580: Move Actor Objects with a `migration` Collection - fediverse/fep - Codeberg.org Fedi thread: jonny (good kind): "Alright it's late and i need to go to bed, but he…" - neurospace.live Hey every…

                                      favicon

                                      SocialHub (socialhub.activitypub.rocks)

                                      julian@activitypub.spaceJ This user is from outside of this forum
                                      julian@activitypub.spaceJ This user is from outside of this forum
                                      julian@activitypub.space
                                      wrote last edited by
                                      #20

                                      jonny@neuromatch.social FWIW there's no need to use SocialHub. You could post a discussion thread anywhere (like ActivityPub.Space, GitHub, a WordPress blog, etc... and yes, even a Mastodon status can be a starting point for discussion.)

                                      silverpill@mitra.socialS 1 Reply Last reply
                                      0
                                      • kopper@not-brain.d.on-t.workK kopper@not-brain.d.on-t.work
                                        @jonny @mariusor from what i can tell the only reason the origin based security model and it's problems are a thing at all is because there is no mechanism to see who "owns" an object from a reference alone, which namespacing would solve

                                        otherwise you have to fetch every single object ever referenced just to check for a backlink. not only is fetching over AP is already inefficient as it is with no batching and surprisingly slow crypto,
                                        nobody serves backlinks of any kind
                                        trwnh@mastodon.socialT This user is from outside of this forum
                                        trwnh@mastodon.socialT This user is from outside of this forum
                                        trwnh@mastodon.social
                                        wrote last edited by
                                        #21

                                        @kopper @jonny @mariusor

                                        > see who "owns" an object from a reference alone, which namespacing would solve

                                        i don't think that's desirable on its own (since id should be opaque) but you can get it anyway if you require actors to have their own dns name (and then tls does the rest). that's how the same-origin model is *supposed* to work in theory; the bit that breaks the assumption is when one origin has multiple tenants

                                        re: actor namespaces, i am writing/exploring a fep for that actually...

                                        trwnh@mastodon.socialT mariusor@metalhead.clubM 2 Replies Last reply
                                        0
                                        • trwnh@mastodon.socialT trwnh@mastodon.social

                                          @kopper @jonny @mariusor

                                          > see who "owns" an object from a reference alone, which namespacing would solve

                                          i don't think that's desirable on its own (since id should be opaque) but you can get it anyway if you require actors to have their own dns name (and then tls does the rest). that's how the same-origin model is *supposed* to work in theory; the bit that breaks the assumption is when one origin has multiple tenants

                                          re: actor namespaces, i am writing/exploring a fep for that actually...

                                          trwnh@mastodon.socialT This user is from outside of this forum
                                          trwnh@mastodon.socialT This user is from outside of this forum
                                          trwnh@mastodon.social
                                          wrote last edited by
                                          #22

                                          @kopper @jonny @mariusor basic idea is similar to toot:attributionDomains but with prefixes instead. the part i'm exploring is whether it makes more sense to just say heck it and use containers instead

                                          1 Reply Last reply
                                          0
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                          • Login

                                          • Don't have an account? Register

                                          • Login or register to search.
                                          Powered by NodeBB Contributors
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • World
                                          • Users
                                          • Groups