I have deeply mixed feelings about #ActivityPub's adoption of JSON-LD, as someone who's spent way too long dealing with it while building #Fedify.
-
@cwebber yes. Like I said, very low risk. If you want to be absolutely safe, wait until your first user reads the content before verifying it. It's usually not immediate. Most users aren't online. (TM)
@patmikemid @kopper @hongminhee@evan @patmikemid @kopper @hongminhee I'm sorry hold on Evan I'm sorry but it's NOT very low risk. That's a COMPLETE misunderstanding of the information landscape we are currently in.
Trust THEN verify?!?! Trust AND THEN verify?!?!!?!?!?!?
"A random several minutes" until we know whether or not the content delivered authentically is from said actor...
Even ONE minute is enough for someone to read, and believe, something false, and to reply, or to *take action*. Or to boost a post, which is then distributed across the fediverse, and then seen by a bunch of other nodes which also have not yet verified?
Trust AND THEN verify doesn't make sense!!!
AAAAAA I am losing my marbles over this one
-
@cwebber lucky you, you get all the first deliveries!
@cwebber some last thoughts on digital signatures for solving the thundering herd problem:
Unless the author's signing key is saturated in the network, you're going to have a thundering herd for the key, anyways. It's just pushing the problem down the line.
-
@cwebber some last thoughts on digital signatures for solving the thundering herd problem:
Unless the author's signing key is saturated in the network, you're going to have a thundering herd for the key, anyways. It's just pushing the problem down the line.
@evan If it's a popular author, which most commonly is the type who causes the thundering herd, then the chances the key is cached is very high!
-
@cwebber some last thoughts on digital signatures for solving the thundering herd problem:
Unless the author's signing key is saturated in the network, you're going to have a thundering herd for the key, anyways. It's just pushing the problem down the line.
If you don't think waiting until the first user loads the content to verify the content is an acceptable risk, there are still other solutions. One I like is using a content-addressed shared cache for public data, like IPFS. We have `alsoKnownAs` as a nice way to include this URI.
-
@evan If it's a popular author, which most commonly is the type who causes the thundering herd, then the chances the key is cached is very high!
@cwebber I think the use case you mentioned was an author with a small following getting boosted by one with a large following.
Regardless, even if the caching level is 90%, you're still doing a big percentage of the original herd.
-
If you don't think waiting until the first user loads the content to verify the content is an acceptable risk, there are still other solutions. One I like is using a content-addressed shared cache for public data, like IPFS. We have `alsoKnownAs` as a nice way to include this URI.
@evan ESPECIALLY if it's on something like IPFS, you need signatures, because there's no "see if it's on this instance" to speak of as a trust step!!!!
Am I am losing my mind over here
-
@evan @patmikemid @kopper @hongminhee I'm sorry hold on Evan I'm sorry but it's NOT very low risk. That's a COMPLETE misunderstanding of the information landscape we are currently in.
Trust THEN verify?!?! Trust AND THEN verify?!?!!?!?!?!?
"A random several minutes" until we know whether or not the content delivered authentically is from said actor...
Even ONE minute is enough for someone to read, and believe, something false, and to reply, or to *take action*. Or to boost a post, which is then distributed across the fediverse, and then seen by a bunch of other nodes which also have not yet verified?
Trust AND THEN verify doesn't make sense!!!
AAAAAA I am losing my marbles over this one
@cwebber it's ok if you don't get it. You don't have to use it. There are other ways to handle the thundering herd, like shared caches.
-
@evan ESPECIALLY if it's on something like IPFS, you need signatures, because there's no "see if it's on this instance" to speak of as a trust step!!!!
Am I am losing my mind over here
@cwebber yes.
-
@evan @patmikemid @kopper @hongminhee I'm sorry hold on Evan I'm sorry but it's NOT very low risk. That's a COMPLETE misunderstanding of the information landscape we are currently in.
Trust THEN verify?!?! Trust AND THEN verify?!?!!?!?!?!?
"A random several minutes" until we know whether or not the content delivered authentically is from said actor...
Even ONE minute is enough for someone to read, and believe, something false, and to reply, or to *take action*. Or to boost a post, which is then distributed across the fediverse, and then seen by a bunch of other nodes which also have not yet verified?
Trust AND THEN verify doesn't make sense!!!
AAAAAA I am losing my marbles over this one
@evan @patmikemid @kopper @hongminhee Okay, sorry for blowing up in public, this is a heated issue for me, and something I strongly regret us not just shipping an answer for, and something I have been troubled by for what's now, well, a decade. But I should have taken this to DMs rather than blowing up in public. Mea culpa.
-
@aeva the thundering herd?
@evan @cwebber @patmikemid @kopper @hongminhee the json ld thing
-
@cwebber it's ok if you don't get it. You don't have to use it. There are other ways to handle the thundering herd, like shared caches.
@evan@cosocial.ca on the topic of shared caches as thundering herd mitigation... I spent a little bit of time trying to guard against it, and most of the load is just AP GETs.
Nginx cache was all I needed to handle the thundering herd, although I'll admit I haven't tested it against a good evanp or tchambers boost.
I'll write it all up as a separate topic another day.
-
@evan @cwebber @patmikemid @kopper @hongminhee the json ld thing
-
@evan @cwebber @patmikemid @kopper @hongminhee the json ld thing
@aeva
Premium pot-stirring there -
@trwnh i was replying to a post that wanted all expanded terms.
@evan @trwnh @cwebber @kopper @hongminhee I think it would be great to have everything expanded besides the required as2 context.
The results of the compaction algorithm would change if new things migrate into schema.org, so technically a document could become invalid or break without being modified, but this would be a lot better otherwise I guess. -
@evan @trwnh @cwebber @kopper @hongminhee I think it would be great to have everything expanded besides the required as2 context.
The results of the compaction algorithm would change if new things migrate into schema.org, so technically a document could become invalid or break without being modified, but this would be a lot better otherwise I guess.@gugurumbe @evan @cwebber @kopper @hongminhee yup, using full IRIs also has the advantage that ld-unaware processors only need to recognize 1 form instead of infinitely many.
the thing is, we have semantics imported from the content type (activity+json) which can also change. which is why i think versioning the context document is also important -- it freezes the semantics at the time of publishing, like pinning your dependencies.
without that, we might well have a simpler profile...
-
@gugurumbe @evan @cwebber @kopper @hongminhee yup, using full IRIs also has the advantage that ld-unaware processors only need to recognize 1 form instead of infinitely many.
the thing is, we have semantics imported from the content type (activity+json) which can also change. which is why i think versioning the context document is also important -- it freezes the semantics at the time of publishing, like pinning your dependencies.
without that, we might well have a simpler profile...
@trwnh as a paranoid person, I sometimes wonder what would happen if schema.org received a court order to partially censor its schema in certain regions of the world. Or inject a backdoor key. If it prevents people from sending memes across the geofence, it’s bad.
-
@evan @kopper @hongminhee The problem is that signing json-ld is extremely hard, because effectively you have to turn to the RDF graph normalization algorithm, which has extremely expensive compute times. The lack of signatures means that when I boost peoples' posts, it takes down their instance, since effectively *every* distributed post on the network doesn't actually get accepted as-is, users dial-back to check its contents.
Which, at that point, we might as well not distribute the contents at all when we post to inboxes! We could just publish with the object of the activity being the object's id uri
@cwebber @evan @kopper @hongminhee For the purpose of "message" wouldn't it then be easier to just sign an entire file that is transmitted? Or is this stream processing.
-
@kopper @hongminhee As the person probably most responsible for making sure json-ld stayed in the spec (two reasons: because it was the only extensibility answer we had, and because we were trying hard to retain interoperability with the linked data people, which ultimately did not matter), I agree with you. I do ultimately regret not having a simpler solution than json-ld, especially because it greatly hurt our ability to sign messages, which has considerable effect on the ecosystem.
Mea culpa
![😕](https://forum.wedistribute.org/assets/plugins/nodebb-plugin-emoji/emoji/android/1f615.png?v=8eaae6ed46b ":\")
I do think it's fixable. I'd be interested in joining a conversation about how to fix it.
@cwebber @kopper @hongminhee If we consider data exchange just in one application context (mastodon), then JSON-LD is overhead because data structures are fixed.
But as soon as we go out of that application context, JSON-LD will make a lot of sense. IMHO, the use will grow over time as this allows to add permissions to post on the data level using ODRL e.g. Or to have privacy considerations.
On a very short term, JSON-LD may be an overhead. But it is an investment into the future.
-
@cwebber @kopper @hongminhee If we consider data exchange just in one application context (mastodon), then JSON-LD is overhead because data structures are fixed.
But as soon as we go out of that application context, JSON-LD will make a lot of sense. IMHO, the use will grow over time as this allows to add permissions to post on the data level using ODRL e.g. Or to have privacy considerations.
On a very short term, JSON-LD may be an overhead. But it is an investment into the future.
@rigo @hongminhee @cwebberBut as soon as we go out of that application context, JSON-LD will make a lot of sense.
i can't see it. you can indeed add new fields to json and even namespace them (just expand the namespaces! it even compresses better! (open in your own instance, my instance doesn't show reply trees logged out yet)), without requiring json-ld processing. almost all json parsers will either drop unknown fields by default or have an option to turn that functionality on. json-ld, seriously, brings nothing to the tablethe use will grow over time as this allows to add permissions to post on the data level using ODRL
i seriously doubt this will happen. for permissions the community (not w3c! the implementations, because implementations are who give standards their value!) is more or less converging on docs.gotosocial.org/en/latest/federation/interaction_controls/ (including mastodon's quote post approvals, which is built upon the same framework)But it is an investment into the future.
i don't know about that. i can't really find anyone other than people who have drunk the w3c kool-aid (including you from your bio) who thinks json-ld as allowing any new form of extensibility that Just Namespaces can't accomplish.
no wonder why atproto decided to work with ietf on their standardization efforts. the strictness and validation capability of their lexicons actually make developers lives easy, letting them auto-generate the boring bits, and even with bluesky's own, infamously (?) strict lexicon that doesn't let any microblog >300 characters from being made, there's nothing stopping extensions from being made -
@rigo @hongminhee @cwebber
But as soon as we go out of that application context, JSON-LD will make a lot of sense.
i can't see it. you can indeed add new fields to json and even namespace them (just expand the namespaces! it even compresses better! (open in your own instance, my instance doesn't show reply trees logged out yet)), without requiring json-ld processing. almost all json parsers will either drop unknown fields by default or have an option to turn that functionality on. json-ld, seriously, brings nothing to the tablethe use will grow over time as this allows to add permissions to post on the data level using ODRL
i seriously doubt this will happen. for permissions the community (not w3c! the implementations, because implementations are who give standards their value!) is more or less converging on docs.gotosocial.org/en/latest/federation/interaction_controls/ (including mastodon's quote post approvals, which is built upon the same framework)But it is an investment into the future.
i don't know about that. i can't really find anyone other than people who have drunk the w3c kool-aid (including you from your bio) who thinks json-ld as allowing any new form of extensibility that Just Namespaces can't accomplish.
no wonder why atproto decided to work with ietf on their standardization efforts. the strictness and validation capability of their lexicons actually make developers lives easy, letting them auto-generate the boring bits, and even with bluesky's own, infamously (?) strict lexicon that doesn't let any microblog >300 characters from being made, there's nothing stopping extensions from being made@kopper @hongminhee @cwebber but the JSON parser will not recognize naming conflicts. We are talking about data flows beyond the application context. I see you arguing in a social-media distributed twitter context and not a millimeter beyond. That's ok
Inside that context, JSON-LD remains valid JSON and your implementation can ignore the context. But as soon as you want to annotate stuff and NOT hardcode it into your app, this context becomes important.
