I have deeply mixed feelings about #ActivityPub's adoption of JSON-LD, as someone who's spent way too long dealing with it while building #Fedify.
-
@mariusor @hongminhee The
@contextis not supposed to be required in the first place, but here we are adding it to every activity and wasting bandwidth because Mastodon developers didn't read the spec.@silverpill I'm sorry, I'm not aware of that and I thought I read the specs pretty thoroughly. Could you point me in the right direction for where you got this information from?
-
@hongminhee I have the same feeling. The idea behind JSON-LD is nice, but it isn't widely available, so developing with it becomes a headache: do I want to create a JSON-LD processor, spending twice the time I wanted to, or do I just consider it as JSON for now and hope someone will make a JSON-LD processor soon? Often, the answer is the latter, because it's a big task that we're not looking for when creating fedi software.
-
@silverpill I'm sorry, I'm not aware of that and I thought I read the specs pretty thoroughly. Could you point me in the right direction for where you got this information from?
@contextis a recommendation, not a requirement.ActivityPub:
ActivityPub
The ActivityPub protocol is a decentralized social networking protocol based upon the [ActivityStreams] 2.0 data format. It provides a client to server API for creating, updating and deleting content, as well as a federated server to server API for delivering notifications and content.
(www.w3.org)
Implementers SHOULD include the ActivityPub context in their object definitions.
ActivityStreams:
Implementations producing Activity Streams 2.0 documents SHOULD include a @context property with a value that includes a reference to the normative Activity Streams 2.0 JSON-LD @context definition using the URL " https://www.w3.org/ns/activitystreams".
-
@contextis a recommendation, not a requirement.ActivityPub:
ActivityPub
The ActivityPub protocol is a decentralized social networking protocol based upon the [ActivityStreams] 2.0 data format. It provides a client to server API for creating, updating and deleting content, as well as a federated server to server API for delivering notifications and content.
(www.w3.org)
Implementers SHOULD include the ActivityPub context in their object definitions.
ActivityStreams:
Implementations producing Activity Streams 2.0 documents SHOULD include a @context property with a value that includes a reference to the normative Activity Streams 2.0 JSON-LD @context definition using the URL " https://www.w3.org/ns/activitystreams".
@silverpill aaah, I see. I think we've had this discussion before (or at least I had it with someone else).
For me "SHOULD" falls in the category of the robustness principle: "be conservative in what you do, be liberal in what you accept from others".
So for me if you treat "SHOULD" in a spec as non mandatory you haven't really implemented the spec.
-
@silverpill aaah, I see. I think we've had this discussion before (or at least I had it with someone else).
For me "SHOULD" falls in the category of the robustness principle: "be conservative in what you do, be liberal in what you accept from others".
So for me if you treat "SHOULD" in a spec as non mandatory you haven't really implemented the spec.
@mariusor I don't remember having such discussion. The SHOULD keyword is defined in RFC-2119:
This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.
There are many valid reasons to not include
@context. We also have almost 10 years of implementation experience and by now full implications are very well understood: by ignoring this recommendation we make messages smaller and developer experience better. No downside at all. -
@mariusor I don't remember having such discussion. The SHOULD keyword is defined in RFC-2119:
This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.
There are many valid reasons to not include
@context. We also have almost 10 years of implementation experience and by now full implications are very well understood: by ignoring this recommendation we make messages smaller and developer experience better. No downside at all.@silverpill regarding size, ActivityPub is such a verbose protocol that the hundred or so of raw bytes you save through omitting context, are most likely negligible through the prism of connection compression. So to me that's not entirely a "valid reason".
And as developer myself, I think that contexts, even in a non valid JSON-LD implementation, offer enough guidance for building a data vocabulary for them to have plenty of value.
Do you propose we replace contexts with Open API specifications, or how do we coordinate what's a valid vocabulary data object in a federated network? And how do you propose that others discover these specs?
-
@pintoch read this thread?
-
@silverpill regarding size, ActivityPub is such a verbose protocol that the hundred or so of raw bytes you save through omitting context, are most likely negligible through the prism of connection compression. So to me that's not entirely a "valid reason".
And as developer myself, I think that contexts, even in a non valid JSON-LD implementation, offer enough guidance for building a data vocabulary for them to have plenty of value.
Do you propose we replace contexts with Open API specifications, or how do we coordinate what's a valid vocabulary data object in a federated network? And how do you propose that others discover these specs?
@silverpill personally I feel like the various activity/object signing methods that get used in recent FEPs are more egregious from a size point of view, when the in spec behaviour for obtaining canonical versions of a resource is to fetch them from their server, instead of relying on random object signing that introduces so much more friction.
-
@silverpill personally I feel like the various activity/object signing methods that get used in recent FEPs are more egregious from a size point of view, when the in spec behaviour for obtaining canonical versions of a resource is to fetch them from their server, instead of relying on random object signing that introduces so much more friction.
@mariusor@metalhead.club I thought the whole point of signing objects or attaching proofs (none of which I do, mind you) are precisely to save the need to make a new request, which comes with its own overhead.
The good thing is fetching from canonical source will never go out of style.
Aside, it seems like I'm only getting Marius's posts, not silverpills. Makes for an interesting one-sided exchange
-
@mariusor@metalhead.club I thought the whole point of signing objects or attaching proofs (none of which I do, mind you) are precisely to save the need to make a new request, which comes with its own overhead.
The good thing is fetching from canonical source will never go out of style.
Aside, it seems like I'm only getting Marius's posts, not silverpills. Makes for an interesting one-sided exchange
@julian I noticed that your inbox endpoint returns 404s (my activities are delivered to personal inbox, not shared).
-
@hongminhee from the point of view of someone who is "maintaining" a JSON-LD processing fedi software and has implemented their own JSON-LD processing library (which is, to my knowledge, the fastest in it's programming language), JSON-LD is pure overhead. there is nothing it allows for that can't be done with
1. making fields which take multiple values explicit
2. always using namespaces and letting HTTP compression take care of minimizing the transfer
without JSON-LD, fedi software could use zero-ish-copy deserialization for a majority of their objects (when strings aren't escaped) through tools like serde_json and Cow<str>, or System.Text.Json.JsonDocument. JSON-LD processing effectively mandates a JSON node DOM (in the algorithms standardized, you may be able to get rid of it with Clever Programming)
additionally, due to JSON-LD 1.1 features like @type:@json, you can not even fetch contexts in parallel, meaning all JSON-LD code has to be async (in the languages which has the concept), potentially losing out on significant optimizations that can't be done in coroutines due to various reasons (e.g. C# async methods can't have ref structs, Rust async functions usually require thread safety due to tokio's prevalence, even if they're ran in a single-threaded runtime)
this is after context processing introducing network dependency to the deserialization of data, wasting time and data on non-server cases (e.g. activitypub C2S). sure you can cache individual contexts, but then the context can change underneath you, desynchronizing your cached context and, in the worst case, opening you up to security vulnerabilities
json-ld is not my favorite part of this protocol -
@hongminhee from the point of view of someone who is "maintaining" a JSON-LD processing fedi software and has implemented their own JSON-LD processing library (which is, to my knowledge, the fastest in it's programming language), JSON-LD is pure overhead. there is nothing it allows for that can't be done with
1. making fields which take multiple values explicit
2. always using namespaces and letting HTTP compression take care of minimizing the transfer
without JSON-LD, fedi software could use zero-ish-copy deserialization for a majority of their objects (when strings aren't escaped) through tools like serde_json and Cow<str>, or System.Text.Json.JsonDocument. JSON-LD processing effectively mandates a JSON node DOM (in the algorithms standardized, you may be able to get rid of it with Clever Programming)
additionally, due to JSON-LD 1.1 features like @type:@json, you can not even fetch contexts in parallel, meaning all JSON-LD code has to be async (in the languages which has the concept), potentially losing out on significant optimizations that can't be done in coroutines due to various reasons (e.g. C# async methods can't have ref structs, Rust async functions usually require thread safety due to tokio's prevalence, even if they're ran in a single-threaded runtime)
this is after context processing introducing network dependency to the deserialization of data, wasting time and data on non-server cases (e.g. activitypub C2S). sure you can cache individual contexts, but then the context can change underneath you, desynchronizing your cached context and, in the worst case, opening you up to security vulnerabilities
json-ld is not my favorite part of this protocol@hongminhee take this part with a grain of salt because my benchmarks for it are with dotNetRdf which is the slowest C# implementation i know of (hence my replacement library), but JSON-LD is slower than RSA validation, which is one of the pain points around authorized fetch scalability
wetdry.world/@kopper/114678924693500011 -
reposting so @julian sees this
"I noticed that your inbox endpoint returns 404s (my activities are delivered to personal inbox, not shared)." says @silverpill
-
@hongminhee take this part with a grain of salt because my benchmarks for it are with dotNetRdf which is the slowest C# implementation i know of (hence my replacement library), but JSON-LD is slower than RSA validation, which is one of the pain points around authorized fetch scalability
wetdry.world/@kopper/114678924693500011@kopper @hongminhee I'm glad I'm not the only one who noticed this.
-
@hongminhee take this part with a grain of salt because my benchmarks for it are with dotNetRdf which is the slowest C# implementation i know of (hence my replacement library), but JSON-LD is slower than RSA validation, which is one of the pain points around authorized fetch scalability
wetdry.world/@kopper/114678924693500011@hongminhee if i can give one piece of advice to devs who want to process JSON-LD: dont bother compacting. you already know the schema you output (or you're just passing through what the user gives and it doesn't matter to you), serialize directly to the compacted representation, and only run expansion on incoming data
expansion is the cheapest JSON-LD operation (since all other operations depend on it and run it internally anyhow), and this will get you all the compatibility benefits of JSON-LD with little downsides (beyond more annoying deserialization code, as you have to map the expanded representation to your internal structure which will likely be modeled after the compacted one) -
@julian I don't know as much as I'd like about AT Lexicons. That is, not so much how they work, but what the grand idea is? I don't even understand if Bluesky imagines them being mixed and matched JSON-LD style. I think not?
-
Manu, maker of JSON-LD who also helped with the AP Confs, made this nice video https://www.youtube.com/watch?v=vioCbTo3C-4
JSON-LD is a normative reference to ActivityPub. The context of AP is only 1 line, maybe 4 if you support the official extensions. It does not make anything much larger.
It is for example important if you want to consume the federated wikipedia, wikidata, European Broadcasting Union or these Public Broadcasters https://www.publicmediaalliance.org/public-broadcasters-create-public-spaces-incubator/ but also to know that e.g. mobilizon uses schema.org for addresses.
I give you an example, if you include
"mz": "https://joinmobilizon.org/ns#", "wd": "https://www.wikidata.org/wiki/Special:EntityData/",
"wdt": "https://www.wikidata.org/prop/direct/"in your context, then you know about mobilizon extension but also the whole common knowledge of the world …
I like that, now you can support the whole vocabulary of wikipedia and wikidata which is just JSON-LD.
You get it in all the languages of the world including the properties name.
No problem, if others don't support it, but sad for users. -
Manu, maker of JSON-LD who also helped with the AP Confs, made this nice video https://www.youtube.com/watch?v=vioCbTo3C-4
JSON-LD is a normative reference to ActivityPub. The context of AP is only 1 line, maybe 4 if you support the official extensions. It does not make anything much larger.
It is for example important if you want to consume the federated wikipedia, wikidata, European Broadcasting Union or these Public Broadcasters https://www.publicmediaalliance.org/public-broadcasters-create-public-spaces-incubator/ but also to know that e.g. mobilizon uses schema.org for addresses.
I give you an example, if you include
"mz": "https://joinmobilizon.org/ns#", "wd": "https://www.wikidata.org/wiki/Special:EntityData/",
"wdt": "https://www.wikidata.org/prop/direct/"in your context, then you know about mobilizon extension but also the whole common knowledge of the world …
I like that, now you can support the whole vocabulary of wikipedia and wikidata which is just JSON-LD.
You get it in all the languages of the world including the properties name.
No problem, if others don't support it, but sad for users.PS, I am using the official JSON-LD processor of Manu and contributors, if support in any language is lacking, we just speak to the JSON-LD Group (glad about the 2 webintents coming together now as well ) …
Cause we are social … -
@hongminhee if i can give one piece of advice to devs who want to process JSON-LD: dont bother compacting. you already know the schema you output (or you're just passing through what the user gives and it doesn't matter to you), serialize directly to the compacted representation, and only run expansion on incoming data
expansion is the cheapest JSON-LD operation (since all other operations depend on it and run it internally anyhow), and this will get you all the compatibility benefits of JSON-LD with little downsides (beyond more annoying deserialization code, as you have to map the expanded representation to your internal structure which will likely be modeled after the compacted one)@kopper@not-brain.d.on-t.work @hongminhee@hollo.social expansion is actually really annoying because the resulting JSON has annoyingly similar keys to lookup in a hashmap, wasting cache lines, and CPU time
-
reposting so @julian sees this
"I noticed that your inbox endpoint returns 404s (my activities are delivered to personal inbox, not shared)." says @silverpill
@liaizon@social.wake.st actually, oddly, I did receive @silverpill@mitra.social's response, so it seems to be I can access some replies from Mitra, but not all.
