Libervia progress note 2022-W45

goffi 6 days ago jabber-xmpp-en SàT Libervia project libre Libervia progress SàT progress XMPP

Hello, it's time for a long overdue progress note.

I'll talk here about the work made on ActivityPub (AP) gateway and on end-to-end encryption around pubsub.

Oh, and if everything goes well, this blog post should be accessible from XMPP and ActivityPub (and HTTP and ATOM feed), using the same identifier goffi@goffi.org.

Forewords

The work made on the AP gateway has been possible thanks to a NLnet/NGI0 grant (with financial support from the European Commission's Next Generation Internet programme).

I especially appreciated that the team was really there to help bring the ideas to life, and not once did they get in the way: little paperwork, no unnecessary pressure, caring, contacts when help was needed, etc.

I wish there were more organizations like this one that really help develop libre projects for the common good.

So once again I want to thank them for all that.

XMPP ⬌ ActivityPub Gateway

There is probably no need to explain here what is ActivityPub, we can simply write that it is an open protocol that allows to do things that XMPP also allows doing, and that until now these 2 protocols could not communicate together. The work on the ActivityPub gateway aims to allow software implementing one of these 2 protocols to communicate as easily as possible. I firmly believe that all open protocols should be able to communicate which each other, to avoid creating more silos, proprietary software is already good enough at that.

To be useful, a gateway must use the full potential of both protocols. A simple bot transcribing messages as we see too often, using unsuitable features (such as instant messaging for blog posts), or using a very limited set of features to ensure compatibility are flaws that I have tried to avoid. Building a good gateway is a difficult and time-consuming task. If done right, the gateway should be as invisible as possible to the end user.

XMPP is featuring blogging since long before AP, however the set of features is not exactly the same. Current use of AP is clearly inspired from commercial "social" networks, and metadata such as subscribers/subscribed nodes (or followers/following in AP terms) are highlighted, feature such as like/favourite were missing in XMPP, and some implementation such as Pleroma do implement reactions. To integrate that in the gateway, I've been working on new specifications:

  • Pubsub Public Subscriptions: a way to publicly announce subscriptions, in an opt-in way. With this it's possible to implement followers/following features in a way respectful of privacy.

  • Pubsub Attachments: a generic way to attach any kind of data to a pubsub item. It's notably used to implements noticed/favourite button (see here and reactions.

  • Calendar Events: handling of events and all the RSVP mechanism. Libervia was handling events for years, but it was an experimental implementation, this specification is a next step in the effort to make it a standard.

Note that this XEP and the others linked below have been accepted but are not yet visible in official list.

You may wonder why there is a specification for Calendar Events… It's because the AP gateway also handles them, making it compatible with Mobilizon. The gateway may evolve in the future to support other non (micro)blogging use cases.

The gateway is now finished in terms of functionalities, however the code is clearly of an alpha quality for the moment. Now the goal in the coming months will be to stabilize and possibly implement other features if there is a demand for it.

Early adopters are encouraged to try and test it as long as they keep in mind that it's not stable. So if you do try it, I recommend keeping a separate ActivityPub account in whatever stable implementation you use at the moment, this way you can check if messages or media are missing, if there is any inconsistency or other bugs, and report them to me. If you test it, please join the XMPP room libervia@chat.jabberfr.org (click here to connect from your browser) for help and feedback. Stabilization will probably take weeks, but I hope to have it done by early 2023.

Installation instructions and details on how the conversion between protocols is done is available in the documentation and notably here

A question I've been asked a lot: yes, you can use the same identifier for XMPP (JID) and AP (WebFinger actor handle) as long as you use "simple" characters (i.e. alphanumeric ASCII chars, _, . and -). If you use something more complicated, you'll have to use the escaping mechanism explained in the doc (this is due to constraints with some AP implementations).

As for blogs on pubsub nodes (what Movim calls "communities"), I made it simple: you can use directly the name of the node that holds the blog in the local part (i.e. before the "@") of your actor handle: a blog named community_bog at the XMPP pubsub service pubsub.example.org can thus be addressed with the AP actor community_blog@pubsub.example.org. This way you can use a rather user-friendly identifier to share your blog with people who are only on ActivityPub.

This gateway should work with any XMPP server, and any client that implement blogging features (Only Libervia itself and Movim implement it for now, but I have heard that other clients are planning support for it). To enjoy the whole feature set of the gateway, the new specifications need to be implemented by the clients, so you can start to fill feature requests…

With this gateway, the door is open to have a client able to talk to the ActivityPub network, while having the feature of XMPP, including e2e encrypted private messages (e2e encrypted only if you communicate with an XMPP account, not with an AP one).

Oh, and please update your graphics, drawing and other texts to include XMPP in the fediverse ;)

End-to-End Encryption

Much effort has also gone into end-to-end encryption.

OMEMO implementation has been updated (OMEMO:2 is now used), including Stanza Content Encryption which allows encrypting arbitrary elements instead of only the \<body/> of the message, I believe that Libervia is the first XMPP client to implement it. OpenPGP for XMPP (or "OX") has also been implemented, all that thanks to the work of Tim Henkes "Syndace", the author of python-omemo.

Beside instant messaging, end-to-end encryption has also been introduced to pubsub. I've made specifications for two methods:

  • An OpenPGP profile for pubsub which is thought to encrypt a whole node, with a system of secret sharing/rotation/revocation. With it, it is easy to give access to new entities after publication, and to retrieve old items for newcomers. This specification can be used to encrypt any pubsub based features: (micro)blogging, calendar events, lists, etc.

  • Pubsub Targeted Encryption which is a way to apply the same cryptographic system used in instant messaging to pubsub. This way, OMEMO can be used with its forward secrecy property. It is not a good option to use this specification to encrypt a whole node, as archive is then not accessible to newcomers, and to add access to a new entity you have to re-encrypt all items, but it's an interesting option to encrypt an element occasionally, for instance to restrict access of a specific post in an otherwise public blog.

Specifications have also been written to sign a pubsub item in a backward compatible way (client which don't implement those specifications can still work normally):

All those specifications are already implemented in Libervia, but they are only usable from CLI frontend at the moment. All you have to do is to use the --encrypt and/or --sign options from pubsub or blog commands (check documentation for details).

Uploaded files were already encrypted with OMEMO Media Sharing which is what is commonly used these days, but this method has not been accepted as a standard as it was a workaround for limitation of legacy OMEMO implementation. The proper way is now specified with Stateless File Sharing and is encrypted with Encryption For Stateless File Sharing. Those methods are currently only usable when OMEMO:2 is implemented in the peer client, and with them metadata on the shared file can be attached, including thumbnails.

Encryption has also been implemented for Jingle (XEP-0391 and XEP-0396), which is notably used for Jingle File Transfer (specially useful for large files transfers).

So to summarize, nearly everything (instant messaging, files uploaded, large file transfers, all pubsub related features) can now be e2e encrypted with Libervia.

Possible Future

With the AP gateway permitting to reach the whole AP network, all the new features implemented, and the work done on e2e encryption, Libervia has everything to be a solid option for communication. After the recent events regarding a famous commercial network, we see a breakthrough of ActivityPub that will hopefully last over time. We can now access AP from XMPP, while having the possibility to have e2e encrypted private conversations or even blogs or calendar events.

As far as I know this is, so far, something unique for a Libre decentralized software. However, there is still work to do on stabilization on UI/UX update before this is really usable.

Those feature were planned for very long (years), but the lack of resources made them slow to come. The grant has made it possible to greatly accelerate the pace of development, and I doubt that it would have been possible to have all that without it.

Regarding how large the project is, and my family life, it's not possible any more to develop seriously this project on my free time alone (and I would like to do other things, sometimes, of my free time).

In other words, I need to find a way to sustain the development of Libervia for the years to come, so I can work full-time on it, and with some luck, build a team. I'm thinking very seriously about it these days, I'll probably write on this topic in a little while. If you are willing to help in any way, please contact me (on the Libervia room linked above for instance).

That's all for this progress note. I'm now working on stabilization and UI/UX update on the web frontend.

E

edhelas 6 days ago

blue 6 days ago

blue 6 days ago

This is awesome! Thank you so much for your work!

debacle 6 days ago

debacle 6 days ago

Great work, goffi!

M

matlag 5 days ago

J

jkdaemon 5 days ago

ericbuijs 5 days ago

L

yvelia 4 days ago

L

lohang_toofast 2 days ago

This is great work. Delighted to see that you are making progress with it! :)

marzanna 2 days ago