Cagou: XMPP based social network on its way to reach your desktop and Android

goffi 14/09/2016, 20:56 SàT projet libre jabber-xmpp-en

Hi to you,

It's time for a blog post on Cagou development.
Cagou is the new desktop/Android frontend for "Salut à Toi" (SàT), the full-featured social network based on standards (XMPP) and with a strong emphasis on ethics.

This frontend is using the very nice Kivy framework, and is already usable on desktop.

Let's see a video on the current state:

As you can see, the interface can be divided easily. This idea is inspired by the excellent Blender interface, and make it easy to adapt to a small or big screen by choosing the number of widgets accordingly.
Instead of being a discreet software that you'll use only when you want to answer to somebody, Cagou is an application which should be used full-screen, with a layout suiting the current needs.

Other important point: the contact list is not the central element (as it is usually in instant messaging software). You can access it by selecting the widget, or put it on the side to simulate traditional layout, but it is not visible by default and you can spend a whole session without using it.

widget selection

About notifications, it will be slighly different on desktop and on platforms with their own notification system (like Android). In the later case, the native way will be used.
On desktop, a specific header is used. 2 cagou icons (thanks to Muchoslava for its contributions) can appear: one on the left for notes (i.e. short messages which will automatically disappear), and one of the right for more important notifications, needing attention from the user.
A click on the left icon will show the last 10 notes, and one the right one show the next important notification.

This was made to avoid popups: they are displayed only after an interaction of user, and will not appear all of a sudden while you are writting a message.

notification example

Thanks to the architecture of SàT, menus are already available with a couple of useful actions.

There are several ways to start a chat: you can click on an item in the contact list, or enter a jid (XMPP identifier) in a chat widget.
Discussions are not closed when you switch widget, and they can be later accessed by swiping the current one (see video below).

There have been a lot of work done and the interface starts to be really usable. Concerning the desktop the greatest part is already finished, the most important remaining thing is to adapt interface to features (e.g. displaying of delivery reciept) and to make new widgets.

The other big thing to cope with would be displaying of HTML content and of course the Android port. An Ubuntu touch port is an eventuality.

And for those who wonder: Cagou will as well benefit from end-to-end encryption (for now OTR, but OpenPGP is planned for next release, OMEMO is considerated), and other SàT features

The project is still in active progress, but it is really demanding. If you can contribue in any way – code, distributions packages, tests, graphics, translations, etc. – please contact us by any mean shown on our website.

Last but not least, we reminds all of the Aziruka contributors of the crowdfunding campaign, that they can claim their counterpart if not done yet (or request it later).

See you soon :-)

Want a decentralized, encrypted, Free (as in freedom) “social” app on Android and desktop ?

goffi 26/10/2015, 14:03 GNU-Linux jabber-xmpp-en SàT projet

http://ftp.goffi.org/media/pictures/arizuka_android_2015_50%25.png

We have just launched a crowdfunding campaign to develop a new frontend to Libervia, and port it to Android (in a native application!).

http://www.arizuka.com/en/projects/libervia (subtitles available in many languages).

If the campaign is successful, we'll have a unique tool. Here are only a few features managed by Libervia:

  • blogging/microblogging: we have a decentralized blogging engine based on XMPP, no need to create an account/validate an email to post a comment
  • instant messaging, single or multi-user, with a lot of features
  • end to end encryption: we already manage OTR for single chat, there is a good probability that we also implement OMEMO/Axolotl
  • file sharing: being file uploading or P2P transfer
  • group permissions: similarly to what others call “circle” or “aspect”, you can share with only your friends or your family
  • a lot more, check http://salut-a-toi.org/features.html (and don't miss the experimental ones)

In addition we are a non-profit association focusing on ethic, we want to make a good tool, not a place for advertisements!

So if you are looking for next-gen communication tool, please support us and share the link: http://www.arizuka.com/en/projects/libervia

Thanks !

http://salut-a-toi.org/static/images/screenshots/libervia/libervia_overview.png

Libervia (Salut à Toi) 0.6.1: XMPP blogging and more

goffi 14/07/2016, 20:19 jabber-xmpp-en projet GNU-Linux SàT Libre

The 0.6.1 release of Libervia (Salut à Toi) is out, and it is a big one (despite of the minor version number). The one which will follow — and is already well advanced — will be the « general public » release, or in other words we'll do what is necessary to have a release stable enough to be used seriously, and easy enough to install so the large public can enjoy it.

You can have a look at the changelog or the repository for more details, but here are the main changes and fixes:

  • import plugin which can be extended, with "importers" for Dotclear and DokuWiki. Please note that, thanks to the use of standards, this can be useful even if you're not using Libervia to display your blog (it's compatible with Movim for instance)

  • in addition to the Dotclear import, its wiki syntax is available for SàT too, it comes with the already available Markdown and XHTML.

  • automatic subscription to PubSub feeds for new accounts, this avoids an empty pages for newcomers

  • MAM (XEP-0313) is implemented for PubSub, this allows to go back in publication history and to search in a more flexible and powerful way. Note that MAM is not available yet for instant messaging, it's planed for 0.7

  • better detection of repository revision

  • jp (CLI) has now an option --output with who you can specify the output format (e.g. JSON so parsing by an other command is easy).

  • jp: new command to import a blog (import), (re-)edit a new/old publication (edit) or have a preview in real time (preview). We have published a tutorial in french to explain how to publish on your XMPP blog with Vim, Emacs or whatever else, any help to translate to English welcome

  • jp: commands to manage your roster: get it (get), have statistic on it (stats), or clean it (purge)

  • jp: the command "message" has been moved to "message send" to be coherent with other commands, and because a "message get" should come in the future

  • Primitivus (console interface): paste detection (a validation is needed in this case, this avoid sending a bad text like a password by mistake), and handling of "bracketed paste mode"

  • Libervia (web interface): a new option "allow_registration" allows to deactivate the new account registration page, specially useful if you are alone on your instance, or if you create accounts by your own means

  • Libervia: a new pop-up with a counter is displayed if connection with server is lost

  • Libervia: new favicon with a notification counter

  • Libervia: connection with an external jid is fixed

  • Libervia: it is now possible to redirect pages or to integrate a local directory in Libervia. That's used to display directly the blog instead of the login page on www.goffi.org.

  • Libervia: TLS chains are now handled (hello Let's Encrypt)

  • Libervia (blog): tags are now handled, including search

  • we have also updated and improved our Docker images, so you can easily install and test Libervia. Please check the wiki page with the explanations.

This is a summary of 0.6.1, but it is not covering everything!

Salut à Toi is an ethical project is staying away from big proprietary companies, if you feel like helping it contact us by email (contact at salut-a-toi d.o.t org) or on our XMPP MUC room. We need more hands!

The 0.6.1 is the last one before the "general public" one and has its issues to be improved. Don't hesitate to report them so that the 0.7 will be rock solid :)

The 0.7 will go through beta testing phase, and will bring in particular:

  • refactoring of messages, which will be needed to implement a lot of extensions which were blocked until now, like last message correction, or server side messages archives (MAM). This part is already finished, and you can see below an example of an experimental language detection plugin for Primitivus (based on langid):

language detection and filtering on Primitivus

  • Cagou, the new desktop/mobile devices frontend that we have promised after the success on our crowdfunding campain. The development is advanced, a post about it should arrive soon.

  • blog and messaging gateways

Beside all this, there is a good probability that we implement new end 2 end encryption system, in addition to the already available OTR. We specially think about OpenPGP and OMEMO.

Cagou(SàT): development progress of the desktop and Android XMPP frontend

goffi 14/03/2017, 21:56 SàT projet libre GNU-linux jabber-xmpp-en

Salut à Vous !

It's been a little more that one year now that the crowdfunding campain has been successfuly completed, and that we have promised to develop a new Desktop/Android frontend for "Salut à Toi", our Multipurpose, multi frontend communication tool based on XMPP.

It's time for an overview of the state of the development. You'll find below a link to the first .apk (pre-alpha), FOR TESTING PURPOSE ONLY.

Cagou (SàT)

As we've already announced, the new Desktop/mobiles (Android only for now) frontend will be named "Cagou", a wind to the Kivy framework that we're using, and to this endemic bird of New Caledonia, which bark and can't fly.

Oh, and yes we know that "Cagou" has different meaning in different languages.

Development

This part is technical, you can go directly to the next one if you're not interested.

After a short time to get familiar with the Kivy ecosystem, the first step has been to integrate "Quick Frontend", which is a common base that we are using, as you guess, to quickly develop a frontend and factorise the code (cache handling, contact list – or "roster" in XMPP world –, widgets, etc.), then to integrate the "bridge" which is the name that we are using for the IPC and which handle the communication between the backend (which is the heart of SàT) and the frontends.

This phase went relatively well, and was followed by a thought on the architecture and user interface.

Once all of this was more or less usable, the Android port could began.

Things have been a big more complicated there. The Kivy community has created several tools to develop on this platform, including python-for-android (compilation and archive creation tool), and Buildozer (multi-platform tool to facilitate the use of the first one). Using these tools take some efforts, specially for a project already well advanced (it's far more easy if you start directly with Kivy and the Android port).

There a 2 "chains" for developing on Android: the old and the new one. After the first unsuccessful tries with the new one, it has been temporarly put aside for the old one, the time to build the foundations of the port.

Pure Python dependencies can be imported easily, but as soon as it get more complicated, you'll have to creates "recipes" to tell how to build something to python-for-android. Fortunately, most of those needed for SàT (Twisted in particular) were already available, and just needed to be updated.

After all this dependencies and building chain problems solved, and after the joy to see the first .apk (no working), 2 other big troubles showed up: D-Bus which is the main "bridge" is not usable on Android, and how to have the backend and the frontend running together?

Being my first Android development, I've had to read a lot or documentation (which luckily is not lacking), and after a first try with a bridge "embedded", allowing to have backend and frontend in the same process, it's finally a new "pb" bridge which solved the issue. "pb" stands for "Perspective Broker", the Twisted IPC. Android native IPC is an other option to be evaluated in the future.

To launch the backend, Kivy comes with modules to start it as an Android service. With it, the backend can stay in background and process messages and other activities when the frontend is not visible to the user (which means frozen until the user show it again on Android).

This section is already long, so I'll skip other problems (like the lack of HTML widget), and let's now talk about the UI.

UI

At the moment Cagou is usable on desktop (GNU/Linux, but other platforms will most certainly follow), and on Android.

The current release is a pre-alpha, the .apk is available below, only to have a rough idea of the software. It is really bugged, doesn't check yet server certificates, doesn't handle SRV record on Android, etc. This is linked for 2 reasons: showing the progress, and having feedbacks early enought to go in the right direction.

You can't create account from the application (this will come before the release), so if you have no account yet you can use Libervia(SàT), the demo instance of our web frontend, to create one.

main concepts

Cagou's UI take inspiration from the excelent Blender. The widget selection and splitting are the most obvious examples. The huge bars that you can see should disappear later in favor of a small button, which may follow Blender example again. The goal here is that a newcomer can switch widgets intuitively, and an advanced user can use this splitting feature.

widgets split

The contacts list is not the central element in the interface, it can be displayed if wanted, but it's not needed to use Cagou.

The upper menu, which is for now always visible, will certainly be only available on Desktop. On Android the menu button or a floating touch one should replace it before the release.

If you have notifications, they should appear for a couple of second on the top, but you can read them later by touching the cagou head on the upper left.

notification in Cagou

If there is an event needing user action (for instance a website needs your authorisation with XMPP), an other cagou will appear on the right, and the dialog will only appear after touching it. The idea is to never have an unwanted popup stealing the focus when you are doing something else: popups are displayed only with explicit user action.

In Android case, it may be replaced by the native notification system in the future, but it's not decided yet because notifications history would not be available anymore.

To change the mode (widget), you just have to click on the upper left button in the current widget. Only 4 widgets are available so far: the selector which display all widgets available, the settings, the contact list, and the chat. Other are planed, notabily the blogging one.

widget/mode selection

Inside a widget (only for chat so far), you can swipe horizontaly to switch between active conversations.

widget swipe in Cagou

For now it's not easy to use the first time (you have to do a very quick swipe), it needs some tuning.

As for other SàT parts, Cagou is thought since the beginning to work with plugins and being easy to extend. All the widgets and file transmitting system (see below) are plugins.

instant messaging (chat)

As we want a frontend usable on small screens, which is simple but without making sacrifice on features, we have to find a compromise between data displayed on the screen and the elements/buttons needed for actions. Too many elements will make the UI more complicated and take room, but not enough will make the actions more difficult to do.

The current design (which can evolve) has a header with an input bar and a button (in addition to widget selection button), the body with messages, and a bottom input bar with a button too.

To talk with one or more contact(s), type some letters belonging to its name (anywhere in the name). For now, only identifiers ("jid") and already opened talks are looked for, but later the search will include names, nicknames and bookmarks.

contact selection for instant messagin

Cagou detect if you want to talk to a single person, or to a group chat room, and will act consequently.

The end 2 end encryption is there, but only with OTR (v2) for the moment. OX (modern OpenPGP) and OMEMO are planed, but it's not sure that they will be available for the next release (they may be there for the following one, 0.8). To use it, you just have to click on the lock, it will be closed if the chat is encrypted, and will have a question mark if your contact is not authenticated.

Let's go directly to the input bar. You'll see on the right a "+" button, click on it and you'll have a sending dialog.

This dialog has 2 buttons on the top, with which you can choose between uploading the file or sending it directly using peer 2 peer. A text under them explain in simple language where your file will go, and if encryption is done (for now all files are sent unencrypted).

This text message is important to let the user understand where the data are transiting, it's the kind of information we plan to put in several locations.

file sending dialog on Android

The buttons below are the various transmitting options. On desktop, you can only use a file browser (for now), but on Android you can also send a picture from your gallery, take a new photo or a video, or record a voice message.

He is a screenshot of voice recording:

recording a voice message

around Cagou

In addition to the work on Cagou itself, other things have been done.

A now mandatory feature with the explosion of mobile devices, carbon copy has been implemented. Server archives is implemented for long fot blogging, but not yet for chat messages, it will be done before the stable release.

Small binary files handling ("BoB" for "Bits of Binary") is now available. Implentation has been motivated by its use on the friend project Movim.

Delivery receipts and HTTP authentification have been contributed by Chteufleur.

Since the 0.6.1 version, messages handling has been improved, making now possible to implement features like last message correction, planed for the release.

Lastly, component (for gateways) and static blogs have also been implemented, but we'll talk about this later.

help

SàT is a huge project with strong ethical values (check the social contract), and it need help! It can be as easy as coming to talk with us (our room is at sat@chat.jabberfr.org, also availble by clicking here).

If you can, financial aid would of course be useful, we have recently opened an account on the excellent Liberapay.

You can also join us in our association, all informations are available here

And of course contributions are useful, specially development but also translations, icons, CSS, etc. Most of the coding is done in Python, and working on SàT is working on a tool you may use everyday. You'll not find us on big centralised forges, but we are in the process of modernising our development tools (more to come on that).

Talk about our association and project around you is always useful.

I think the most important things have been said, I'll try to keep you up to date with shorted posts.

Oh, and the link with the pre-alpha version (once again: FOR TESTING PURPOSE ONLY): https://www.goffi.org/public/Cagou-0.1-debug.apk

Collective radio (with video)

goffi 02/02/2012, 00:00 GNU-Linux jabber-xmpp-en SàT projet Libre

G'day everybody

after the shell pipe over XMPP, here is a new feature in "Salut à Toi": collective radio.

The idea is: you are in a chat room with a common playlist, and everybody can add a song to the playlist. The song is played simultaneously for everybody, making it a collective radio experiment.

So far, there are maximum 2 songs in queue, as soon as one song is played, anybody can add a new one. In the future, we can imagine a more sophisticated system to choose who can add a song in queue (e.g. each one in turn, with a vote if songs played are appreciated, or with a game: the one who can answer a quiz question can choose next song).

Maybe in the future we'll see thematic rooms: jazz, rock, experimental, discovery, song with nice lyrics, etc. Or maybe we'll spend some evenings with friends, each one adding nice songs after the other, or playing blind test. We can also imagine party were everybody can participate to the playlist.
If in addition you use it on your phone: you are your own radio disc jockey :)

Following is a video showing the current implementation of the proof of concept, it's in french but quite easy to understand.

For the record: I'll be in Brussels next weekend for the FOSDEM, where I'll make a demo of the project at the XSF stand; don't hesitate to come to have a chat :). In addition I'll have a short talk in the devroom on saturday evening (18:00), you can have more information by clicking on this link.

To play the video, you nead a recent browser (e.g. Firefox 4+ or the last Chromium).
You can also use VLC (>=1.1 only), by using this url as a flux: http://www.goffi.org/videos/pr%c3%a9sentation_S%c3%a0T_5_radio_collective.webm

Last but not least, you can use mplayer: mplayer "http://www.goffi.org/videos/pr%c3%a9sentation_S%c3%a0T_5_radio_collective.webm"

This video is licensed under Creative Common BY-SA

Movim Network 16/03/2012, 15:23

Hello ! We need to interconnect with some other social networks, if they wish to do. We really think a social network can't stay alone.

You could discover us on our Wiki http://wiki.movim.eu/ or come to speak with us on our Mailing List (not very active, don't hesitate to post) http://wiki.movim.eu/mailing_list

There is also a MUC XMPP but we aren't always on it

Vincent, french coordinator of communications of the Movim Project

http://movim.eu/


author website

Libervia (Salut à Toi) 0.6.0 : decentralized blog and more

goffi 03/12/2015, 23:26 jabber-xmpp-en SàT

We are pleased to announce the 0.6.0 of Libervia (Salut à Toi).

Salut à Toi is a “Swiss Army knife” of communication : a multipurpose, multi frontend, free (as in freedom) and decentralised communication tool, and Libervia is it's web frontend.

With it you can publish publicly or to a private group, do end-to-end encryption for instant messaging, share files, play games, etc.

For this release there has been a lot of work on blogging/microblogging, and we can now enjoy a decentralized blogging system, with template engine, tags, etc. Being based on XMPP, we can communicate with projects like Movim or Jappix

P2P file sharing has also been improved with the implementation of the Jingle protocol.

A lot of others feature have been worked on, like a Docker image, HTTP upload, visual improvements, etc.

You can see the release announce on https://libervia.org/blog/salut-a-toi, a blog hosted on a Libervia instance.

We are in the last day of our crowdfunding campain, so if you want to help us to port the project on desktop and Android, it is *really* the time !

official website: salut-a-toi.org
demo: libervia.org
crowdfunding: www.arizuka.com/fr/projects/libervia last day !
Feel free to contact us: salut-a-toi.org/community.html

The “Salut à Toi” team

Let's talk XMPP - episode 4 - group conversations

goffi 26/08/2015, 09:41 jabber-xmpp-en technique talk_xmpp Libre

This article was initially published in French, and it got some interest, so we started to translate the whole series in English. If you can read French, you can follow the whole series there: http://www.goffi.org/tag/parlons_xmpp

The translation to English was done thanks to: Poulet , Éfrit , Paco , mbarbarosa , Weyfonk and Chteufleur . The whole series is under CC By-SA , and any help would be appreciated to help with the translation of next articles.
(Follow the corresponding link to read the previous episodes).

In the software development field, and much in the libre software, group’s discussions are really common, mostly with IRC (Internet Relay chat).
This venerable protocol do what we ask it and XMPP is strongly inspired by it. Let’s see that closely.

Group’s discussions used nowadays are called MUC (Multi-User Chat) and are defined by XEP-0045. This one standardizes and extends the first solution called Groupchat. As all that comes from IRC, I’ll explain as they come the major differences between them.

It is possible to acces a chat room located on any server from any server (again, while it is not explicitly prohibited). Chat rooms do, as the users, have a jid, which has the pattern chat_room_name@service. For example, that of Salut à Toi is sat@chat.jabberfr.org:  “sat” is the chat room’s name, “chat.jabberfr.org” the service.

The resource is used for occupants of the chat room: sat@chat.jabberfr.org/goffi corresponds to the occupant “goffi” in the chat room sat@chat.jabberfr.org. Ah little detail that I forgot in previous articles: everything is unicode in XMPP, including jid. So you can use arabic or russian nickname. But beware: some unicode characters are strongly similar, therefore it is possible to get 2 words graphically similar mixed up, we name it “homoglyphs”. For example “gοffⅰ” looks like “goffi” but it uses different characters. This issue is mentioned in a unicode technical report: http://www.unicode.org/reports/tr36/. Also, do not rely exclusively on a nickname to identify someone (espcially that it is possible that it can be reused by someone else between two sessions).

The nickname is linked to the chat room and not to the service. You can have a nick “toto” in a chat room and “titi” in another one, and someone else can have “titi” on a third chat room. This is a big difference with IRC where we have only one nickname on a server that will be used in every chat room (channel on IRC).

To go in or go out a chat room, or to change nickname, we send an available (or not) presence directly to chat_room@example.net/desired_nick, but this is normaly managed by your client.

It is also possible to write directly to all chat room’s occupants (under the hood this is a “groupchat” message that is sent to the chat room’s bare jid), or to have a private chat with a member (we write to the full jid of the recipient).

A chat room can be public or hidden (it will not appear in the chat room list), not anonymous or semi-anonymous (in the first case everyone can see the occupant’s real jid, in the second case only moderators and administrators can), persistent or temporary, open or accessible only by white list or can be protected by password, moderated or not.

All those parameters are normally defined at the creation of the chat room, or they can be modified after with the suitable option of your client (on Gajim: right click on chat room tab => Manage Room => Configure Room). According to the service you use, you can configure more or less things, for example limit the occupant’s maximal number.

A feature often implemented is the history or “back log”: when you get in a chat room, the service sends you the last X messages, allowing you to understand the context of the current conversation.

Also, if a public archive of the chat room is kept (we say “logged” chat room), the service must warn you (it’s mandatory in the XEP), this is another good point compared to IRC. For sure, one has to remember that anyone in the room can keep a log and could publish it without your consent.

So far, so good, but a great strength of IRC is its simplicity: no need to create an account, you just have to pick a nick (unique), with a server, and that’s it! So, you won’t be disapointed, XMPP has exactly the same thing with connections called “anonymous”. No anonymity in terms of Tor here, but rather the possibility to get a temporary account, with a jid more or less random, for the connection time. This comes built-in but it must often be explicitly enabled in the server configuration, and most of the time, anonymous connections are limited to the local network, no communication with other servers (to avoid spam).

If you want to chat as with IRC in a simple and intuitive way, and if you like the console, I would strongly recommend Poezio which is an excellent XMPP client that is easy to use: initially, without changing the configuration, you will be anonymously connected to the MUC service of Poezio. It is inspired by Irssi/Weechat and use the same commands (and more generally those of IRC). Below the welcome message, without changing the configuration, we see the anonymous jid assigned for the session time.

Poezio screenshot

Well this episode is long enough, but I am not finished with MUC, therefore we will talk about it next time, probably with transports.

Let's talk XMPP - episode 3 - core and extensions (part 2)

goffi 05/08/2015, 07:09 jabber-xmpp-en technique talk_xmpp Libre

This article was initially published in French, and it got some interest, so we started to translate the whole series in English. If you can read French, you can follow the whole series there: http://www.goffi.org/tag/parlons_xmpp

The translation to English was done thanks to: Poulet, Éfrit, Paco, mbarbarosa, Weyfonk and Chteufleur. The whole series is under CC By-SA, and any help would be appreciated to help with the translation of next articles.
(Follow the corresponding link to read the previous episodes).
 
Some functionalities can be added to this central part, hence the X of XMPP (which stands for “eXtensible”).
 
Extensions are written using “XEP” (XMPP Extension Protocol) , following an idea taken from Python (if I am not mistaken). This is why the features supported by a server or a client are identified as XEP-0XXX. Obviously, no need to know this to use a XMPP client, but it can be useful to read an extension (you can find them at  https://xmpp.org/xmpp-protocols/xmpp-extensions/) to understand correctly the utility of a functionality. Two parts are particularly useful without the need to enter in the implementation details: the “abstract” part (summary) in the top that indicates what the XEP is doing, and the “introduction” section (the very first section) that provides further details about the cause of the extension and its use cases.
 
XEP-0045 abstract
 
An XEP can describe a feature, a process (for instance the XEP-0001 explains the life cycle of the XEPs themselves), a historical legacy (related to something created before the XMPP Standard Foundation), information (like good practices), even a joke (yes, there are also lousy jokes in the XEPs!). It can have several statuses, detailed in the XEP-0001. It is interesting to note that a lot of XEPs are “experimental” and so technically not (yet) standards, but often implemented anyway. Such XEPs can be widely modified before they get the “Draft” status and eventually become a standard.
 
Why am I talking about all this? For you to get one thing loud and clear: XMPP is not only about instant messaging!
 
Some interesting examples of extensions:
 
  • Extended Stanza Addressing (XEP-0033): to send messages to several recipients at once, or to do carbon copies, as well as blind carbon copies (as the… yes, yes you see what I mean).
  • Multi-User Chat (XEP-0045): IRC-like group chat.
  • Ad-Hoc Commands (XEP-0050): a generic system to handle any kind of commands. Related to users' permissions, it is a great tool!
  • vcard-temp (Business cards, temporary version, XEP-0054): The legacy way to manage electronic business cards, which are a kind of public profile. A new extension will eventually replace it (XEP-0292).
  • Jabber Search (XEP-0055): used to search for jid, mostly used by directories.
  • Publish/Subscribe (XEP-0060): a big piece, used to publish all kinds of information, and its recovery based on permissions, with a notification system in real time.
  • XHTML-IM (XEP-0071): to publish with an XHTML subset, which enables formatting (to write bold text or include a picture for example).
  • Gateway Interaction (XEP-0100): to manage gateways, i.e. links to external networks.
  • Personal Eventing Protocol (XEP-0163): a kind of simplified Publish/Subscribe system.
  • Jingle (XEP-0166): Peer-to-peer session establishment with a wide range of possible applications, the most well-known being video conference.
  • Serverless Messaging (XEP-0174): unsurprisingly used to communicate without servers.
  • Message Archive Management (XEP-0313): to recover messages or other objects (works with Publish/Subscribe as well) following criteria such as a date. In particular, used to get a conversation history on a server, which is useful to access it from all your clients.
 
Keep in mind that we’ve barely scratched the surface here.
 
Further details will be provided about some of these extensions in future articles.
Please note that extensions can be referred by using short names, for instance “MAM” for “Message Archive Management”. Such names are usually mentioned at the end of the related XEPs, in the “Document Information” appendices.
 
This may lead to situations where clients and servers don’t support the same extensions subsets… How to make both parties agree on which one to use? This problem is solved through another extension (though so essential and widely implemented that it could nearly be considered as a main standard): “Service Discovery”, aka XEP-030, short name: “disco”.
 
The idea is simple: each client, server or component discloses who it is, what it can do, or associated items. A component is a service that plugs itself into a server, see below.
 

who it is

An XMPP address (or jid) can be used by all kinds of elements, be it a server, a client, a gateway, or a bot for instance. The latter is a robot, i.e. a program that automates tasks, and is often used to behave like a client.
 
When a client or another element sees a jid, it may need to know what kind of element is associated to that jid, for instance to render it in a special way in the user interface. This is how clients and gateways are shown in distinct ways in your contacts list.
 
Disco’s identity provides this through a category (“client” or “server” for instance), a type (e.g. a client can be “bot”, “web”, “game”…), and a free name (eg “ejabberd”). A list of categories and their associated types can be found on  https://xmpp.org/registrar/disco-categories.html.
 

what it can do

 
XMPP’s extensibility makes it very feature-rich, therefore knowing what the software we are talking to is able to do is mandatory. Disco provides a list of such supported features, which is why you may see disabled buttons or icons (e.g. for video conference) while talking to someone. This means the client you are talking to indicates that it does not support this feature, or more accurately: it does not reveal that it supports it.
 
These features are linked to namespaces that are mentioned in the related XEPs. A list of XEPs based on namespaces is available here: https://xmpp.org/resources/schemas.
 

associated items

 
Beside its identity and available features, an XMPP entity can have associated elements. They can be servers indicating where to find chat rooms, or gateways to other networks.

 

Let's try

Let’s make this clear with an example. Salut à Toi’s command line interface “jp” can retrieve an entity’s disco using the command “jp info disco”, see below with jabber.fr:
 
 
% jp info disco jabber.fr
Features:
 
http://jabber.org/protocol/commands
http://jabber.org/protocol/disco#info
http://jabber.org/protocol/disco#items
http://jabber.org/protocol/stats
iq
jabber:iq:register
jabber:iq:time
jabber:iq:version
msgoffline
presence
presence-invisible
urn:xmpp:time
vcard-temp
--
Identities:
 
ejabberd (server/im)
--
Extensions:
 
http://jabber.org/network/serverinfo
 
--
Items:
 
[chat.jabberfr.org]
[irc.jabberfr.org]
[j2j]
[presence.jabberfr.org]
[proxy.jabberfr.org]
[users.jabberfr.org]
 
 
As we can see, jabber.fr supports legacy vCards management (vCard-temp, XEP-0054) as well as ad-hoc commands (http://jabber.org/protocol/commands, XEP-0050), and the server (server/im) used for instant messaging is called “ejabberd”. We can also notice that various services are available, one of them being char.jabberfr.org. Let’s take a closer look:
 
% jp info disco chat.jabberfr.org
Features:
 
http://jabber.org/protocol/disco
http://jabber.org/protocol/muc
http://jabber.org/protocol/muc#unique
jabber:iq:browse
jabber:iq:last
jabber:iq:register
jabber:iq:time
jabber:iq:version
urn:xmpp:ping
vcard-temp
--
Identities:
 
Public Chatrooms (conference/text)
--
Items:
 
[...]
JabberFR (13) [jabberfr@chat.jabberfr.org]
[...]
 
 
We see that we are dealing with a chat rooms service (conference/text), which use the protocol “Multi-User Chat” (http://jabber.org/protocol/muc). It is at the moment the only one which is available for groups’ discussions (we will talk about it later). “Items” elements contain rooms’ list, with the name, the occupiers’ number with the corresponding JID. I shorten the list which was very long.
 
In jabber.fr’s information, you may have noticed the “extensions” section, it is the XEP-0128 which allow to extend disco for all kind of information. In the case of ejabberd here, it is to give contact addresses of server, but they aren’t indicated for jabber.fr
 
Below the disco’s window of Gajim:
 
Gajim disco
The first time I used XMPP, with Psi in the early 2000s, I was a bit intimidated by the “service discovery” menu, which shows near directly all information we just saw. This kind of menu is often, at my opinion, showed too abruptly in XMPP clients: using disco directly (that is all except what is automated by the client) is already advanced use.
 
 
Furthermore, I will quickly mention the “software version” extension (XEP-0092) which allows, as indicated by its name, to know to which software we are talking to, and the operating system used. jp allows to show these information with “jp info version”, let’s try on jabber.fr:
 
% jp info version jabber.fr
Client name: ejabberd
Client version: 2.1.13
Operating System: unix/linux 3.2.0
 
We now know the ejabberd version that is used. Useful when we know if a feature is supported or if a bug is fixed. And this work with every entity which implement the XEP, not only servers.
 
% jp info version chat.jabberfr.org
Client name: MU-Conference
Client version: 0.9-svn (Jan 27 2014)
Operating System: Linux 3.2.0-4-amd64
 
That’s it. Knowing about the extensions allows to truly know what we can do with a client or a server. Next time I think I will explain groups’ discussions and see what changed in relation to IRC.

Let's talk XMPP - episode 2 - core and extensions

goffi 30/07/2015, 12:18 jabber-xmpp-en technique talk_xmpp Libre

This article was initially published in French, and it got some interest, so we started to translate the whole series in English. If you can read French, you can follow the whole series there: http://www.goffi.org/tag/parlons_xmpp

The translation to English was done thanks to: Poulet, Éfrit, Paco and mbarbarosa and Weyfonk. The whole series is under CC By-SA, and any help would be appreciated to help with the translation of next articles.

Now that we know what we’re talking about, let’s see what the core of the protocol looks like.
 
Originally, XMPP is defined by 3 (previously 2) RFCs: RFC 6120, RFC 6121, and RFC 6122 (there are others, but these 3 are the main ones). They explain the core of the system, such as sending messages, presence information, statuses and so on.
 
Without getting too much into details related to developers, we can quickly explain that XMPP is based on 3 element types, or “stanzas”:
  • <presence/> to primarily indicate… our presence information (sometimes, we also add other things like our avatars, our capabilities, but let’s stay focused). The presence is broadcast by the server to all the people that you have given permission to (see below).
    We can associate a state and a message to our presence. The state can be one of the following (names can change depending on the client):
    • available (default): you’re online.
    • away: you’re away for a short period
    • chat: you want to talk
    • dnd (do not disturb): you’re occupied (often called “busy” in clients)
    • xa (eXtended Away): you’re away for a long while.
    The status messages allow you to specify your availability in a clear language (for example: “I’m watching a movie, do not disturb”), even though, in practice, it is used for any kind of message (a lot of people use a quote, for example).
  • <message/> A message’s sending of type “send and forget”. There are 5 types of messages::
    • chat: the most well-known message, used for simple instant messaging;
    • error: this one is usually managed directly by client software. It is often shown by means of a notification window in your client, indicating that something wrong occurred;
    • groupchat: as “chat”, but for discussion with multiple people. In practice the difference concerns only developers and it should be transparent in the client;
    • headline: an important message, an announcement. Normally, these messages aren’t kept offline, so if you aren’t connected while the message is sent, you shouldn’t receive it. These messages don’t expect answers. A typical example is an announcement for an imminent server maintenance;
    • normal: a little known yet interesting type. It is a message which generally has a subject, but outside of an instant conversation. Yes, exactly like an email! It is more or less its XMPP equivalent.
    Conversation threads are managed as well, but we’ll talk about it another time.
  • <IQ/> (Info/Query): used for everything that works on a question/answer pattern (it is mandatory to provide an answer to an request, even if you need to reply with an error). This is used mainly by developers, as it serves as a basis for most of the features you need. Using it is like saying "I need to know or edit this information".
 
I don't want to delve into technical details, but it seems essential for a user to know different message and presence types to understand their client.
 
One should note that there is an excellent feature that is largely underused in XMPP: it natively knows how to handle different languages because of its inheritance from XML (xml:lang). In other words, you can specify a normal or status message in French, German and Slovak at the same time. This is a major asset that we intend to use in Libervia.
 
Now let's talk about another essential part: the contact list.
 
It is called "roster" in the XMPP world. You can assign 0, 1 or several groups ("family", "friends", etc.), a name (which is an alias chosen by yourself, not by your contact), and subscription information to every contact you add to your roster.
 
Subscription lets you know whether you have allowed your contact to see your presence information and whether you are allowed to see theirs. Therefore, every time someone adds you to their contact list, your client (e.g. Gajim) asks you whether you wish to grant them access to your presence information. Note that these permissions don't need to be symmetrical: a contact may have been allowed to see your contact information without enabling you to see theirs, and vice versa. It might even be possible that neither of you can see each other's presence information (but I think most clients remove the contact from the roster in that case).
 
Groups are dependent on contacts, not the other way around (it's not a list of groups which contains the contacts): this explains why having an empty group (i.e. without any contact) isn't possible. In my opinion, groups too are underused in the XMPP world, but we'll get back to this.
 
 
That's it for today. I opted to keep the part on extensions for the next post to keep this one lighter.

Let's talk XMPP - episode 1 - the basics

goffi 29/07/2015, 09:42 jabber-xmpp-en technique talk_xmpp Libre

This article was initially published in French, and it got some interest, so we started to translate the whole series in English. If you can read French, you can follow the whole series there: http://www.goffi.org/tag/parlons_xmpp


The translation to English was done thanks to: Poulet, Éfrit, Paco and mbarbarosa. The whole series is under CC By-SA, and any help would be appreciated to help with the translation of next articles.

G'day,
 
Well, as it’s a shame that XMPP is not well known or understood, I decided to start a series of articles to explain what it is.
 
These articles are for a technical audience, but not necessarily for developers, and I hope it will help you to understand the advantages of this protocol and to use your software in a better way.
 
Being a developer of the “Salut à Toi” project, I’ll probably give examples with it quite often.
 
So let’s start with basis.
 
What’s XMPP? It’s a standard message and presence protocol, and extensible (XMPP means « eXtensible Message and Presence Protocol », that is it’s documented and used as a reference (validated by standards organization: the IETF). It allows all software which use it to speak the same language, and so to be interoperable. It’s a libre protocol, that is you can get the documentation and use it for free, without any legal or technical restrictions, and you can improve or modify it (but if you divert and don’t suggest your modifications, you risk to lose the compatibility with other software).
 
XMPP is decentralized and federated, that is you can have servers anywhere, they could (if they don’t forbid it explicitly in their configuration) communicate with each other.
 
It’s a popular protocol, many software allow to use it: for servers, there are Prosody, Ejabberd, Openfire, Tigase, Mongooselm, Metronome, etc. For clients: Gajim, Poezio, Pidgin, Psi, Swift, Jappix, Movim and of course Libervia/Salut à Toi (I let you find the official websites by yourself). A complete list (servers, clients and libraries) is available here: https://xmpp.org/xmpp-software/.
 
If you cannot install your server at home, many public ones are available: in France one can cite those of APINC ((jabber.fr, im.apinc.org, etc), of the Quadrature du Net, etc. A small list (it’s not up-to-date, don’t hesitate to contribute) is available here in French,, if not you can check on https://xmpp.net/directory.php or http://www.jabberes.org/servers (yes, there are a lot!). But I strongly suggest you to install your own server or to go meet a local organization that you can contact easily: on the one hand if you install it by yourself you’ll have a better control on your own data and on the other hand if you want a specific feature you’ll be able to ask admins for an update.
 
Knowing all that, let’s try to install an account.
 
Once the server installed or a public server chosen, you can create an account. You’ll have then an address, a “JID” (Jabber ID, “Jabber” is the former name of the protocol, this name is now owned by a private company and is kept here for historical reasons).
 
This address is on the form of local_name@domain.tld/resource, or in canonical form (or what is called “bare JID”) local_name@domain.tld. For example, mine is goffi@jabber.fr. Does it sound familiar? Yes, it really looks like email addresses!
 
So what is this resource? The resource is linked with the client software that you use to connect: XMPP has been designed from the beginning to allow several clients to connect at once (10 years ago, a few messaging protocols were able to do that, and connecting again from a different place was often resulting in the disconnection of the first client) and this resource is used to identify them. In other words, you have a resource only once connected, and it will be a different one for each client that you are using: if I connect with Libervia, Gajim and Movim, I’ll have 3 different resources.
 
It used to be several strategies to name the resource, sometimes it was used to show the connection location (“home”, “work”), clients often used their names as default value (“gajim”, “psi”). Today, it’s well accepted that it’s better to have a resource that we cannot guess, as somebody can know if you are online or not (even if you don’t want this information to be public) by doing a request to this resource. The best way is to let the server choose the resource for you.
 
Finally, the resource is linked with a priority: it allows to indicate, if several resources are available, which one will get the message. But we’ll see that later.
 
In the next episode, I'll talk about extensions an features discovery
 
That’s it. Let me know if you are interested in this series, if it seems too technical for you, or if you have any comment or fix to suggest. Everything is under the license CC By-SA, so don't hesitate to re-use, share or modify !