At the risk of sounding like an unhinged fanboy: XMPP is pretty awesome !

I’ve been involved in one way or another with XMPP, the network protocol that is an open standard for messaging and presence, for the last two decades. Much of that revolves around development of Openfire , our XMPP-based real-time communications server.

TL;DR:

• I built a thing: https://xmppnetwork.goodbytes.im
• XMPP is pretty awesome (as stated above)!
• Openfire is a great development platform!

Decentralisation with XMPP

There’s much to say about what I like about XMPP, but let me focus on one thing in this text: decentralisation. Not only the protocol, but the entire XMPP ecosystem - with all its different server and client implementations - is based on the principle of federation. This allows anyone to set up their own solution for instant messaging, voice and/or video conferencing, data sharing, and much, much more. All of this is done without creating a dependency on any central entity or organisation. At the same time, you’ve created a solution that allows you to communicate with others that are not part of your own domain.

Some of the benefits of decentralisation are obvious: you get to control your own data. When you’re not sharing data with one monolithic IM solution provider, then there’s a lot less to worry about with regards to their privacy policies, marketing strategies, cookie policies and data security.

Another benefit of using a decentralised approach is diversity. I know of at least seven or eight different XMPP server implementations - all of which are mature projects that have a proven track-record of interoperability for many, many years. Each of the server implementations have their own strengths. Some favour raw network performance, others offer a more complete social media feature set. Some focus on being active in the Small and Medium-sized Enterprises segment, others try to cater to family & friends type of communities. Some are open source, others are commercial products. There are products that offer a turn-key, no-config-needed full blown instant messaging solution, while others can act as a development platform that is useful when you’re looking to develop your own networking application. As you might be able to tell from all this, diversity gives you the option to select the best software suite for your needs.

I digress. XMPP federation is based on server-to-server connections. Whenever a user on one domain starts to interact with a user on another domain, servers of both domains will connect to each other in the background to facilitate this interaction. As you might imagine, when enough users start to interact with each other, this leads to interesting webs of interconnected domains.

Reviving an old idea: creating a network graph!

Over the last holiday season, I remembered an old, now defunct project by Thomas Leister, that generated a visual representation of interconnected servers. Its visuals were pretty amazing. I remember Tom’s solution to be based on an out-of-band exchange of data (through a simple webservice), and recalled his desire to replace this with a solution that used XMPP’s own protocol and functionality. His stated goal was to use XMPP’s Publish/Subscribe functionality , but never seemed to have been able to get around to implementing that. I had some spare time over the holidays and challenged myself to build this new approach. I started work on a new version of that project, aiming to build a web application that renders a semi-live network graph of XMPP domains with their connections to other XMPP domains.

The path from prototype to a fully working solution was an interesting one, involving a couple of different aspects of development within Openfire, but also the XMPP community as a whole.

Using Openfire as a development platform

Perhaps unsurprisingly, but I love working with Openfire. It’s so incredibly versatile when it comes to adding new features and functionality.

For this new project, I needed a couple things.

1. An API to add functionality to Openfire. Check. Openfire’s Plugin API gives you structured access to pretty much everything in Openfire. It’s easy to use, yet very versatile.
2. A web server. Check. Openfire ships with Eclipse Jetty , an embedded web server. It’s used for Openfire’s Administration Console, but can just as easily be instantiated to serve different content. Openfire’s BOSH connectivity makes use of this, but also plugins like the Random Avatar Generator and Presence Service that expose user-specific, as well as the inVerse and JSXC plugins, that each serve a full-fledged web based XMPP client.
3. A Publish-Subscribe service. Check. Openfire out of the box implements XMPP’s pub-sub standard.
4. Database storage. Check. Openfire ships with support for most popular relational databases. Crucially, it allows a plugin to define and manage its own database structures.
5. A javascript graphing engine. From a quick search, various applicable projects popped up. I opted to go with vis.js , for no other reason than that it was the first thing that popped up, looked reasonably mature and had easy-to-follow documentation. I later added VivaGraph , which offers WebGL support. Turns out that if you render thousands of nodes in a network, CPUs tend to get busy. Who knew? WebGL helped make things more efficient.
6. Basic HTML and CSS design skills. I am many things, a good designer is not one of them.

My first prototype wrapped all of this into a solution that:

• Periodically iterated over all server-to-server connections
• Stored all information in a simple data structure
• Persisted the data structure in the database
• Created a web service that exposes the data as ‘nodes’ and ‘edges’ to be consumed by the graphing software
• Have a simple, static webpage that consumes that webservice, and renders the graph using the third-party graphing engine.

In all, I was pretty proud to have been able to write all this in a single evening!

The approach above gave me a nice hub-and-spoke graph, where my server was the hub, showing spokes to every connected remote domain.

To be able to install this on more than one domain, I separated the plugin into two:

1. One plugin that aggregates the connectivity data, to be installed on all servers on the network
2. Another one that generates the website installed only on the server that acts as the public interface to the website.

I’ve used the XMPP’s Publish-Subscribe feature to bridge the gap between the two plugins. After some quick modifications, the first plugin creates a pub-sub node on the local pub-sub service, to which the second plugin subscribes. The second plugin then aggregates all of the data in its database, and uses that to populate the webservice, just as before.

Using this mechanism, it is pretty straight-forward to have many servers feeding one website. With a bit more work, I was even able to write a quick crawler, that tries to find pub-sub nodes with similar information on all of the XMPP domains that are reported as being remotely-linked domains, which removed the need to have every server sign up to the website manually.

Finally, I paid a hoster a little bit of extra money to have a new server to host a new Openfire server that would act as the public website, going through the motions of having a domain name and corresponding TLS certificate. Having done this before, I automated most of that, allowing me to create a new Openfire server from scratch in about ten minutes. I manually installed the new plugin, installed a reverse proxy to serve web content on standard ports, and, presto! The XMPP Network Graph suddenly became a publicly available service!

Some of the community members at IgniteRealtime.org were happy to install my plugin, which quickly contributed to the network graph growing.

Working with the XMPP community

To be able to grow the XMPP network graph, it is desirable to have support added to more server implementations than just the one for Openfire. As luck would have it, the XMPP ecosystem, as stated above, thrives on diversity.

To allow for a great deal of extensibility and flexibility, and to optimise interoperability, the XMPP Standards Foundation manages a pretty nifty process for extending the XMPP, through documents aptly named XMPP Extension Protocols (XEPs). The full process is documented in the very first XEP-0001 . Have a read, if you’re interested.

The standardised way to get XMPP entities to interoperate on a bit of functionality is simple:

• Write a XEP to document the functionality
• Submit the XEP to the XSF for review and publication
• Incentivise others the adopt the XEP

I did just that , and found the added value of this process to be unexpectedly high.

A submitted XEP makes for a convenient discussion subject. My original document quickly drew feedback .

Although I was aware of Thomas’ implementation, others apparently also toyed with creating network graphs of the XMPP network. Seems that I’m even further from having had an original idea than what I expected.

The feedback from the XMPP community showed the expertise and experience that lies within that community. Several technical issues were discussed, which led to improvements of the protocol. Probably the most important bit of feedback that was given related to privacy concerns, which we discussed at length.

The XMPP ecosystem consists of servers of all sizes. There are various XMPP service providers that each have many thousands of users. There are also plenty of servers that are running for family and friends, or even a single user. It is these servers that were the subject of the privacy concern.

If a connection is visible between two of these small servers, it becomes reasonably evident that two specific individuals are communicating with each-other. If both individuals agree to have this information published, then there’s no privacy concern - but what if only one individual does so? If John makes public that they’re connecting to Jane, then the fact that Jane is communicating with John is implicitly made public too. If other friends of Jane (Jack, Jill and Johan) similarly publish all their connections, then determining who Jane’s friends are becomes pretty straightforward - without Jane having consented to any data publication.

This, rightly, got flagged in early feedback from XSF members. We’ve discussed the impact of this problem, the need to address it, and various strategies to resolve the issue. We ended up with a solution that allows any server to publicise their connections, but require them to automatically verify that their peer opts-in to having a connection to their server be identifiable (those that do not show up as anonymous blips in the graph).

Based on the feedback, this and other improvements were quickly made to the XEP and my Openfire implementation. Now that there was a stable-ish protocol definition, it became easy for compatible implementations to be created for other XMPP servers. To date, there are implementations for Tigase , Prosody and ejabberd - and there’s mine for Openfire , of course. Not a bad score, after only a few weeks of development!

Wrapping up.

My XMPP Network Graph project has been maturing nicely in the last few weeks, as you can see from the screenshot above. You can have a look at and interact with the network graph at xmppnetwork.goodbytes.im . At the time of writing, it contains over 6,600 domains. It is pretty powerful to see how many people are interacting over XMPP, and that only in the small part of the network that is being mapped by the graph!

You can now add your own XMPP server to the graph! The plugin that I created for Openfire can be found here . Plugins or modules are available for other XMPP servers too. Have a look at the FAQ section of the XMPP Network Graph for instructions on how to add your server to the network graph!

I’ve enjoyed the process of setting all this up. Having most of the development pieces already in place, as mentioned above, allowed for rapid development. To me this is a testament to the power of not only Openfire as a development platform but also XMPP as the versatile Swiss Army knife of network protocols.

I’d love to learn what you make of this! Do you have success stories of your own to share? I’d like to hear from you!

For other release announcements and news follow us on Mastodon or X

1 post - 1 participant

Read full topic

The HTTP File Upload plugin is a plugin for Openfire that allow users to easily share files (such as pictures) in a chat

A new release is now available for this plugin: version 1.4.0.

This release introduces a couple of interesting security improvements:

• an additional guard has been added that should prevent scripts embedded in data to be executed without the recipient’s approval
• It is now possible to configure a virus scanner that will process all uploaded content.

Configuration details are available on the plugin’s archive page , and in its readme file.

The update should be visible in the Plugins section of your Openfire admin console within the next few days. You can also download it from the plugin’s archive page .

For other release announcements and news follow us on Mastodon or X

1 post - 1 participant

Read full topic

The Presence Service plugin is a plugin for Openfire. It provides a service that provides simple presence information over HTTP. It can be used to display an online status icon for a user or component on a web page or to poll for presence information from a web service.

A new release is now available for this plugin: version 1.7.2.

In this release, an incompatibility with the recently released Openfire 4.8.0 was fixed. Also, a reportedly infrequent issue with loading images has been addressed.

The update should be visible in the Plugins section of your Openfire admin console within the next few days. You can also download it from the plugin’s archive page .

For other release announcements and news follow us on Mastodon or X

1 post - 1 participant

Read full topic

🎉 It’s here! We’re happy to introduce the January 2024 Snikket Server release.

This is the core software of the Snikket project - a self-hostable “personal messaging server in a box”. If you wish for something like Messenger, WhatsApp or Signal, but not using their servers, Snikket is for you. Once deployed, you can create invitation links for family, friends, colleagues… any kind of social group is a good fit for Snikket. The invitation links walk people through downloading the Snikket app and joining your private Snikket instance.

What’s new in this release?

Changes to Circles

While Snikket is designed for groups of people to easily communicate with each other, we know that often people have multiple social groups. Our Circles feature allows the admin of the Snikket instance to decide which people will see each other within the Snikket apps, by grouping them into “circles”. For example, you could use this to separate your family from your friends, even within the same Snikket instance.

In previous releases, the Snikket server automatically created a group chat, and added everyone in the circle to that chat automatically. We received a lot of feedback that these chats were either not really used, or sometimes confusing (for example, because they are managed automatically by the server and you cannot manage them yourself within the Snikket app). Other people liked the group chats, but wished that more than one could be made!

In this new release, creating a circle will no longer create a group chat automatically. However you can also now create as many “circle chats” as you want, and give them individual names. This can be useful for creating per-topic chats for all members of a circle.

Of course if you just want normal private group chats, you can still create those within the Snikket app as usual, and manage the group yourself.

Last activity display

Sometimes people drop off Snikket, intentionally or unintentionally. For example, if they get a new phone and forget to reinstall the app or have problems connecting. In the web interface you can now see when the user was last active.

You can use this information to clean up unused accounts, or reach out to people who might need help regaining access to their account.

Connectivity and security

We have made a number of connectivity improvements. Snikket now enables IPv6 by default (previously it had to be enabled manually). If you don’t have IPv6, that’s fine… thanks to new changes we have made, Snikket will now adapt automatically to network conditions and connect using the best method that works. We expect IPv6-only networks to become increasingly common in the years ahead, so if your server is not currently set up for IPv6, consider doing that.

The new release now also supports DNSSEC and DANE 🔒, both of these are used to improve connection security. Currently these are disabled by default, however, because Snikket does not know if your system’s DNS resolver actually supports DNSSEC. We may enable it automatically in future releases if Snikket can determine that reliably. For now, it’s opt-in .

Faster and stronger authentication

We’ve also been working on optimizing and strengthening app-to-server authentication. A lot of this work was funded by NGI0+NLnet and is available in our sister project, Prosody. You can read more details in the blog post Bringing FASTer authentication to Prosody and XMPP .

Snikket already supported a neat security measure called “channel binding”, but it previously only worked over TLS 1.2 connections. TLS 1.3 usage has increased rapidly in recent years, and we now support channel binding on TLS 1.3 connections too. Channel binding prevents machine-in-the-middle attacks if the TLS certificate is compromised somehow.

All these features help protect against certain kinds of attack that were deemed unlikely until recently .

Dropping older security protocols

Mainly for compatibility reasons, Snikket previously supported an authentication mechanism where the client sends the user’s password to the server, but only over TLS-encrypted connections. This is how almost all website login forms work today, from your webmail to your online banking. However the Snikket apps actually use a more secure login method , which has many additional security features that you won’t find on most other online services.

Prioritizing security over compatibility, we have decided to disable less secure mechanisms entirely. If you use your Snikket account with third-party XMPP apps, bots or utilities that are not up to date with modern best practices, this may affect you.

Similarly, we have again reviewed and updated the TLS versions and ciphers that Snikket supports, in line with Mozilla’s recommendations , as we do in every release. This change also has the potential to affect connectivity from some very old apps and devices.

Easy account restoration

The Snikket apps, as well as many third-party apps, allow people to delete their Snikket account from within the app.

However, as the number of Snikket users has grown, so have reports from people who accidentally deleted their account! This can be due to confusion - e.g. intending to remove the account from the app, rather than removing it from the server. A number of these cases were due to confusing or buggy third-party apps. It doesn’t happen very often, but it was happening too often.

Of course, deleted accounts can be restored from backups (which you have, of course 😇) - but it was a complex time-consuming process to selectively restore a single account without rolling back everyone else’s data.

In this release, when a request is received from an app to delete a user’s account, the server will lock the account and schedule its deletion in 7 days (or whatever the server’s data retention time is set to). During this time, the account can be restored easily from the web interface if it turns out to have been a mistake.

Farewell to the welcome message

In previous releases, new accounts would receive an auto-generated “welcome message” from the server. This had a number of issues , and we have decided to remove it for now. Instead we will work on integrating any “welcome” functionality directly into the apps.

Languages and translations

Many languages received updates in this release, including French, German, Indonesian, Polish, Italian and Swedish.

We added support for two additional languages: Russian and Ukranian.

Many thanks to all translators for their help with this effort!

Other changes

We only listed a handful of the main features here. The reality is that beneath the hood, we have made a large number of changes to improve security, performance and reliability. And we have in place the foundations for other exciting things we have in the pipeline!

Installing and upgrading

Choose your adventure:

• If you’re new to Snikket, you can head straight to the setup guide for instructions on how to get started.

• To upgrade an existing self-hosted instance to the new release, read the upgrading guide .

• Customers on our hosting platform can expect the new release to be rolled out soon, we’ll be in touch! If you have any questions, you can contact support .

Happy chatting!

P.S. If you’re planning to be at FOSDEM in a few weeks, we’ll be there, come and say hi! We’d love to meet you :)

The XMPP Standards Foundation (XSF) will hold its 26th XMPP Summit in Brussels, Belgium this year again! These are the two days preceding FOSDEM 2024 . The XSF invites everyone interested in development of the XMPP protocol to attend, and discuss all things XMPP in person and remote!

Time & Address

The venue will take place at the Thon Hotel EU including coffee break (from 08:30 o’clock) and lunch (12:00 to 14:00 o’clock) paid by the XSF in the hotel restaurant.

Date: Thursday 1st - Friday 2nd February 2024 Time: 09:00 - 17:00 o’clock (CET) (both days)

Thon Hotel EU
Room: FRANCE
Wetstraat / Rue de la Loi 75
1040 Brussels
Openstreetmap

Furthermore, the XSF will have its dinner Thursday night which is paid for XSF members. Everyone else is of course invited to participate, however at their own expense. Please reach out if you are participating as a non-member (see list below).

Participation

So that we can make final arrangements with the hotel, you must register before Monday 15th January 2024!

Please note that, although we welcome everyone to join, you must announce your attendance beforehand, as the venue is not publicly accessible. If you’re interested in attending, please make yourself known by filling out your details on the wiki page for Summit 26 . To edit the page, reach out to an XSF member to enter and update your details or you’ll need a wiki account, which we’ll happily provide for you. Reach out in the XSF public chatroom . When you sign-up please book the accomodation and your travel. Please also unsign if you will not attend anymore.

Please also consider signing up if you plan to:

• present in the Lightning talks or Show-and-Tell
• remote participate

Communication

To ensure you receive all the relevant information, updates and announcements about the event, make sure that you’re signed up to the Summit mailing list and the Summit chatroom .

Spread the word also via our communication channels such as Mastodon and Twitter .

Sponsors

We would like to kindly thank the direct sponsors for the event which are so far Isode , Snikket and the two individuals of Alexander Gnauck and Edward Maurer. We appreciate their support so that we can keep the event open and accessible for everyone.

We are really excited seeing so many people already signing up. Looking forward to meeting all of you!

The XMPP Standards Foundation

Today marks the 25th birthday of Jeremie Miller’s announcement of “a new project to create a complete open-source platform for Instant Messaging” on Slashdot.

How have things progressed since then!

By far most of the projects that we maintain here in the IgniteRealtime.org community make direct use of the XMPP protocol, which is the name used for the IETF standards based on the Jabber technology, and we’re still going strong.

With countless different people and organisations creating and using XMPP applications, even today, it has truly proven itself to be a rock-sold, tried and tested, versatile protocol. It’s not often that so much development happens around a technology that’s older than … well, some of us!

Happy birthday, Jabber!

1 post - 1 participant

Read full topic

We’re very excited to be back at FOSDEM in person in 2024. Once again, many members of the XSF and the XMPP community will be attending, and we hope to see you there!

Realtime Lounge

As usual, we will host the Realtime Lounge , where you can come and meet community members, project developers, see demos and ask us questions. We’ll be in our traditional location - find us on the K building 2nd floor, beside the elevator (map below). Come and say Hi! Yes, we got stickers :-)

Map of the K building level 2

Talks

There are talks in the Real Time Communications devroom that relate to XMPP. These are so far:

• Bridging Open Protocols: XMPP and ActivityPub Gateway via Libervia . In this session, we’ll explore the architecture of this gateway, detailing how it facilitates communication between XMPP and ActivityPub. We’ll delve into the intricacies of protocol mapping and discuss how Libervia integrates features such as microblogging, reactions, likes/favorites, mentions, and calendar events across these platforms.

XMPP Summit 26

Prior to FOSDEM, the XSF will also hold its 26th XMPP summit . This is where community members gather to discuss protocol changes and exchange within the developer community. We’ll be reporting live from the event and also from FOSDEM.

Spread the word

Please share the news on other networks:

• Mastodon
• Twitter
• YouTube
• LinkedIn
• Lemmy instance (unofficial)
• Reddit (unofficial)

Automate all the Things

During the past year, the team behind the XMPP Providers project worked on automating the process of gathering data about XMPP providers. Automating this process reduces manual work significantly (for example, checking websites by hand, verifying information, listing sources, etc.) and helps to sustain the team’s efforts. Automation also enables the project to be up to date – every day!

Last month , the project reached a state that allowed the suite of tools to automatically query many provider properties via XMPP and HTTP. All of these tools are working together in a GitLab pipeline running daily to keep the data up to date.

API v2

Much of the work needed to be done manually previously. After automating it, some provider properties did not seem fitting anymore. Thus, we changed them. While automating the process, additional properties were added because they were available through the tools.

Changed Properties

• lastCheck has been replaced by latestUpdate specifying when at least one provider property changed since checks now run daily.
• company has been replaced by organization allowing for a finer distinction of an organization’s type.

New Properties

• alternativeJids : A list of JIDs a provider offers for registration other than the main JID.
• serverTesting : Whether tests against the provider’s server are allowed (e.g., certificate checks and uptime monitoring).
• inBandRegistrationEmailAddressRequired : Whether an email address is required for registering an account.
• inBandRegistrationCaptchaRequired : Whether a CAPTCHA is needed to be solved for registering an account.

The FAQ section explains how these properties can be provided by server admins.

Provider Files for More Automation

There are properties that should be provided by the XMPP server instead of retrieving them via other methods. To enable automatic collection of those properties via XMPP, the team works on extending existing standards and, if necessary, creating new ones.

Until standards have been extended or created, and until those changes have been implemented and deployed in the wild, a provider file shall fill the gap. A provider file is a JSON file containing only the provider properties that cannot be retrieved via other methods. Each provider can generate a provider file and supply it via its web server.

To make this as easy as possible, a Provider File Generator has been developed. It generates a provider file from the information you enter in the form.

As soon as a provider file is discovered by the tools, all properties listed in the provider file are automatically fetched and processed.

Spread the Word

The project lives from the community and client implementations, so follow us and spread the word !

XMPP Providers Logo

A few months ago, we published details about an important security vulnerability in Openfire that is identified as CVE-2023-32315.

To summarize: Openfire’s administrative console (the Admin Console), a web-based application, was found to be vulnerable to a path traversal attack via the setup environment. This permitted an unauthenticated user to access restricted pages in the Openfire Admin Console reserved for administrative users.

Leveraging this, a malicious actor can gain access to all of Openfire, and, by extension (through installing custom plugins), much of the infrastructure that is used to run Openfire. The Ignite Realtime community has made available new Openfire releases in which the issue is addressed, and published various mitigation strategies for those who cannot immediately apply an update. Details can be found in the security advisory that we released back in May.

In the last few days, this issue has seen a considerable increase in exposure: there have been numerous articles and podcasts that discuss the vulnerability. Many of these seem to refer back to a recent blogpost by Jacob Banes at Vulncheck.com , and those that do not seem to include very similar content.

Many of these articles point out that there’s a “new way” to exploit the vulnerability. We indeed see that there are various methods being used, in the wild, in which this vulnerability is abused. Some of these methods leave less traces than others, but the level of access that can be obtained through each of each of these methods is pretty similar (and, sadly, similarly severe).

Given the renewed attention, we’d like to make clear that there is no new vulnerability in Openfire. The issue, solutions and mitigations that are documented in the original security advisory are still accurate and up to date.

Malicous actors use a significant amount of automation. By now, it’s almost safe to assume that your instance has been compromised if you’re running an unpatched instance of Openfire that has its administrative console exposed to the unrestricted internet. Tell-tale signs are high CPU loads (of crypto-miners being installed) and the appearance of new plugins (which carry the malicious code), but this by no means is true for every system that’s compromised.

We continue to urge everyone to update Openfire to its last release, and carefully review the security advisory that we released back in May, to apply applicable mitigations where possible.

For other release announcements and news follow us on Twitter and Mastodon .

1 post - 1 participant

Read full topic