Prosody 0.12.0 released

ÄNTLIGEN! It’s finally here! After 3 years of development and through some chaotic times, Prosody 0.12.0 is released!

What’s the significance of this release? Like many software projects, Prosody follows a “branch” development/release model. We frequently make minor releases with bug fixes and improvements from our stable branch, while we implement more adventurous changes in our development branch, ready for the next major release.

Well, this is one of those adventurous major releases! Specifically, the first release in the 0.12 series, which will now be our new stable branch.

We can’t wait to share the many exciting features and changes with everyone. With over 2400 commits, this release couldn’t have been achieved without all the help from our many contributors, testers and other community members - you know who you are!

---

Before continuing with the release announcement, we want to acknowledge that some people are facing bigger problems right now. We deeply believe that communication is a superior path to resolving disputes than armed conflict, we are saddened by recent developments in Europe and unreservedly condemn such violence.

We hope that, by building decentralized open communication tools without barriers, we can use our particular skills to contribute positive change in the world in what way we best can. We encourage all our community, including you, to reflect on what you can do, or keep doing, to contribute positive change - no matter how small it seems.

---

XMPP Compliance

It makes sense to start off by looking at how the new release progresses Prosody with regards to compliance and interoperability.

Every year, the XMPP Standards Foundation publishes updated Compliance Suites, which provide the evolving guidance about what features and XEPs modern XMPP software should be implementing.

We’re glad to report that Prosody 0.12 meets the criteria for “Advanced Server”, “Advanced IM”, “Advanced Web”, “Core Mobile” and”Advanced A/V” according to the latest (2022) XMPP Compliance Suites (XEP-0459).

What does this mean for you? It means that when you install Prosody, you can be sure of the latest XMPP features and great compatibility with the latest XMPP software.

If you are upgrading from a previous release, do see the release notes which contain some advice about ensuring your server and configuration is up-to-date with all the new changes.

Mobile and connectivity optimizations

In 0.11 we introduced support for mobile connection optimizations that help preserve bandwidth and battery power of mobile devices, using the CSI protocol framework.

In this new release, we are finally adding official support for another piece of this puzzle - fast reconnects using XEP-0198. This functionality has long been available in the community modules repository, in mod_smacks. Originally written by Matthew Wild, with substantial improvements in recent years from Kim Alvefur and, more recently, Thilo Molitor, the module has a long history.

With the module finally transitioning into Prosody, we have streamlined the module’s code (as we no longer need compatibility with multiple Prosody versions) and improved efficiency - especially regarding memory usage.

HTTP file sharing

If you remember a time on XMPP before the HTTP upload file transfer method came along - you know what an improvement it has been. The first file transfer methods in XMPP (and indeed most other IM protocols at the time) would always attempt to stream the file directly from sender to their contact. As well as being extremely prone to failure due to NATs/firewalls and other network issues, it also made it impossible to send files to people who were offline, or to groups.

A new mechanism was standardized in XEP-0363 that solves all these issues by building upon an existing standard protocol that is already great at transferring files - HTTP! With this new method, the sender uploads the file to a server, and simply sends the URL to their contact(s).

To allow easy deployment, Prosody now includes mod_http_file_share. It allows authenticated users to upload files to the server, so they can share them with their contacts.

This new module is quite comprehensive. Some features include:

• Configurable file size limit
• Optional per-user daily upload quota, to ensure fair use of resources
• Global quota, to prevent excessive disk space consumption on the server by uploaded files.
• Support for an external upload server
• Configurable expiry to clean up old uploads
• Monitoring of disk usage

As well as the features listed above, mod_http_file_share also takes advantage of internal HTTP server improvements in Prosody 0.12 that allow it to efficiently handle large files. Say goodbye to that 10 MB limit!

Upgrading from other HTTP upload modules

Many Prosody deployments already support HTTP file transfer using community modules such as mod_http_upload and mod_http_upload_external. These will continue to work but, especially if you are using mod_http_upload, we encourage you to upgrade to enjoy the new improvements.

To migrate existing files from an existing mod_http_upload setup, check out the handy mod_migrate_http_upload.

Audio/video calling support

Audio and video calls on XMPP have really taken off in the past few years! Although we’ve had support for calling in XMPP for a long time, modern clients are building atop the WebRTC stack for more efficient, reliable and secure encrypted calls (contrary to what you may think, WebRTC can be used outside browsers too). A few of the clients supporting these include Conversations, Dino, Movim, Beagle IM and Siskin IM.

Prosody itself is not especially involved in the actual calls - it mainly relays the call negotiation between the clients, and assists in the discovery of STUN and TURN services that are used to improve call reliability through NATs and firewalls.

To facilitate this last part, a server operator needs to do two things:

1. Set up a STUN and TURN server (usually provided by the same software - eturnal and coturn are popular choices)
2. Configure Prosody to advertise these services to your users’ clients.

The second step has traditionally been achieved using a community module, mod_turncredentials. From Prosody 0.12 we now bundle a simpler (but just as effective) module, mod_turn_external.

We have even implemented a simple STUN/TURN client into prosodyctl so it can verify that your setup is working correctly.

To get started, see our full overview of setting up STUN/TURN with Prosody.

Monitoring with OpenMetrics/Prometheus

We’ve supported various options for monitoring Prosody for quite a while in various forms, including internal statistics gathering and output to statsd-compatible collectors and agents.

Although statsd has served us well, and is still a good choice for certain environments, Prometheus has emerged as a very popular alternative. So much so, Prometheus’s protocol has been reworked as an open standard - OpenMetrics, with ongoing work to publish it as an IETF RFC.

Starting from Prosody 0.12, anything using our internal metrics API can now be exposed in a Prometheus-compatible OpenMetrics HTTP endpoint.

To see how, head over to the mod_http_openmetrics documentation.

Many thanks to Jonas for leading and implementing this feature, and providing this screenshot of it in use at search.jabber.network.

Invitations!

It’s no secret that we’re big fans of invitation-based registration. As explained in our blog post, Great Invitations, we believe it is the safest and most user-friendly way to let people register on your XMPP server.

This release brings core support for invitations into Prosody.

Specifically, the following modules are now bundled with Prosody:

• mod_invites provides core invitation handling, including the ability to create new invitations via prosodyctl
• mod_invites_register allows people to register on your server by invitation, from XMPP clients that support XEP-0445 (which includes Conversations, yaxim, Siskin and Snikket).
• mod_invites_adhoc allows admins (and optionally users) to generate invitations from within their XMPP client. With native support in Conversations, yaxim and Snikket, you can also manually use ad-hoc command support in supporting clients such as Gajim.

For the full experience, consider also setting up the following community modules:

• mod_invites_page to turn your invitations into a welcoming, user-friendly web page
• mod_invites_register_web to allow someone to register without leaving their browser
• mod_invites_api provides a HTTP API that allows you to integrate the invitation mechanism with anything you want!

The best place to get started with invitations is the mod_invites documentation.

Direct TLS

Direct TLS (XEP-0368) is now officially supported, along with SNI to allow more than one certificate per port, as well as reloading those certificates without a restart. The legacy_ssl_ports configuration option has thus been renamed to c2s_direct_tls_ports. The prosodyctl check dns command can also help you make sure the corresponding SRV records are set up correctly.

Server-to-server connections also support Direct TLS, enabled with s2s_direct_tls_ports and is automatically used with remote servers that have published the corresponding DNS record.

What are the benefits of direct TLS over the more traditional “STARTTLS” connections?

• Faster connection time (fewer network round-trips)
• Compatibility with TLS middleware such as load balancers and proxies
• Simpler implementation for clients.
• Improved traversal of restrictive firewalls, e.g. by running XMPP over port 443, the port usually used for HTTPS (typically not blocked).

If you’re curious how to support XMPP connections over port 443 alongside a web server, check out the XEP-0368 tech page on the XMPP wiki.

Other new modules

A number of other new modules have been added in this release that are worth mentioning.

mod_s2s_bidi

Enables using a single connection instead of two between servers, reducing resource usage and latency.

mod_auth_ldap

Previously a community module, we have imported this into Prosody for out-of-the-box LDAP support. This replaces our Cyrus SASL support, which has been moved into the community modules repository.

mod_cron

More and more modules have periodic tasks, e.g. to expire old messages or files, this module aims to coordinate them.

mod_mimicking

This module prevents registration of addresses that look very similar to previously registered addresses, e.g. “zash” vs “zаsh” (that’s Cyrillic a, U+0430).

This is useful for public servers, and helps protect against impersonation attempts that utilize Unicode homoglyphs.

mod_tombstones

Another recommending module for public servers, this one prevents re-registration of deleted accounts. It ensures that a deleted account cannot be re-registered by someone else in order to gain access to resources the account had access to - e.g. chat rooms where the account is still listed as a member.

mod_bookmarks

XMPP has evolved through a few ways to store “bookmarks” (essentially the list of group chats that a user is joined to).

With clients supporting different mechanisms, and attempting to keep them in sync, this module supports the latest recommendations and provides compatibility layers between three different ways to store group chat bookmarks, ensuring open group chats are always synchronized between users’ devices, whatever client they are using.

We encourage client developers to upgrade to XEP-0402: PEP Native Bookmarks for the best experience. Eventually the older mechanisms will be retired as they are no longer needed.

Plugin Installer

Our extensive range of community modules, ranging from the silly mod_swedishchef to the amazingly powerful mod_firewall, are almost a defining feature of Prosody.

Thankfully it’s now easier than ever to install community modules thanks to the work of João Duarte, a student who participated in Google Summer of Code and implemented a plugin installer for Prosody. It uses LuaRocks, which needs to be installed on your system.

For example, maybe you want to add a nice web client to your XMPP server? There’s a module for that. Simply install mod_conversejs by running prosodyctl install mod_conversejs and then enable it!

More documentation about this new feature can be found in the plugin installer documentation.

Built-in HTTP server improvements

As we mentioned earlier, Prosody’s built-in HTTP server can now efficiently handle large uploads - writing files directly to disk instead of consuming more and more memory until completion.

Other things have also been improved. You should never have to worry about CORS again! CORS is a security feature of the modern web. However, most HTTP services in Prosody already have some sort of authentication and generally don’t benefit from CORS, which mostly just causes problems for people setting up XMPP web clients. From 0.12 they will work by default (via BOSH and/or websockets).

The built-in HTTP server also has better support for reverse proxies, and will understand it when HTTPS is handled by the proxy.

Evolving the Prosody admin shell

Prosody has had The Telnet Console for a long time. This is an incredibly powerful administrative interface. However, it is also far from perfect. One of the biggest issues is the lack of authentication. Although Prosody ensures it is only accessible to localhost by default, that does not make it safe on any system with untrusted users.

To improve this, the console itself can now be accessed via a UNIX socket, which is secured using file system permissions. This admin socket functionality is internally quite powerful - it’s actually running a pseudo-XMPP stream, over which multiple kinds of data can flow. Expect to see some fun stuff building on top of this in the future!

Get started with sudo prosodyctl shell!

Tip: You can also now run simple commands directly from the command-line, useful for scripting. E.g. rather than connecting to run 'module:list(), you can now simply run: prosodyctl shell module list.

Future support for 0.11

If you are currently running 0.11.x and not in a hurry to upgrade, you may be wondering what to expect now that 0.12 is here.

Our traditional policy is to support release branches that are included in Debian, for as long as that version of Debian is supported by Debian’s security team (note: this does not include the extended “Debian LTS” period, which is a sponsored effort by a separate team). We will continue with this policy for 0.12.

The latest version of Debian, 11 (“bullseye”), supplies Prosody 0.11. Although the pace of 0.11 releases will undoubtedly slow down now, you can expect us to continue with security and important bug fixes for a couple of years. That should give everyone plenty of time to evaluate 0.12 and plan their upgrade (right?!).

Next steps

Hopefully we’ve shown you enough of 0.12 to get you interested! Head to the Prosody downloads page to find out how to get the new release or upgrade an existing installation.

If you are upgrading an existing installation, you’ll find relevant advice (along with a more complete list of changes in this release) in our release notes as usual.

Happy chatting!

---