Mac and iOS betas are now up.

NOTE
THIS BETA WILL DELETE ALL EXISTING MUC MESSAGES AND MUC BUDDIES.
MUC SUPPORT IS CURRENTLY IN A ALPHA STATE.

Ready

• basic group message support (without (pleasing) UI)
• Message quote action
• Fixed appex / main app race conditions
• Displaying group messages in activeChatsViewController
• Displaying ToFU
• Using transactions (read / write)
• Added image quality slider
• Connected download slider
• Buddies can be muted per account

Todo

• Show Emoji in bigger font
• Better unread msgs counter handling
• Delete message_history backup table. Let’s see how many crashes we see
• MUC UI
• Display message quotes in a fancy style
• Adding upload preview to the chatViewController
• MUC OMEMO
• voice messages may not work as expected

I have to first start by admitting that my judgment is probably compromised by my desire to have a paycheck, which is part of why I’m pushing this so hard. I’m likely about to lose my 9 to 5 and to prepare for that I’m applying for part-time work in retail hoping I can stretch my savings long enough to actually drum up some freelance work or find some other way not to have to work for large tech companies that pay well but treat their employees like dirt. I’ll be trying to take on freelance work either way, but I’d prefer to do it cooperatively with others.

That being said, though it’s consuming me a bit right now, I’m not just looking to make a paycheck out of this. I’d like to create the kind of place where anyone can feel comfortable applying and not feel like they have to be on their toes to pass a culture fit interview conducted exclusively by 20-or-30-something white men with beards. I also like the idea of a business that doesn’t try to operate on the thinnest of margins and focus exclusively on growth at all costs.

It’s important to me that the whole co-op be comfortable with (and know about) any clients we take. I’m not against discussing most things, and am open to being convinced otherwise, but I would likely vote against Defense/Police/ICE work, Right wing political campaigns, adtech, etc.

That being said, I would also not vote against someone joining the co-op who disagrees with me on any of those points.

A quick aside on the word “free speech” that Redoak mentions in his writeup: Organizations that aim to promote “free speech” but really just mean “we don’t moderate and we’re trying to co-opt the term ‘free speech’ and make it meaningless” I would vote against, but organizations that actually understand what free speech means (ACLU, EFF, FFRF, etc.) I would like to support.

Goals for the Co-op

One thing I’ve thought about doing for a while is teaching. It’s always been something I enjoy, but never something I’ve had the opportunity to do much of. I’d like to eventually be able to expand the co-op to include training, and I’d particularly like us to teach both introductory courses (probably at a steep discount for individuals wanting to learn about software) and more team-oriented corporate courses (the actual money maker, and hopefully we can improve the state of software for the users).

We will likely make use of a great deal of open source, and if we’re ever successful it would be important to me that we give back to those who made our success possible. The cooperative principals should be applied to software as well, so if we use an open source library and make money from doing so I would argue we should take some portion of that (in capital or labor) and set it aside to donate upstream if we are able to do so.

Target Areas

We’d likely have to take work where we can get it of course, but here are some areas I’d personally love to work in one day:

• Other co-ops
• Theaters (not a lot of money, but they often need tech and no one is developing it; maybe this would be more from a product development side)
• Solar/clean power sector
• Accessibility improvements in any sector
• Local government or administrative agencies (which are an under appreciated national treasure)
• Local improvements where our members live
• Open Source (I have at least one lead on a small open-source-first company looking for contractors that I will follow up on and hopefully bring in as a client for us to consider)
• Libraries (a friend does this sort of work out in Portland; Library software is terrible)

Personal Goals

I am not good at putting myself out there and talking to clients. Selfishly, this is another reason I’d prefer to look for freelance work with other people who may be able to do this better than I can. However, it’s also something I’d like to learn.

In general, I don’t feel like I’ve learned much at all from my last few jobs, so having a team around me that has a lot of different skills (and hopefully is okay sharing them) is something that appeals to me a lot in general, and I hope I’d have something to contribute to the groups knowledge in return.

My Skills

I am a backend developer who has worked primarily in Go. I have been working in this space since approximately 2013. I have also worked extensively in Rust, Python, and (to a lesser extent) Clojure among other languages.

I also have experience in realtime communications and have served on the XMPP Standards Foundation (XSF) council (the technical governing body) as well as an editor for the XEP series of documents.

To a lesser extent I have been minimally involved at the IETF with the PRECIS (internationalization), KITTEN (authorization), and TLS (TLS) working groups.

On a scale of 1–5 where 5 is “expert” and 1 is “don’t make me do this, the client will regret it” and an asterisk is “needs refreshing to get back to this number”:

• Go (5)
• XMPP (5)
• Rust (4)
• Terraform (4)
• Linux/FreeBSD administration (4)
• Hugo static site templates (3)
• HTML/CSS (3)
• Technical Writing (3)
• PostgreSQL (3)
• Python (3*)
• Clojure (3*)
• Android (2*)
• SmartOS (3*)
• Web Design (2)

A few days ago, VaxBot, a new XMPP-based vaccination appointment notification service was launched in the USA. The service is recommending Monal and yaxim as client applications, and both apps by default register accounts on yax.im.

This has brought us hundreds of new users (per day) and a significant amount of new traffic from the notifications, sometimes up to 200 messages per second. The additional traffic load caused some short service interruptions in the last days, and we are working together with the VaxBot team on implementing mitigations.

The VaxBot service started in mid-March in Massachusetts, and is slowly expanding to more and more states, hoping to also spread over the ocean. It was even featured on TV (archive links for EU citizens: FOX5, WBTV)!

This has lead to a significant uptick in yax.im account registrations, as can be seen in this graph:

Once a user registers with VaxBot, it will automatically send them appointment notifications for their region as soon as they become available.

This means that for each potential appointment, a message will be sent to each registered user in the region. When a large chain opens up additional capacities, thousands of messages will be generated and sent out in a burst.

As those are chat messages, they need to be stored in the respective user’s account, delivered to online devices, and forwarded to the respective push service to wake up a mobile device.

Due to how the server is processing messages, a large message flood from one connection can “capture” the processor for multiple seconds or even longer, leading to the starvation of other connections, causing delivery delays and even disconnects.

As a preliminary measure, we have implemented a rate limiting mechanism to reduce the impact of message bursts from VaxBot, and we are working on optimizing the number of messages generated by the bot and on increasing the server performance to be able to further scale up.

A long-term solution based on XEP-0060: Publish-Subscribe would be interesting and probably much more efficient for the infrastructure, but that would require significant changes to all clients, and vaccination can not wait.

DSEI is the premier showcase for military technology of all types. Held every other year, DSEI attracts one the largest international audiences, with over 75,000 visitors from 114 countries at DSEI 2019.

Isode will be displaying a number of unique capabilities at DSEI 2021, which you can see on our stand on the UK Pavilion, including:

Military Email

Isode’s end-to-end solution fronted by Harrier, our web based military messaging client, and including message servers and gateways covering all of the major military messaging standards.

Isode Harrier Web-based Email Client for Military Messaging supports the draft and release process to support formal release and approval of messages by an appropriate officer.

Military Instant Messaging

Instant messaging, using the XMPP standard, is an increasingly important component of military communications systems. Isode clients, servers and gateways allow XMPP traffic over standard and constrained (SatCom/HF) links a s well as between XMPP and legacy instant messaging systems such as IRC, all of which can be controlled using Isode’s extensive security labelling infrastructure.

At DSEI 2021 we will be giving the first public demonstration of the web version of our Swift XMPP Client.

You can find more information on Isode’s products set for Military Messaging and Military Instant Messaging by following the links.

Isode Military Messaging and Military Instant Messaging products have been successfully deployed with the land, air and naval forces of over 30 countries.

The new ejabberd 21.04 release includes many bugfixes and a few improvements. This release includes minor improvements to fully support Erlang/OTP 24 and Rebar3. At the same time, it maintains support back to the old Erlang/OTP 19.3 and Rebar2.

There are no configuration, hooks or API changes that require attention when upgrading. Nonetheless, there’s a new API command (get_user_subscriptions), a new configure option (--enable-lua), and an improvement in the MySQL database definition.

MySQL database definition improvement

We updated the database definition to fix the “specified key was too long” warnings. By default, the new character set and collation (utf8mb4 and utf8mb4_unicode_ci) will only be used with newly created databases. The existing installations don’t need to convert anything.

However, if you feel like it, after you upgrade to ejabberd 21.04, you can apply the following SQL command to convert your existing MySQL database character set to the latest definition:

alter table push_session convert to character set utf8mb4 collate utf8mb4_unicode_ci;
alter table mqtt_pub convert to character set utf8mb4 collate utf8mb4_unicode_ci;

Windows support

As you may have noticed, the last binary installer for Windows is from a year ago. Since then, the recommended method to install ejabberd on Windows is using the ejabberd Docker image, and we’ve just published two tutorials on how to do it:

• How to install ejabberd on Windows 10 using Docker Desktop
• How to install ejabberd on Windows 7 (or 8) using Docker Toolbox

ejabberd 21.04 download & feedback

As usual, the release is tagged in the Git source code repository on Github.

The source package and binary installers are available at ejabberd XMPP & MQTT server download page.

If you suspect that you’ve found a bug, please search or fill a bug report on Github.

A detailed list of changes

API Commands

• add_rosteritem/...: Add argument guards to roster commands
• get_user_subscriptions: New command for MUC/Sub
• remove_mam_for_user_with_peer: Fix when removing room archive
• send_message: Fix bug introduced in ejabberd 21.01
• set_vcard: Return modules errors

Build and setup

• Allow ejabberd to be compatible as a dependency for an Erlang project using rebar3
• CAPTCHA: New question/answer-based CAPTCHA script
• --enable-lua: new configure option for luerl instead of –enable-tools
• Remove support for HiPE, it was experimental and Erlang/OTP 24 removes it
• Update sql_query record to handle the Erlang/OTP 24 compiler reports
• Updated dependencies to fix Dialyzer warnings

Miscellaneous

• CAPTCHA: Update FORM_TYPE from captcha to register
• LDAP: fix eldap certificate verification
• MySQL: Fix for “specified key was too long”
• Translations: updated the Esperanto, Greek, and Japanese translations
• Websocket: Fix PONG responses

Modules:

• mod_block_strangers: If stanza is type error, allow it passing
• mod_caps: Don’t request roster when not needed
• mod_caps: Skip reading roster in one more case
• mod_mam: Remove queryid from MAM fin element
• mod_mqtt: When deregistering XMPP account, close its MQTT sessions
• mod_muc: Take in account subscriber’s affiliation when checking access to moderated room
• mod_muc: Use monitors to track online and hard-killed rooms
• mod_muc: When occupant is banned, remove his subscriptions too
• mod_privacy: Make fetching roster lazy
• mod_pubsub: Don’t fail on PEP unsubscribe
• mod_pubsub: Fix gen_pubsub_node:get_state return value
• mod_vcard: Obtain and provide photo type in vCard LDAP

Six weeks after previous release, ejabberd 21.01 contains as usual several improvements and bugfixes. There are no changes required in the API, configuration or databases.

Fixed sqlite3 dependency version

erlang-sqlite3 versions got messed up, causing ejabberd build to regress to 1.1.6 from Jan 2018 when using rebar3 which fails to build with OTP23. Update to correctly tagged version 1.1.9, which also has matching hex.pm package, fixes this.

Integrate nicely with systemd

Support systemd’s watchdog feature and enable it by default in the unit file, so that ejabberd is auto-restarted if the VM becomes unresponsive. Also, set the systemd startup type to ‘notify’, so that startup of followup units is delayed until ejabberd signals readiness. While at it, also notify systemd of configuration reload and shutdown states.

Note: NotifyAccess=all is required as long as ejabberdctl foreground runs the VM as a new child process, rather than “exec”ing it. This way, systemd views the ejabberdctl process itself as the main service process, and would discard notifications from other processes by default.

New ejabberdctl foreground-quiet

This starts ejabberd without detaching the process, but setups console logging to display only critical messages.

STUN

The ‘stun’ application now rejects Teredo and 6to4 TURN peers unconditionally. Therefore, remove those networks from the default ‘turn_blacklist’.

Block loopback addresses by default:

Don’t accept loopback addresses as TURN peers by default. This makes sure the TURN service won’t allow remote clients to access local UDP services.

However, this will break the case where the turn_ipv4_address was set to 127.0.0.1 as fallback and TURN worked “by accident” if both clients were using the same TURN service. The service then talked to itself on the loopback interface.

Translations

The gettext-formatted PO files are now located in a specific repository, ejabberd-po, and have been published with MIT license. This allows translators continue improving them, and be included in ejabberd packages without requiring the translators so sign a Contribution License Agreement.

Summary of changes:

Miscellaneous:

• log_rotate_size option: Fix handling of ‘infinity’ value
• mod_time: Fix invalid timezone
• Auth JWT: New check_decoded_jwt hook runs the default JWT verifier
• MUC: Allow non-occupant non-subscribed service admin send private MUC message
• MUC: New max_password and max_captcha_whitelist options
• OAth: New oauth_cache_rest_failure_life_time option
• PEP: Skip reading pep nodes that we know won’t be requested due to caps
• SQL: Add sql script to migrate mysql from old schema to new
• SQL: Don’t use REPLACE for upsert when there are “-” fields.
• Shared Rosters LDAP: Add multi-domain support (and flexibility)
• Sqlite3: Fix dependency version
• Stun: Block loopback addresses by default
• Several documentation fixes and clarifications

Commands:

• decide_room: Use better fallback value for room activity time when skipping room
• delete_old_message: Fix when using sqlite spool table
• module_install: Make ext_mod compile module with debug_info flags
• room_unused_*: Don’t fetch subscribers list
• send_message: Don’t include empty in messages
• set_room_affiliation: Validate affiliations

Running:

• Docker: New Dockerfile and devcontainer.json
• New ‘ejabberdctl foreground-quiet’
• Systemd: Allow for listening on privileged ports
• Systemd: Integrate nicely with systemd

Translations:

• Moved gettext PO files to a new ejabberd-po repository
• Improved several translations: Catalan, Chinese, German, Greek, Indonesian, Norwegian, Portuguese (Brazil), Spanish.

ejabberd 21.01 download & feedback

As usual, the release is tagged in the Git source code repository on Github.

The source package and binary installers are available at ejabberd XMPP & MQTT server download page.

We’ve discovered some issues with the Windows installer that we are still working on, so its publication is delayed.

If you suspect that you’ve found a bug, please search or fill a bug report on Github.

Do you want to install ejabberd on your Windows 10 machine? Do you miss the binary installers for Windows? Don’t worry, you can install ejabberd on Windows 10 using Docker Desktop, and this tutorial guides you through the process.

This tutorial requires Windows 10 or newer. For older systems like Windows 7 or 8, follow the tutorial on how to install ejabberd on Windows 7 using Docker Toolbox.

For some time now we have been phasing out the traditional installation wizards, customary to the end users on macOS and Windows, in favour of the more streamlined command line approach, well known on Linux desktops and servers.

First, we have phased out the macOS binary installer in favour of a quick brew install ejabberd command. Then, since ejabberd 20.07, we have phased out the Windows installer in favour of a container solution. However, setting ejabberd in Docker requires setting volumes, ports and some customizations, so we’ve written a batch script that performs all those tasks for you.

This tutorial explains how to get any ejabberd version installed on Microsoft Windows 10 using Docker Desktop and ejabberd-docker-install.bat script.

Docker Desktop is only available for Windows 10. If you use Windows 7 or 8, you can use Docker Toolbox, which is old and obsolete, but it still seems to work correctly, so give it a try. We published a tutorial explaining how to install ejabberd on Windows 7 using Docker Toolbox.

1. Install Docker Desktop

First of all, download and install Docker Desktop for Windows. The process is pretty straightforward, and it will ask you to restart your machine.

The installation wizard may ask you to install Microsoft’s WSL2 and restart the Docker Desktop app.

2. Download ejabberd-docker-install.bat

Download ejabberd-docker-install.bat to your machine.

3. Edit the install options

Edit this batch file with your favourite text editor and set, at the very least, the PASSWORD option you want for your new ejabberd administrator account.

Additionally, you can set some other options: INSTALL_DIR_WINDOWS10, HOST, USER, VERSION, and PORTS.

4. Run the script

When you run the script, it will open a console window to inform you about the process: download the ejabberd image, create the container, register the admin account and prepare the configuration file…

If installation completes correctly, you can close that window and proceed to next step.

If there was any error, solve it and run the script again. You can delete the script and download it again, or delete the ejabberd container, or delete the ejabberd installed directory… and run the script again.

5. Start ejabberd

Now you can finally go to Docker Desktop, where you can see the new ejabberd container, and click the “Start” icon:

Wait a few seconds till ejabberd is started in that container and accepting connections:

Next steps

At this point, you have ejabberd installed and running on your machine, and you may be asking yourself how to administrate it. Here are some remarks:

ejabberd.yml, database and logs

The configuration files, Mnesia internal database spool and logs directories are available for you to edit and inspect in Windows, in the path that you specified in the INSTALL_DIR_WINDOWS10 option.

There is also an ejabberd-modules directory, where you can later put additional modules from ejabberd-contrib, or any other place.

Whenever you update to a newer ejabberd, it is a good practice to backup the conf and database directories.

ejabberd WebAdmin

The “Open in browser” icon will open a browser with the ejabberd webadmin page.

Alternatively, you can open it yourself by going to http://localhost:5180` (swaplocalhostfor the value of theHOST` variable, if you changed it in the installation script).

You will be welcomed by a browser authentication prompt, where you should type in the login details defined in the installation script: USER@HOST and PASSWORD. You will then see the usual ejabberd webadmin console, where you can easily manage your server instance.

CLI with ejabberdctl

The next icon opens a console in the ejabberd container where you can use ejabberdctl, and that means you can use any ejabberd Administration API.

ejabberd-contrib

In addition to the modules already included in ejabberd releases, there are several more published in ejabberd-contrib, and many other on the internet, and you can even write your own modules.

To start with all this, open the CLI as explained previously, and execute:

bin/ejabberdctl modules_update_specs

For the next steps, check this ejabberd-contrib documentation.

Update from old binary installer

If you already have ejabberd installed using a binary installer downloaded from ProcessOne website:

1. Stop ejabberd using the “Stop ejabberd” desktop shortcut as usual
2. It is always a good practice to backup the conf and database directories
3. Uninstall ejabberd
4. Follow the steps described in this tutorial
5. Check that ejabberd runs perfectly with the basic configuration and empty database

Now it’s time to get back your configuration and database:

1. Stop ejabberd in the Docker Desktop
2. Copy your old conf and database directories to the new location
3. Start ejabberd and check if it runs correctly

Update to a new ejabberd version

When a new ejabberd version is released, go to ejabberd Docker Hub, and check if the new version is available in Tags.

How to install it?

1. Delete your old ejabberd container
2. Edit the VERSION option in ejabberd-docker-install.bat
3. Run the script

It will download the new image and create a new container.

If something goes terribly wrong

As mentioned previously, if something goes terribly wrong, don’t worry! You can delete the script, or the installed directory, or the ejabberd container, and start from scratch.

Docker Desktop in macOS and Linux

Docker Desktop is also available for the macOS and Linux systems. While the above installation script is designed for Windows, it could be modified for these other platforms as well. This means you now have several methods of installing and running ejabberd on any given operating system: using a package manager (like apt on Debian or brew on macOS), using a Docker container, with a binary installer (on Linux) or building from source.

Questions, problems, suggestions

The batch script to use Docker and this tutorial may have problems or incorrections. So, please add a comment here, or join the ejabberd chatroom, or send an email to the ejabberd mailing list or fill a bug/suggestion in the ejabberd tracker or docker-ejabberd trackers.

Photo by Frank Mckenna on Unsplash

Did you read our previous tutorial how to install ejabberd on Windows 10 using Docker Desktop? Do you have a Microsoft system older than Windows 10? Don’t worry, instead of Docker Desktop you can use Docker Toolbox, and this tutorial guides you over the process.

This tutorial explains how to get any ejabberd version installed on Microsoft Windows 7 or 8 (and probably others) 64bits using Docker Toolbox and ejabberd-docker-install.bat.

Docker Toolbox is an old and obsolete program, suitable for Microsoft systems older than Windows 10. If you have Windows 10, you will surely prefer to use Docker Desktop, check our tutorial how to install ejabberd on Windows 10 using Docker Desktop.

1. Install Docker Toolbox

First, download and install Docker Toolbox, specifically the file DockerToolbox-19.03.1.exe. The process is pretty straightforward, and it will ask you to restart your machine.

After installing that file, you will get several icons in your desktop:

Now run Docker Quickstart. It will take some time to complete.

2. Download ejabberd-docker-install.bat

Download ejabberd-docker-install.bat to your machine.

3. Edit some install options

Edit this batch file with your favourite text editor and set at least the PASSWORD option for the new administrator account.

Additionally, you can set some other options: HOST, USER, VERSION, and PORTS.

Please notice that you cannot configure the installation directory. The reason is that Docker Toolbox can only mount volumes from C:/Users/Your-User. So, the script must install your ejabberd files there.

4. Run the script

When you run the script it will open a console window to inform what it is doing: download the ejabberd image, create the container, register the admin account and prepare the configuration file…

If installation completes correctly, you can close that window and proceed to next step.

If there was any error, solve it and run the script again. You can delete the script and download it again, or delete the ejabberd container, or delete the ejabberd installed directory… and run the script again.

5. Start ejabberd

Now you can finally go to Kitematic (alpha), where you can see the new ejabberd container. Simply click the “Start” icon to run this container:

After a few seconds, ejabberd is started in that container and accepting connections.

Next steps

If you are here it means you have ejabberd installed and running in your machine, and you may be asking yourself how to administrate it. Here are some remarks:

ejabberd.yml, database and logs

The configuration files, Mnesia internal database spool and logs directory are available for you to edit and inspect in Windows, in C:/Users/your user/ejabberd.

There is also a ejabberd-modules directory where you can later put additional modules from ejabberd-contrib, or any other place.

Whenever you update to a newer ejabberd, it is a good practice to backup the conf and database directories.

ejabberd WebAdmin

To open a web browser pointing to ejabberd webadmin, go to Settings » Hostname/Ports and click on the 5180 port.

Alternatively, you can open it yourself by going to http://localhost:5180` (swaplocalhostfor the value of theHOST` variable if you changed it in the installation script).

You will be welcomed by a browser authentication prompt, where you should type in the login details defined in the installation script: USER@HOST and PASSWORD. You will then see the usual ejabberd webadmin console, where you can easily manage your server instance. That’s it!

CLI with ejabberdctl

The EXEC icon opens a console in the ejabberd container where you can use ejabberdctl, and that means you can use any ejabberd Administration API.

ejabberd-contrib

In addition to the modules already included in ejabberd releases, there are several more published in ejabberd-contrib repo, and many other in internet if you search, and you can even write your own modules.

To start with all this, open a CLI as explained previously, and execute:

bin/ejabberdctl modules_update_specs

For the next steps, check this ejabberd-contrib documentation.

Update from old binary installer

If you already have ejabberd installed using a binary installer downloaded from ProcessOne website:

1. Stop ejabberd using the “Stop ejabberd” desktop shortcut as usual
2. It is always a good practice to backup the conf and database directories.
3. Uninstall ejabberd
4. Follow the steps described in this tutorial
5. Check ejabberd runs perfectly with the basic configuration and empty database.

Now it’s time to get back your configuration and database:

1. Stop ejabberd in the Docker Desktop
2. Copy your old conf and database directories to the new location
3. Start ejabberd and check if it runs correctly now too.

Update to a new ejabberd version

When a new ejabberd version is released, go to ejabberd Docker Hub, and check if the new version is available in Tags.

How to install it?

1. Delete your old ejabberd container
2. Edit the VERSION option in ejabberd-docker-install.bat
3. And run the script

It will download the new image and create a new container.

If something goes terribly wrong

As mentioned previously, if something goes terribly wrong, don’t worry! You can delete the script, or the installed directory, or the ejabberd container, and start from scratch.

Docker Desktop in macOS and Linux

Docker Desktop is also available for the macOS and Linux systems. While the above installation script is designed for Windows, it could be modified for these other platforms as well. This means you now have several methods of installing and running ejabberd on any given operating system: using a package manager (like apt on Debian or brew on macOS), using a Docker container, with a binary installer (on Linux) or building from source.

Questions, problems, suggestions

The batch script to use Docker and this tutorial may have issues or mistakes. Please add a comment here, or join the ejabberd chatroom, or send an email to the ejabberd mailing list or fill a bug/suggestion in the ejabberd tracker or docker-ejabberd trackers.

Photo by Rinson Chory on Unsplash

Welcome to the XMPP newsletter covering the month of March 2021.

Many projects and their efforts in the XMPP community are a result of people’s voluntary work. If you are happy with the services and software you may be using, especially throughout the current situation, please consider to say thanks or help these projects!

Read this Newsletter via our RSS Feed!

Interested in supporting the Newsletter team? Read more at the bottom! Other than that - enjoy reading!

Newsletter translations

Translations of the XMPP Newsletter will be released here (with some delay):

• French on linuxfr.org and jabberfr.org.
• Read the German translation of the previous month's newsletter

XSF Announcements

The XMPP community is now holding regular virtual office hours, short weekly talks, demos, or round table discussions about XMPP or XMPP-adjacent topics! Meeting details, a list of future talks, and the sign up sheet can be found on the XMPP Wiki. And we are on YouTube now!

Events

XMPP Office Hours each week!

Berlin XMPP Meetup (remote): Monthly Meeting of XMPP Enthusiasts in Berlin - always 2nd Wednesday of every month.

Articles

Nicola Fabiano wrote two articles on "Consciously looking at messaging apps: when users want broad freedom and full control over their data" and "Aware digital communication respecting privacy and the apps or services you choose"

Arnaud Joset wrote a small tutorial on how to use Prosody with a HTTP Reverse Proxy.

Marek Foss from ProcessOne wrote several articles on ejabberd background and setups:

• Starting with MQTT protocol and ejabberd MQTT broker
• Install and configure MariaDB with ejabberd
• Getting started with WebSocket API in ejabberd
• Publish-Subscribe pattern and PubSub in ejabberd

Alex Akinbia and Ehizojie Ojieb published "Forensic analysis of open-source XMPP multi-client social networking apps on iOS devices". They did their study based on Monal 4.5 and Siskin 5.8.1.

Software news

Clients and applications

Conversations 2.9.8 and 2.9.9 have been released. These releases bring better compatibility with non-libwebrtc WebRTC implementations, verification of audio and video (A/V) calls with preexisting OMEMO sessions, and better TOR support.

Gajim Development News (March): Gajim’s new main window is coming together step by step while many core functionalities are being rewritten. This month also brought better accessibility for Gajim’s message input and improved status icon behavior. Gajim 1.3.1 has been released as well and they added a setting to explicitly enable GSSAPI authentication and improvement changes.

JSXC - the Javascript XMPP Client got a funding for group chat calls!

Monal is now on Mastodon! The second beta release has been published as well, high-way to Monal 5.0! Besides, a setup has been created to support the vaccination process in many US states, but also Puerto Rico. More than one million notifications have been sent already. XMPP is once again supporting in the pandemic crisis. There were several blog posts in March on this topic, you can start with this selected one. Stay tuned and please support by testing the current beta!

qXMPPconsole is a new browser based XMPP console. It's main purpose is to aid in learning about the XMPP protocol.

Servers

No server updates out there :(

Libraries

Smack 4.4.2 has been released, containing mostly bugfixes.

Extensions and specifications

Developers and other standards experts from around the world collaborate on these extensions, developing new specifications for emerging practices, and refining existing ways of doing things. Proposed by anybody, the particularly successful ones end up as Final or Active - depending on their type - while others are carefully archived as Deferred. This life cycle is described in XEP-0001, which contains the formal and canonical definitions for the types, states, and processes. Read more about the standards process. Communication around Standards and Extensions happens in the Standards Mailing List (online archive).

Proposed

The XEP development process starts by writing up an idea and submitting it to the XMPP Editor. Within two weeks, the Council decides whether to accept this proposal as an Experimental XEP.

• Content Rating Labels has been proposed and accepted.

New

• Version 0.2.0 of XEP-0456 (Content Rating Labels)
  • This specification provides a wire format in the form of a Service Discovery extension to allow services of various kinds to publish information about the kind of content they allow and/or endorse on their platform.
  • Describe the conversion algorithm. (jsc)

Deferred

If an experimental XEP is not updated for more than twelve months, it will be moved off Experimental to Deferred. If there is another update, it will put the XEP back onto Experimental.

• No XEPs deferred this month.

Updated

• Version 1.1 of XEP-0294 (Jingle RTP Header Extensions Negotiation)

  • Add mapping for a=extmap-allow-mixed (ph)

• Version 0.2.0 of XEP-0408 (Mediated Information eXchange (MIX): Co-existence with MUC)

  • Fix reference to MIX-CORE namespaces (@mathieui)

• Version 0.3.0 of XEP-0406 (Mediated Information eXchange (MIX): MIX Administration)

  • Fix reference to MIX-CORE namespaces (@mathieui)

• Version 2.12.0 of XEP-0004 (Data Forms)

  • Clarify that the 'reported' element must appear before any 'item' element. (fs, jsc)

Last Call

Last calls are issued once everyone seems satisfied with the current XEP status. After the Council decides whether the XEP seems ready, the XMPP Editor issues a Last Call for comments. The feedback gathered during the Last Call help improving the XEP before returning it to the Council for advancement to Draft.

• XEP-0280 Message Carbons
• XEP-0313 Message Archive Management

Draft

• No Drafts this month.

Call for Experience

A Call For Experience - like a Last Call, is an explicit call for comments, but in this case it's mostly directed at people who've implemented, and ideally deployed, the specification. The Council then votes to move it to Final.

• No Call for Experience this month.

Thanks all!

This XMPP Newsletter is produced collaboratively by the community.

Thanks to anubis, Bastoon, emus, jeybe, jonas-l, Julien Jorge, Holger, pmaziere, Sam Whited, seveso, vanitasvitae, wurstsalat3000 and Ysabeau for their help in creating it!

Spread the news!

Please share the news on "social networks":

• Twitter
• Mastodon
• YouTube
• LinkedIn
• Facebook
• Reddit

Find and place job offers in the XMPP job board.

Also check out our RSS Feed!

Help us to build the newsletter

We started drafting in this simple pad in parallel to our efforts in the XSF Github repository. We are always happy to welcome contributors. Do not hesitate to join the discussion in our Comm-Team group chat (MUC) and thereby help us sustain this as a community effort.

You have a project and write about it? Please consider sharing your news or events here, and promote it to a large audience! And even if you can only spend a few minutes of support, these would already be helpful!

Tasks which need to be done on a regular basis are for example:

• Aggregation of news in the XMPP universe
• Short formulation of news and events
• Summary of the monthly communication on extensions (XEP)
• Review of the newsletter draft
• Preparation for media images
• Translations: especially German and Spanish

License

This newsletter is published under CC BY-SA license.

We don’t track the number of users Monal has. We only do see thousands of downloads in the App Store statistics. Any user counting and crash tracking provided by Apple is strictly opt-in only. So it always undercounts. However because every user registers a push device with the push server we do see how many devices there are using Monal. This is a very rough estimate that is a bit inflated because you can reset your push device IDs. With all that said, there are currently:

• 66,476 (+2876) devices have registered to receive push notifications.
• On 3/31 , the push server sent 95,922 (+53%) push notifications.
• Apple reports Monal iOS installs are up 79%, Mac installs are up 8%
• The US vaccine drive has been very successful, bringing in thousands of new users to XMPP. 1.12 million vaccine notifications were sent in March. See this website.

We are working heavily on the new release for you! Please support our 100% volunteering developers and test the Monal Beta!

We’re pleased to announce a new addition to the Isode family, Chris Marshall, who joined our pre-sales team earlier this year.

Chris has a BSc in Computer Science from the University of Wolverhampton and over 30 years of experience in the IT sector. He is a keen traveler, recently exploring the USA with his family for a month. During his free time enjoys racing his laser dinghy at the local sailing club.

Chris joins us from an Isode partner, where he spent 20 years supporting customers in large, highly secure organisations around the world with their messaging projects. He is familiar with Isode software and has wasted no time getting stuck in providing support to our customers.

Welcome to the team Chris.

I’ve long thought that the definition of “Free Software” is off putting at best. At worst, its viral licensing, moral rigidity, and absolutist value judgements are harmful to actual software freedom. Meanwhile, the definition of “Open Source” makes for more legally compatible software and provides flexibility for software authors and users. However, it’s broad enough as to be almost meaningless, making it easily appropriated by corporate interests. Finally, both put a great deal of emphasis on software distribution and other legal matters while ignoring project governance. A better system would account for equity in distribution, governance, and use of the software.

More recently, I became aware of a post about the “philosophy and understanding of the role of computing and software in our society”:

This got me thinking about the values I apply to software development, governance, and distribution. I realized that they’re the same values I apply when searching for services to use, or starting a business: cooperative values. Naturally this would be called “cooperative software” or “cooperative technology”: software (or technology more generally) that roughly follows the principals of the 1995 “Statement on the Cooperative Identity”.

Finally, today, I became aware of an early draft of an essay titled “Towards A Communal Software Movement” and the authors decision to rename it “cooperative software”. Since others are thinking about this too, I wondered if we might all come together and try to better define what we mean when we use the term. To start, here is my small contribution.

Definition of Cooperative Technology

Cooperative technology is technology that is jointly-owned and democratically-controlled in accordance with cooperative principals.

If we are talking specifically about software I believe that this means that all Cooperative Software is FOSS, but not all FOSS is Cooperative.

For example, the Go programming language is Open Source Software (OSS) because it is released under a BSD style license and accepts contributions from the community. However, it is not Cooperative Software because it is not democratically controlled by members, instead decisions are made by Google employees. Similarly the Benevolent dictator for life (BDFL) model of governance used by Clojure and Linux means that these projects are not Cooperative Software.

Some projects use a meritocracy model that appears to be cooperative at first blush because it involves individuals working together to establish consensus. However, if only developers who showed a certain level of merit are allowed to govern the project, the users of the software, the designers, the technical writers, etc. become ineligible to participate. This violates the cooperative principle of voluntary and open membership.

Benefits of Cooperative Technology

Just like there are Worker Cooperatives and Consumer Cooperatives in the business world, Cooperative Technology provides a great deal of flexibility in governance while still ensuring equity. A cooperative business can survive in a capitalist market economy, but doesn’t re-enforce the inequities inherent in such a system. It also doesn’t require any major changes to adapt if it becomes a part of a more equitable system in the future.

Co-ops also help prevent the cult of personality that sometimes forms around individual founders because they strive to be inclusive. Even if a member does become a polarizing figure, their impact is limited due to the 1-person-1-vote principal.

Open Questions for Future Posts and Debate

If the software communicates over the network but uses a network protocol that is not documented (so other software cannot communicate with it without reverse engineering the protocol from the code and hoping it does not change and break them), can it still be Cooperative Technology? Likewise, are programming languages that have a reference implementation instead of a spec Cooperative if they meet the rest of the definition, or does this make it too difficult to create alternative implementations? Is this a violation of the “Concern for Community” principle, or the general value of solidarity?

What licenses are acceptable for Cooperative Software to use? These licenses presumably must guarantee user freedoms, but also not hurt or exclude users. Maybe any license is fine and Cooperative Software is more about governance, but I’d also argue that licenses with a viral component are incompatible because it reduces the ability to cooperate with other Cooperative Software using an incompatible license. Cooperatives realize that not everyone will agree on the exact way that the cooperative should be run and they seek to reach consensus, not legally force other groups to agree with them or self-segregate.

What types of cooperative governance map well to developing technology, and which ones make decision making too difficult? In non-cooperative software if every user feature request is honored we just get Jira: over complicated and nobody really knows what it’s for or how to use it. But in cooperative software if users must all come together to reach consensus, does this help them all distill their features and use cases down to the most fundamental form?

Once we begin to answer these questions, should there be a steward for the definition of Cooperative Technology in the vein of the Open Source Initiative (OSI) or the Free Software Foundation (FSF)?

If you’re interested in helping answer these questions, or in improving the definition of Co-op Technology, consider joining the co-op group chat: co-op@mellium.chat.

Gajim’s new main window is coming together step by step while many core functionalities are being rewritten. This month also brought better accessibility for Gajim’s message input and improved Status Icon behavior.

Changes in Gajim

During March, we steadily worked on Gajim’s new main window, reimplementing feature after feature. Since these changes often lead to core functionalities being rewritten, we decided to take a step forward in many areas. For example: instead of adapting the old Contact Information window, we decided to go for the new implementation we originally planned for some time later, because it gave us more freedom regarding contact specific actions. For instance, the new Contact Information window includes group management and subsciption management, which renders some of the old contact list dialogs obsolete (i.e. we don’t have to re-implement those). We’ll go into details with the coming blog posts. Stay tuned!

Meanwhile, the chat message input received a focus-indicating border. You already know this from single line Entries elsewhere in Gajim: once you focus the entry, a blue border appears around it, making it obvious where the current focus is. It’s the same now for the chat message input (being a multi line Textview instead of a single line Entry). We also removed the ‘Write a message…’ placeholder, since it did not always work reliably.

Furthermore, the window behavior for Gajim’s contact list has been improved. When clicking on the status icon in you system tray while Gajim’s window resides somewhere in the background behind other windows on your desktop, it will be raised to the top (this is new). If it’s minimized to the tray, it will be restored; if it’s focused, it will be minimized (this was already the case before).

What else happened

• An issue with handling missing avatar hashes has been fixed, which would prevent Gajim from starting #10428
• A domain validation issue has been resolved #10010

Plugin updates

No plugin updates this month.

Changes in python-nbxmpp

A module for handling roster operations has been implemented and improved while working on Gajim’s new main window, which features a minimal version of Gajim’s current contact list/roster combination. While implementing the new Contact Information window, a module for XEP-0202 Entity Time has been added as well.

As always, feel free to join gajim@conference.gajim.org to discuss with us.

We are happy to announce the availability of Smack 4.4.2, the second patch level release of Smack’s 4.4 series. It mostly contains bugfixes. Thanks to everyone who contributed by reporting and fixing bugs. As always, this Smack release is available via Maven Central.

We would like to use this occasion to point at that Smack now ships with a NOTICE file. Please note that this adds some requirements when using Smack as per the Apache License 2.0. The content of Smack’s NOTICE file can conveniently be retrieved using Smack.getNoticeStream().

1 post - 1 participant

Read full topic

Quick update on the vaccination project with monal. We have crossed 1000 new XMPP users using the service. The rate of growth is accelerating and the next thousand will likely be faster than the first. we have also sent out almost 250 thousand vaccine notifications. This number will likely not grow as fast as we put geographic limits on the locations we sent alerts about so they are more relevant — something that will probably drive user growth. The database now has 39 states with more coming. This will accelerate the growth as we move beyond a target audience of only 7 million to 300 million.

For the first time, Isode will be taking a stand in independent capacity, for DSEI 2021.

DSEI is the premier showcase for military technology of all types. Held every other year, DSEI attracts one the largest international audiences, with over 75,000 visitors from 114 countries at DSEI 2019.

We have attended DSEI for a number of years but always in support of demonstrations, by our valued partners, of their solutions for military messaging.

This year, we are joining over 1000 exhibitors in taking our own stand at DSEI.

Jon Purvis, Hannah George and Jeff Tillotson on the Isode exhibition stand.

As well as enabling us to provide on-site support of demonstrations by partner companies, this will also provide a central point for showcasing Isode’s unique capabilities in secure messaging and instant messaging to the many key decision makers who regularly attend DESI.

Isode messaging and instant messaging solutions are now deployed with the land, air and naval forces of over 30 countries.

You’ll find Isode on the UK Pavilion at DSEI, which will be held at the ExCel Centre, London on 14-17 September 2021.

Publish–Subscribe is a messaging pattern where senders of messages, called publishers, do not send the messages directly to specific receivers, called subscribers. Instead, publishers categorize messages into channels without knowledge of which subscribers, if any, there may be. Similarly, subscribers express interest in one or more channels and only receive messages that are of interest, without knowledge of which publishers, if any, there are.

» Don’t want to configure PubSub yourself?
ProcessOne experts will make your business instantly connected. Contact us »

ejabberd and PubSub

In case of ejabberd, the Publish-Subscribe pattern (PubSub) is implemented by the native module mod_pubsub. From the outside, it works very similar to ejabberd MQTT module mod_mqtt. However, mod_pubsub uses XML and iq stanzas for communication. Therefore, ejabberd PubSub brings all the advantages as well as complexities that come with XML.

Create a PubSub node

Before we start, make sure you are using an XMPP client that has an option of sending raw XML input to your server. One of such clients is Psi, available for many platforms.

Next, we need to keep in mind that in ejabberd mod_pubsub is enabled by default with plugins flat and pep. The permission to create nodes is limited to local accounts.

To create your first node, send the following command using the “XML Input” within the Psi XML Console. Substitute marekfoss.org in to and node params with your server domain, and mf in the /home/marekfoss.org/mf/open with your username. The node in this example is called open.

<iq type="set" to="pubsub.marekfoss.org" id="create1">
  <pubsub xmlns="http://jabber.org/protocol/pubsub">
    <create node="/home/marekfoss.org/mf/open"/>
    <configure/>
  </pubsub>
</iq>

To verify that the node was created, you can send the above command again, at which point you should get a 409 error saying “Node already exists”. To create more nodes, remember to increment or change the id value of the iq element, like create1, create2 etc. and change the node’s name in the /home/marekfoss.org/mf/... path.

Please note that the correct node path: /home/[domain]/[username]/[nodename] is important. With the default mod_pubsub permissions, the plugin will allow you to create this node. If you would like to create your node in another path, refer to the docs.

Subscribe the PubSub client

Now we can subscribe our client to receive messages from the node we just created. To do this, we use the following command, where we subscribe the specified jid to the specified node:

<iq type="set" to="pubsub.marekfoss.org" id="sub1">
  <pubsub xmlns="http://jabber.org/protocol/pubsub">
    <subscribe node="/home/marekfoss.org/mf/open"
               jid="mf@marekfoss.org"/>
  </pubsub>
</iq>

If successful, you should see in your XML terminal a response containing a section like this:

<subscription subid="65026C52DADF1" node="/home/marekfoss.org/mf/open" jid="mf@marekfoss.org" subscription="subscribed"/>

Publish with PubSub client

Now we can test our new PubSub communication channel by sending the first message to the node we created. To do this, we use the following command:

<iq type="set" to="pubsub.marekfoss.org" id="publish1">
  <pubsub xmlns="http://jabber.org/protocol/pubsub">
    <publish node="/home/marekfoss.org/mf/open">
      <item>
        <entry xmlns="http://www.w3.org/2005/Atom">
          <title>Hello Brave New World</title>
          <summary>
To be, or not to be: that is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles,
And by opposing end them?
          </summary>
        </entry>
      </item>
    </publish>
  </pubsub>
</iq>

In response, you should see the message relied back to you, as well as an acknowledgement looking like the following:

<iq to="mf@marekfoss.org" from="pubsub.marekfoss.org" type="result" id="publish1">
  <pubsub xmlns="http://jabber.org/protocol/pubsub">
    <publish node="/home/marekfoss.org/mf/open">
      <item id="650270397BE77"/>
    </publish>
  </pubsub>
</iq>

When publishing, the item entry can contain more tags with things like dates, links etc. Please refer to this post for more examples.

What can we do with ejabberd PubSub?

If you look at the ejabberd PubSub features you can see that it contains a vast amount of options at your disposal, like fine-grained authorization, subscription monitoring and management, message retractions and more.

When you compare it to the earlier tutorial on ejabberd MQTT broker, you can see a much wider range of features available out-of-the-box when using PubSub.

In this ejabberd tutorial series:

• How to move the office to ejabberd XMPP server
• How to set up ejabberd video & voice calling (STUN/TURN)
• How to configure ejabberd to get 100% in XMPP compliance test
• Check ejabberd XMPP server useful configuration steps
• Starting with MQTT protocol and ejabberd MQTT broker
• Getting started with WebSocket API in ejabberd
• Install and configure MariaDB with ejabberd
• Publish-Subscribe pattern and PubSub in ejabberd

Photo by Sawyer Bengtson on Unsplash

I made my XMPP based vaccine notification bot available to the general public about a week ago. The service is available at vaccine.monal.im . In the last three days I have sent 129 thousand notifications. Over the past week hundreds of Americans of all ages and from all walks of life have downloaded Monal and Yaxim and tried XMPP for the first time. Monal downloads for iOS are up 745.8% Yaxim is up 117%. This is just the beginning and I expect thousands if not millions of people to install our apps as we expand from Massachusetts, which is a state of only 6 million to larger states like Texas that cover hundreds of millions of people and the vaccine becomes available to the general public rather than select groups. While this is for the US only right now, I believe I have proven the model. We need to replicate this for vaccination drives in other countries.

The growth has become organic. It is spreading via word of mouth and social media posts now. A major part of this project has been explaining XMPP to people, explaining why this was possible with XMPP and not possible with other messaging systems or even SMS. The costs of sending 129k notifications with SMS via service like Twillio would be prohibitively expensive. I am starting to scale up my AWS hardware as more users come onboard but I know that the costs of currently running this whole stack costs less a month than the daily cost of running an SMS service that performs a similar task. Once I explained it to some people and it clicked they have been able to evangelize to others on their own and walk them through the process. If you look at how much the name and even the language had changed on the landing page, I have certainly learned what people understand and what they do not. For example I thought push and bot were scary words and used notification and alert but it turns out non technical people know what push notification means and chat bot actually conveys a lot of meaning to almost everyone about what kind of service this is.

Not everyone who installs this will use XMPP, in fact I expect many to uninstall after they are vaccinated. But some will stick around and this can only be good for the eco system. Even those who uninstall will remember how they got vaccinated. When we talk about XMPP hopefully it’s not a conversation introducing something new it’s a conversation where we can start with, “Remember that program you used to get the vaccine, it can do more”. A year ago we started this pandemic with lockdown and it was incredibly depressing. As software engineers there wasn’t a lot we could do other than staying at home and helping stop the spread. We are doing something to end it now. Many people have sent me selfies getting vaccinated and touching personal notes about how this service has relieved stress on their family. I would like to thank Thilo, Friedrich, Emus and Jim who work on Monal as well as Georg who makes yaxim, for helping make this possible.

This is service is heavily dependent on open source tools made by others teams. I will follow up with additional posts with the technical details of the stack. I hope to make the whole thing open source so others can replicate this service elsewhere.

If you are in Massachusetts, I post a daily update on reddit with vaccine availability data generated by the bot. If you are interested, this is today’s.

It's been a while since I first installed prosody on Agayon.be. I use it to experiments with my bots, to keep contact with the XMPP community and discover new cool stuffs to do. Recently I struggled a bit because I wanted to hide the prosody small HTTP server behind my Proxy. For various reasons, I still use Apache 2.4 and I could not get it to work with prosody. I mostly use the HTTP server for bosh authentication with Converse.js and with the http_upload module. When the 5281 port was accessible and Prosody handled the requests directly on the internet it worked well. But when I followed the documentation to use a proxy, it stopped working. All my PUT requests got a 404 error. I tested my setup with Slixmpp and the http_upload example.

Here is my configuration before the fix:

Prosody

Main config

[...]
https_ports = { 5281 }
https_interfaces = {  "127.0.0.1", "::1" }
trusted_proxies = { "127.0.0.1", "::1"}
[...]

VirtualHost

[...]
Component "upload.example.com" "http_upload"
       http_max_content_size = 10485760
       http_external_url = "https://upload.example.com/"
[...]

Apache VirtualHost

[...]
ProxyPass / http://localhost:5280/
ProxyPassReverse / http://localhost:5280/
[...]

Logs

Client

[...]
Client:
DEBUG    SEND: <iq id="23efd54cf4b2487386852e800f2ea411" to="upload.example.com" type="get"><request xmlns="urn:xmpp:http:upload:0" filename="robot.png" size="118037" content-type="image/png" /></iq>
DEBUG    RECV: <iq type="result" id="23efd54cf4b2487386852e800f2ea411" from="upload.example.com" to="test@example.com/test"><slot xmlns="urn:xmpp:http:upload:0"><get url="https://upload.example.com/upload/au5rOiUMomJbDI3q/robot.png" /><put url="https://upload.example.com/upload/au5rOiUMomJbDI3q/robot.png" /></slot></iq>
ERROR    Could not upload file: 404 (<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>404 Not Found</title>
example.com
[...]

Server

[...]
Mar 21 10:02:42 c2s5586c8e88960 debug   Received[c2s]: <iq id='23efd54cf4b2487386852e800f2ea411' type='get' to='upload.example.com'>
Mar 21 10:02:42 c2s5586c8e88960 debug   Given upload slot "au5rOiUMomJbDI3q/robot.png"
Mar 21 10:02:42 c2s5586c8e88960 debug   Sending[c2s]: <iq type='result' id='23efd54cf4b2487386852e800f2ea411' from='upload.example.com' to='memo@agayon.be/test'>
Mar 21 10:02:42 socket  debug   server.lua: accepted new client connection from ::1:49436 to 5280
Mar 21 10:02:42 http.server debug   Firing event: PUT /upload/au5rOiUMomJbDI3q/robot.png
Mar 21 10:02:42 http.server debug   Firing event: PUT localhost/upload/au5rOiUMomJbDI3q/robot.png

What is important to see is the second event:

PUT localhost/upload/au5rOiUMomJbDI3q/robot.png

on a working configuration it is

PUT upload.example.com/upload/au5rOiUMomJbDI3q/robot.png

The fix

After a small discussion with Link Mauve from JabberFr, he suggested me to use setup the proxy to tell him to force its headers:

Apache

    [...]
    RequestHeader set Host "upload.example.com"
    ProxyPreserveHost On
    ProxyPass / http://localhost:5280/
    ProxyPassReverse / http://localhost:5280/
    [...]

Nginx

[...]
proxy_set_header Host "upload.example.com";
[...]

Two small lines and now it is working as expected :-).

Links

• http_upload module
• Posody http configuration

The WebSocket API, as neatly explained by the MDN, is a technology that makes it possible to open a two-way interactive communication session between the user’s browser and a server. With WebSocket API, you can send messages to a server and receive event-driven responses without having to poll the server for a reply. In a sense it’s similar to XMPP or MQTT, but created with web applications in mind.

» Don’t want to configure WebSocket API yourself?
ProcessOne experts will make your business instantly connected. Contact us »

ejabberd and WebSocket API

WebSocket API listener is present in every latest version of ejabberd installation. If you followed my earlier tutorials, WebSocket API is available on the same port as the admin console, wss://example.com/5443/ws. Let’s see what we can do with it.

We begin with a ConverseJS quick start. ConverseJS is a very nice XMPP client for web browsers that supports BOSH and WebSocket API. Our aim is to set up a chat widget on our website, so visitors can instantly chat with us via a predefined login.

First, we add ConverseJS resources to the HTML page on which we want to have the chat widget, ideally in the <head> section:

<link rel="stylesheet" type="text/css" media="screen" href="https://cdn.conversejs.org/dist/converse.min.css">
<script src="https://cdn.conversejs.org/dist/converse.min.js" charset="utf-8"></script>

Then, we add the JavaScript code to initialize ConverseJS. Ideally just before the closing </body>:

<script>
    converse.initialize({
        bosh_service_url: 'https://example.com:5443/bosh', // Please use this connection manager only for testing purposes
        show_controlbox_by_default: true
    });
</script>

At this point, when we reload our HTML page we should see ConverseJS popup in the lower right corner. However, it asks for user login and doesn’t connect to our ejabberd server. We want an automatic login of a predefined user from our own server, joining a specified chat room. And we want to use WebSocket API instead of BOSH:

<script>
    converse.initialize({
        allow_logout: false,
        allow_muc_invitations: false,
        allow_contact_requests: false,

        authentication: 'login',
        auto_login: true,
        auto_reconnect: true,

        jid: 'www@example.com',
        password: 'pass123',

        websocket_url: 'wss://example.com:5443/ws/',
        auto_join_rooms: [
            'open@conference.example.com',
        ],
        notify_all_room_messages: [
            'open@conference.example.com',
        ],

        theme: 'concord',
        keepalive: true,
        hide_muc_server: true,
        play_sounds: true,
        singleton: true,
        show_client_info: false,
        show_controlbox_by_default: false,
        strict_plugin_dependencies: false
    });
</script>

In order for the above ConverseJS initialiser to work, you need to create a new ejabberd user, for example www@example.com with password pass123. There are other authentication methods described in the ConverseJS docs that enable secure ways to handle www user’s password.

To make the widget look more like a quick chat window instead of an IM messenger, you can paste the following CSS style right after the <script> initialiser section:

<style>
    #conversejs.converse-overlayed .chatbox.chatroom .box-flyout {
    max-width: 300px !important;
    max-height: 60vh !important;
        min-width: 0 !important;
        margin-left: 9rem !important;
    }

    #conversejs.converse-overlayed .chat-head {
        min-height: 0 !important;
        padding-bottom: 0 !important;
    }

    .occupants,
    .toggle-occupants,
  .chatbox-btn.close-chatbox-button,
    .chatbox-btn.show-room-details-modal {
        display: none !important;
    }
</style>

At this point, if you open the HTML page, a popup will appear asking just for the nickname and then connecting via WebSocket API straight to open@conference.example.com chat room. It will even keep the visitor connected to the chat room while they are browsing throughout the website, thanks to WebSocket API. Notice how we achieved all that without even touching ejabberd configuration file.

This WebSocket API demo is visible on my blog. Visitors can chat using the MUC room – with me and between each other.

In this ejabberd tutorial series:

• How to move the office to ejabberd XMPP server
• How to set up ejabberd video & voice calling (STUN/TURN)
• How to configure ejabberd to get 100% in XMPP compliance test
• Check ejabberd XMPP server useful configuration steps
• Starting with MQTT protocol and ejabberd MQTT broker
• Getting started with WebSocket API in ejabberd
• Install and configure MariaDB with ejabberd
• Publish-Subscribe pattern and PubSub in ejabberd

Photo by Paul Hanaoka on Unsplash

By default, ejabberd uses the Mnesia internal database. It is great for home and small office environments, but in larger companies, as the amount of chat logs and users grows, we need more scalability. Today, I will show you how to install MariaDB, a MySQL-compatible database, migrate your data and configure ejabberd to use MariaDB instead of Mnesia.

» Don’t want to migrate data yourself?
ProcessOne experts will make your communication scalable. Contact us »

Installing MariaDB

We assume the usual Debian configuration as in my previous tutorials. I have updated my ejabberd to version 21.01 (the update process is the same as the initial ejabberd installation, so check my first tutorial).

To install MariaDB simply use:

apt-get install mariadb-server

Then run the installation wizard and follow the instructions:

mysql_secure_installation

Preparing MariaDB for ejabberd

To get MariaDB ready for ejabberd, we need to create a new database, its user, and then populate the database with the ejabberd SQL schema.

First, let’s create the database using your MariaDB root user:

echo "CREATE DATABASE ejabberd;" | mysql -h localhost -u root -p

Next, let’s create a dedicated ejabberd user authenticated with a password, and assign it to this database. The Enter password prompt is again asking about the root MariaDB user:

echo "GRANT ALL ON ejabberd.* TO 'ejabberd'@'localhost' IDENTIFIED BY 'password';" | mysql -h localhost -u root -p

Finally, let’s download the latest ejabberd SQL schema and load it into our database. This time, we are switching to using the ejabberd MariaDB user, and the Enter password prompt is asking for the password we just specified in the GRANT command above:

wget https://raw.githubusercontent.com/processone/ejabberd/master/sql/mysql.sql
mysql -h localhost -D ejabberd -u ejabberd -p < mysql.sql

To verify that everything is correct, run a command to display all the database tables, again using the ejabberd MariaDB user, and the output should look something like that:

echo "SHOW TABLES;" | mysql -h localhost -D ejabberd -u ejabberd -p --table
Enter password: 
+-------------------------+
| Tables_in_ejabberd      |
+-------------------------+
| archive                 |
| archive_prefs           |
| bosh                    |
| caps_features           |
| last                    |
| mix_channel             |
| mix_pam                 |
| mix_participant         |
| mix_subscription        |
| motd                    |
| mqtt_pub                |
...

Configuring ejabberd for MariaDB

Now that our MariaDB tables are ready, we need to configure ejabberd to use this MySQL-compatible database. Edit your ejabberd.yml config and add the following settings, where password refers to the ejabberd MariaDB user:

sql_type: mysql
sql_server: "localhost"
sql_database: "ejabberd"
sql_username: "ejabberd"
sql_password: "password"

Migrating Mnesia data to MariaDB database

At this point, if you restart your ejabberd, it won’t be using MariaDB just yet. Let’s first migrate Mnesia data into our new SQL database using ejabberdctl – we first export the data into a mnesia.sql file, and then we import it into the MariaDB database:

cd /opt/ejabberd-21.01/bin/
./ejabberdctl export2sql marekfoss.org /tmp/mnesia.sql
mysql -h localhost -D ejabberd -u ejabberd -p < /tmp/mnesia.sql
rm /tmp/mnesia.sql

It’s a good practice to remove the /tmp/mnesia.sql after we are done with it. Now, add default_db: sql to your ejabberd.yml configuration file:

default_db: sql

sql_type: mysql
sql_server: "localhost"
sql_database: "ejabberd"
sql_username: "ejabberd"
sql_password: "password"

Then, restart your ejabberd instance – it will now use the MariaDB database! Please note that the Mnesia database will still be started up and used for non-persistent data and clustering.

In this ejabberd tutorial series:

• How to move the office to ejabberd XMPP server
• How to set up ejabberd video & voice calling (STUN/TURN)
• How to configure ejabberd to get 100% in XMPP compliance test
• Check ejabberd XMPP server useful configuration steps
• Starting with MQTT protocol and ejabberd MQTT broker
• Getting started with WebSocket API in ejabberd
• Install and configure MariaDB with ejabberd
• Publish-Subscribe pattern and PubSub in ejabberd

Photo by Amy Asher on Unsplash

MQTT stands for Message Queuing Telemetry Transport. It is a simple and lightweight publish/subscribe messaging protocol: MQTT broker sends & relays messages to MQTT clients. It was designed for constrained devices and low-bandwidth, high-latency or unreliable networks. All while attempting to ensure reliability and some degree of assurance of delivery. This makes the protocol ideal for the Internet of Things (IoT). Another good fit are mobile apps where bandwidth and battery power are at a premium.

» Don’t want to manage ejabberd MQTT broker yourself?
ProcessOne experts will make your business instantly connected. Contact us »

ejabberd and MQTT

ejabberd MQTT broker is included in every installation of latest ejabberd versions and is available on port 1883 out-of-the-box. In my first ejabberd tutorial I installed ejabberd on my private server. I didn’t do any additional configuration towards an MQTT broker, so lets see if it’s really there.

To do this test, we need an MQTT client. We can use one of existing the GUI clients like MQTT FX or one of the best CLI clients, Mosquitto. It supports Linux, Windows and macOS. You can easily install it on the macOS using Homebrew with: brew install mosquitto

Subscribe the MQTT client

ejabberd acts as an MQTT broker, so first we need a client to subscribe to receive relayed messages. We do this with the following command on the machine where we installed Mosquitto:

mosquitto_sub -h mqtt.fluux.io -t "test/channel/1" -d

Publish with MQTT client

Once there’s a first subscriber, our ejabberd MQTT broker will relay any message sent from the publisher:

mosquitto_pub -h mqtt.fluux.io -t "test/channel/1" -m "This is a test message"

You will see “This is a test message” appear under the mosquitto_sub process. However, mqtt.fluux.io is a public MQTT broker, not my private server, meaning anyone can publish, subscribe and intercept messages. It’s not really meant for production.

ejabberd MQTT broker initial configuration doesn’t allow unauthorized connections, which is good. To connect, we need to use a username and password that’s already registered on our ejabberd server.

Subscribe and publish to MQTT broker with authenticated user

Lets first go to the ejabberd admin console, which is at example.com:5443/admin if we assume the configuration from my tutorial series. Lets create a new user, broker@example.com. Then, we need to add this user to the ACL in our ejabberd configuration file:

acl:
  ...
  publisher:
    user:
      - "broker@example.com"
  subscriber:
    user:
      - "broker@example.com"

Finally, we need to let mod_mqtt know who can interact with our ejabberd MQTT broker:

  mod_mqtt:
    access_publish:
      "#":
        - allow: publisher
        - deny
    access_subscribe:
      "#":
        - allow: subscriber
        - deny

Here we specify to let the publishers and subscribers, defined earlier in ejabberd ACL, have access to any channel. We block access to all channels for any unauthenticated connections. To test this, start the subscriber again with broker@example.com and its password:

mosquitto_sub -h example.com -t "test/channel/1" -u broker@example.com -P ******* -d

In another window, lets publish a test message:

mosquitto_pub -h example.com -t "test/channel/1" -m "This is a test message" -u broker@example.com -P *******

The subscriber process should display the published message. However, we still have all these MQTT messages coming and going in cleartext, unencrypted. To make it even more secure, lets set up TLS encryption for our MQTT broker.

Configure MQTT broker TLS encryption in ejabberd

There are two ways to do this. You can either create a new listener on another port, and keep the cleartext 1883 open. Or, as I prefer, simply add tls: true to the existing mod_mqtt listener:

listen:
  ...
  -
    port: 1883
    module: mod_mqtt
    backlog: 1000
    tls: true

This configuration will use your existing global certificate files to encrypt the mod_mqtt connections with TLS. To be able to connect our subscribers and publishers to the TLS encrypted MQTT broker, we need to let them know which certificate to use.

First, download the certificate files your ejabberd MQTT broker is using. These are the files defined at the top of your ejabberd configuration file in certfiles section. Then, download the latest cacert.pem. Finally, supply these certificates to the subscriber and publisher:

mosquitto_sub -h example.com -t "test/channel/1" -u broker@example.com -P ******* -d -p 1883 --cafile /path/to/cacert.pem --cert /path/to/ejabberd/fullchain.pem --key /path/to/ejabberd/privkey.pem

mosquitto_pub -h example.com -t "test/channel/1" -m "This is a test message" -u broker@example.com -P ******* -p 1883 --cafile /path/to/cacert.pem --cert /path/to/ejabberd/fullchain.pem --key /path/to/ejabberd/privkey.pem

You should see the test message appear under the subscriber process.

What can we do with ejabberd MQTT broker?

To conclude, thanks to ejabberd we now have a powerful XMPP server running together with a scalable MQTT broker. Connections on both protocols are TLS encrypted and limited to authenticated users. We have the capability to chat one-to-one, operate one-to-many multi-user chatrooms and interconnect publish-subscribe many-to-many client networks.

It’s an especially interesting setup to create interactions between digital and physical world. Next time, we will try to send MQTT messages based on XMPP events. But why? Well, for example, we could control MQTT IoT devices via commands sent to a chat with an XMPP bot assistant. And the bot could talk back to us about the events at our home. The possibilities are endless!

In this ejabberd tutorial series:

• How to move the office to ejabberd XMPP server
• How to set up ejabberd video & voice calling (STUN/TURN)
• How to configure ejabberd to get 100% in XMPP compliance test
• Check ejabberd XMPP server useful configuration steps
• Starting with MQTT protocol and ejabberd MQTT broker
• Getting started with WebSocket API in ejabberd
• Install and configure MariaDB with ejabberd
• Publish-Subscribe pattern and PubSub in ejabberd

Photo by Josh Hild on Unsplash