Planet Jabber

I have added support for aesgcm:// links on both iOS and Mac clients. There is still a bit of optimization to be done but it should be in the next client releases. It appears that the documentation for this functionality is wrong which made things a little tricky. It turns out the IV is 16 bytes and not 12 as it is written. Additionally unlike what the xep says, OMEMO uses GCM 128 and this uses GCM 256 and has a different key length.

There was an iOS App Store release. I mentioned in the release notes that there was a security fix. The specific fix was a bug that didn’t save user preferences when deselecting OMEMO keys in the UI. I am generally good about clarity with security changes (see the SSL changes) I will be better about it in the future.

I am working more on iOS at the moment. The chat UI is a bit more refined. I have adjusted the dates and the spacing between cells depending on the message before it. iOS users who have used Messages will find this familiar. I will work on aesgcm:// links next. It may not make it into next weeks release but likely the one after that.

In the last blog post, I took a closer look at how the Extended Triple Diffie-Hellman Key Exchange (X3DH) is used in OMEMO and which role PreKeys are playing. This post is about the other big algorithm that makes up OMEMO. The Double Ratchet.

The Double Ratchet algorithm can be seen as the gearbox of the OMEMO machine. In order to understand the Double Ratchet, we will first have to understand what a ratchet is.

Before we start: This post makes no guarantees to be 100% correct. It is only meant to explain the inner workings of the Double Ratchet algorithm in a (hopefully) more or less understandable way. Many details are simplified or omitted for sake of simplicity. If you want to implement this algorithm, please read the Double Ratchet specification.

A ratchet is a tool used to drive nuts and bolts. The distinctive feature of a ratchet tool over an ordinary wrench is, that the part that grips the head of the bolt can only turn in one direction. It is not possible to turn it in the opposite direction as it is supposed to.

In OMEMO, ratchet functions are one-way functions that basically take input keys and derives a new keys from that. Doing it in this direction is easy (like turning the ratchet tool in the right direction), but it is impossible to reverse the process and calculate the original key from the derived key (analogue to turning the ratchet in the opposite direction).

Symmetric Key Ratchet

One type of ratchet is the symmetric key ratchet (abbrev. sk ratchet). It takes a key and some input data and produces a new key, as well as some output data. The new key is derived from the old key by using a so called Key Derivation Function. Repeating the process multiple times creates a Key Derivation Function Chain (KDF-Chain). The fact that it is impossible to reverse a key derivation is what gives the OMEMO protocol the property of Forward Secrecy.

The above image illustrates the process of using a KDF-Chain to generate output keys from input data. In every step, the KDF-Chain takes the input and the current KDF-Key to generate the output key. Then it derives a new KDF-Key from the old one, replacing it in the process.

To summarize once again: Every time the KDF-Chain is used to generate an output key from some input, its KDF-Key is replaced, so if the input is the same in two steps, the output will still be different due to the changed KDF-Key.

One issue of this ratchet is, that it does not provide future secrecy. That means once an attacker gets access to one of the KDF-Keys of the chain, they can use that key to derive all following keys in the chain from that point on. They basically just have to turn the ratchet forwards.

Diffie-Hellman Ratchet

The second type of ratchet that we have to take a look at is the Diffie-Hellman Ratchet. This ratchet is basically a repeated Diffie-Hellman Key Exchange with changing key pairs. Every user has a separate DH ratcheting key pair, which is being replaced with new keys under certain conditions. Whenever one of the parties sends a message, they include the public part of their current DH ratcheting key pair in the message. Once the recipient receives the message, they extract that public key and do a handshake with it using their private ratcheting key. The resulting shared secret is used to reset their receiving chain (more on that later).

Once the recipient creates a response message, they create a new random ratchet key and do another handshake with their new private key and the senders public key. The result is used to reset the sending chain (again, more on that later).

Principle of the Diffie-Hellman Ratchet.
Image by OpenWhisperSystems (modified by author)

As a result, the DH ratchet is forwarded every time the direction of the message flow changes. The resulting keys are used to reset the sending-/receiving chains. This introduces future secrecy in the protocol.

Chains

A session between two devices has three chains – a root chain, a sending chain and a receiving chain.

The root chain is a KDF chain which is initialized with the shared secret which was established using the X3DH handshake. Both devices involved in the session have the same root chain. Contrary to the sending and receiving chains, the root chain is only initialized/reset once at the beginning of the session.

The sending chain of the session on device A equals the receiving chain on device B. On the other hand, the receiving chain on device A equals the sending chain on device B. The sending chain is used to generate message keys which are used to encrypt messages. The receiving chain on the other hand generates keys which can decrypt incoming messages.

Whenever the direction of the message flow changes, the sending and receiving chains are reset, meaning their keys are replaced with new keys generated by the root chain.

An Example

I think this rather complex protocol is best explained by an example message flow which demonstrates what actually happens during message sending / receiving etc.

In our example, Obi-Wan and Grievous have a conversation. Obi-Wan starts by establishing a session with Grievous and sends his initial message. Grievous responds by sending two messages back. Unfortunately the first of his replies goes missing.

Session Creation

In order to establish a session with Grievous, Obi-Wan has to first fetch one of Grievous key bundles. He uses this to establish a shared secret S between him and Grievous by executing a X3DH key exchange. More details on this can be found in my previous post. He also extracts Grievous signed PreKey ratcheting public key. S is used to initialize the root chain.

Obi-Wan now uses Grievous public ratchet key and does a handshake with his own ratchet private key to generate another shared secret which is pumped into the root chain. The output is used to initialize the sending chain and the KDF-Key of the root chain is replaced.

Now Obi-Wan established a session with Grievous without even sending a message. Nice!

Initial Message

Now the session is established on Obi-Wans side and he can start composing a message. He decides to send a classy “Hello there!” as a greeting. He uses his sending chain to generate a message key which is used to encrypt the message.

Note: In the above image a constant is used as input for the KDF-Chain. This constant is defined by the protocol and isn’t important to understand whats going on.

Now Obi-Wan sends over the encrypted message along with his ratcheting public key and some information on what PreKey he used, the current sending key chain index (1), etc.

When Grievous receives Obi-Wan’s message, he completes his X3DH handshake with Obi-Wan in order to calculate the same exact shared secret S as Obi-Wan did earlier. He also uses S to initialize his root chain.

Now Grevious does a full ratchet step of the Diffie-Hellman Ratchet: He uses his private and Obi-Wans public ratchet key to do a handshake and initialize his receiving chain with the result. Note: The result of the handshake is the same exact value that Obi-Wan earlier calculated when he initialized his sending chain. Fantastic, isn’t it? Next he deletes his old ratchet key pair and generates a fresh one. Using the fresh private key, he does another handshake with Obi-Wans public key and uses the result to initialize his sending chain. This completes the full DH ratchet step.

Decrypting the Message

Now that Grievous has finalized his side of the session, he can go ahead and decrypt Obi-Wans message. Since the message contains the sending chain index 1, Grievous knows, that he has to use the first message key generated from his receiving chain to decrypt the message. Because his receiving chain equals Obi-Wans sending chain, it will generate the exact same keys, so Grievous can use the first key to successfully decrypt Obi-Wans message.

Sending a Reply

Grievous is surprised by bold actions of Obi-Wan and promptly goes ahead to send two replies.

He advances his freshly initialized sending chain to generate a fresh message key (with index 1). He uses the key to encrypt his first message “General Kenobi!” and sends it over to Obi-Wan. He includes his public ratchet key in the message.

Unfortunately though the message goes missing and is never received.

He then forwards his sending chain a second time to generate another message key (index 2). Using that key he encrypt the message “You are a bold one.” and sends it to Obi-Wan. This message contains the same public ratchet key as the first one, but has the sending chain index 2. This time the message is received.

Receiving the Reply

Once Obi-Wan receives the second message and does a full ratchet step in order to complete his session with Grevious. First he does a DH handshake between his private and the Grevouos’ public ratcheting key he got from the message. The result is used to setup his receiving chain. He then generates a new ratchet key pair and does a second handshake. The result is used to reset his sending chain.

Obi-Wan notices that the sending chain index of the received message is 2 instead of 1, so he knows that one message must have been missing or delayed. To deal with this problem, he advances his receiving chain twice (meaning he generates two message keys from the receiving chain) and caches the first key. If later the missing message arrives, the cached key can be used to successfully decrypt the message. For now only one message arrived though. Obi-Wan uses the generated message key to successfully decrypt the message.

Conclusions

What have we learned from this example?

Firstly, we can see that the protocol guarantees forward secrecy. The KDF-Chains used in the three chains can only be advanced forwards, and it is impossible to turn them backwards to generate earlier keys. This means that if an attacker manages to get access to the state of the receiving chain, they can not decrypt messages sent prior to the moment of attack.

But what about future messages? Since the Diffie-Hellman ratchet introduces new randomness in every step (new random keys are generated), an attacker is locked out after one step of the DH ratchet. Since the DH ratchet is used to reset the symmetric ratchets of the sending and receiving chain, the window of the compromise is limited by the next DH ratchet step (meaning once the other party replies, the attacker is locked out again).

On top of this, the double ratchet algorithm can deal with missing or out-of-order messages, as keys generated from the receiving chain can be cached for later use. If at some point Obi-Wan receives the missing message, he can simply use the cached key to decrypt its contents.

This self-healing property was eponymous to the Axolotl protocol (an earlier name of the Signal protocol, the basis of OMEMO).

Acknowledgements

Thanks to syndace and paul for their feedback and clarification on some points.

The yax.im service was unavailable to ~11% of our users since March 28th. This was a result of a bug in a community module that was triggered by our upgrade to prosody 0.11.

The yax.im XMPP service is running on the host xmpp.yaxim.org and uses SRV records to let clients know the actual host name. However, a certain percentage of clients (around 11%) are running on IT infrastructure from the last century and fail to obtain those SRV records. These clients fall back to connecting directly to the yax.im host instead.

To accomodiate the clients, we are using HAProxy and mod_net_proxy to properly forward the connections from the web server to the XMPP server.

After the upgrade to prosody 0.11, mod_net_proxy failed to accept new connections from HAProxy, so that clients not using SRV were caught in an infinite “Connecting…” loop. Unfortunately, this was not detected by our technical monitoring, and only was brought up recently by an administrator’s relative.

The communication between HAProxy and mod_net_proxy has been restored today, and the service should be fully available from now on. We will also improve our monitoring to make sure that similar problems are detected in time in the future.

@gdt wrote:

The Ignite Realtime community is happy to announce the immediate release of version 1.7.2 of the Search plugin for Openfire!

The search plugin adds Jabber Search (XEP-0055) capabilities to Openfire.

This update provides better compatibility with XEP-0004, as well as other minor changes.

Your instance of Openfire should automatically display the availability of the update in the next few hours. Alternatively, you can download the new release of the plugin at the Search plugin archive page.

For other release announcements and news follow us on Twitter .

Posts: 1

Participants: 1

Read full topic

There is a new iOS. I have been focusing on polishing up the chat screen, which hasn’t seen a lot of change in about 4 years. There is a lot of things going on in the screenshot below, all of them are improvements for the better. We have headers for the day now so the dates on each row are smaller. I am working on compressing cell spacing and using the space to convey time difference between messages (this is a WIP). Link detection is improved. The OMEMO locks are now in the cells. Messages consistently show the OMEMO lock. I am also gone back and adjusting the point count on all spacing to keep them visually consistent.

Still some spacing issues, but it is getting closer to my vision

@gdt wrote:

The Ignite Realtime community is happy to announce the immediate release of version 1.0.3 of the Bookmarks plugin for Openfire!

The Bookmarks plugin will broadcast messages to all users or to specific groups in Openfire.

This update fixes an issue with support for older versions of Openfire.

Your instance of Openfire should automatically display the availability of the update in the next few hours. Alternatively, you can download the new release of the plugin at the Bookmarks plugin archive page.

For other release announcements and news follow us on Twitter .

Posts: 1

Participants: 1

Read full topic

This is the first part of a small series about the cryptographic building blocks of OMEMO. This post is about the Extended Triple Diffie Hellman Key Exchange Algorithm (X3DH) which is used to establish a session between OMEMO devices.
Part 2: Closer Look at the Double Ratchet

In the past I have written some posts about OMEMO and its future and how it does compare to the Olm encryption protocol used by matrix.org. However, some readers requested a closer, but still straightforward look at how OMEMO and the underlying algorithms work. To get started, we first have to take a look at its past.

OMEMO was implemented in the Android Jabber Client Conversations as part of a Google Summer of Code project by Andreas Straub in 2015. The basic idea was to utilize the encryption library used by Signal (formerly TextSecure) for message encryption. So basically OMEMO borrows almost all the cryptographic mechanisms including the Double Ratchet and X3DH from Signals encryption protocol, which is appropriately named Signal Protocol. So to begin with, lets look at it first.

The Signal Protocol

The famous and ingenious protocol that drives the encryption behind Signal, OMEMO, matrix.org, WhatsApp and a lot more was created by Trevor Perrin and Moxie Marlinspike in 2013. Basically it consists of two parts that we need to further investigate:

• The Extended Triple-Diffie-Hellman Key Exchange (X3DH)
• The Double Ratchet Algorithm

One core principle of the protocol is to get rid of encryption keys as soon as possible. Almost every message is encrypted with another fresh key. This is a huge difference to other protocols like OpenPGP, where the user only has one key which can decrypt all messages ever sent to them. The later can of course also be seen as an advantage OpenPGP has over OMEMO, but it all depends on the situation the user is in and what they have to protect against.

A major improvement that the Signal Protocol introduced compared to encryption protocols like OTRv3 (Off-The-Record Messaging) was the ability to start a conversation with a chat partner in an asynchronous fashion, meaning that the other end didn’t have to be online in order to agree on a shared key. This was not possible with OTRv3, since both parties had to actively send messages in order to establish a session. This was okay back in the days where people would start their computer with the intention to chat with other users that were online at the same time, but it’s no longer suitable today.

Note: The recently worked on OTRv4 will not come with this handicap anymore.

The X3DH Key Exchange

Let’s get to it already!

X3DH is a key agreement protocol, meaning it is used when two parties establish a session in order to agree on a shared secret. For a conversation to be confidential we require, that only sender and (intended) recipient of a message are able to decrypt it. This is possible when they share a common secret (eg. a password or shared key). Exchanging this key with one another has long been kind of a hen and egg problem: How do you get the key from one end to the other without an adversary being able to get a copy of the key? Well, obviously by encrypting it, but how? How do you get that key to the other side? This problem has only been solved after the second world war.

The solution is a so called Diffie-Hellman-Merkle Key Exchange. I don’t want to go into too much detail about this, as there are really great resources about how it works available online, but the basic idea is that each party possesses an asymmetric key pair consisting of a public and a private key. The public key can be shared over insecure networks while the
private key must be kept secret. A Diffie-Hellman key exchange (DH) is the process of combining a public key A with a private key b in order to generate a shared secret. The essential trick is, that you get the same exact secret if you combine the secret key a with the public key B. Wikipedia does a great job at explaining this using an analogy of mixing colors.

Deniability and OTR

In normal day to day messaging you don’t always want to commit to what you said. Especially under oppressive regimes it may be a good idea to be able to deny that you said or wrote something specific. This principle is called deniability.

Note: It is debatable, whether cryptographic deniability ever saved someone from going to jail, but that’s not scope of this blog post.

At the same time you want to be absolutely sure that you are really talking to your chat partner and not to a so called man in the middle. These desires seem to be conflicting at first, but the OTR protocol featured both. The user has an IdentityKey, which is used to identify the user by means of a fingerprint. The (massively and horribly simplified) procedure of creating a OTR session is as follows: Alice generates a random session key and signs the public key with her IdentityKey. She then sends that public key over to Bob, who generates another random session key with which he executes his half of the DH handshake. He then sends the public part of that key (again, signed) back to Alice, who does another DH to acquire the same shared secret as Bob. As you can see, in order to establish a session, both parties had to be online. Note: The signing part has been oversimplified for sake of readability.

Normal Diffie-Hellman Key Exchange

From DH to X3DH

Perrin and Marlinspike improved upon this model by introducing the concept of PreKeys. Those basically are the first halves of a DH-handshake, which can – along with some other keys of the user – be uploaded to a server prior to the beginning of a conversation. This way another user can initiate a session by fetching one half-completed handshake and completing it.

Basically the Signal protocol comprises of the following set of keys per user:

IdentityKey (IK)	Acts as the users identity by providing a stable fingerprint
Signed PreKey (SPK)	Acts as a PreKey, but carries an additional signature of IK
Set of PreKeys ({OPK})	Unsigned PreKeys

If Alice wants to start chatting, she can fetch Bobs IdentityKey, Signed PreKey and one of his PreKeys and use those to create a session. In order to preserve cryptographic properties, the handshake is modified like follows:

DH1 = DH(IK_A, SPK_B)
DH2 = DH(EK_A, IK_B)
DH3 = DH(EK_A, SPK_B)
DH4 = DH(EK_A, OPK_B)

S = KDF(DH1 || DH2 || DH3 || DH4)

EK_A denotes an ephemeral, random key which is generated by Alice on the fly. Alice can now derive an encryption key to encrypt her first message for Bob. She then sends that message (a so called PreKeyMessage) over to Bob, along with some additional information like her IdentityKey IK, the public part of the ephemeral key EK_A and the ID of the used PreKey OPK.

Visual representation of the X3DH handshake

Once Bob logs in, he can use this information to do the same calculations (just with swapped public and private keys) to calculate S from which he derives the encryption key. Now he can decrypt the message.

In order to prevent the session initiation from failing due to lost messages, all messages that Alice sends over to Bob without receiving a first message back are PreKeyMessages, so that Bob can complete the session, even if only one of the messages sent by Alice makes its way to Bob. The exact details on how OMEMO works after the X3DH key exchange will be discussed in part 2 of this series

X3DH Key Exchange TL;DR

X3DH utilizes PreKeys to allow session creation with offline users by doing 4 DH handshakes between different keys.

A subtle but important implementation difference between OMEMO and Signal is, that the Signal server is able to manage the PreKeys for the user. That way it can make sure, that every PreKey is only used once. OMEMO on the other hand solely relies on the XMPP servers PubSub component, which does not support such behavior. Instead, it hands out a bundle of around 100 PreKeys. This seems like a lot, but in reality the chances of a PreKey collision are pretty high (see the birthday problem).

OMEMO does come with some counter measures for problems and attacks that arise from this situation, but it makes the protocol a little less appealing than the original Signal protocol.

Clients should for example keep used PreKeys around until the end of catch -up of missed message to allow decryption of messages that got sent in sessions that have been established using the same PreKey.

Welcome to the XMPP newsletter.

If you have an article, tutorial or blog post you'd like us to include in the newsletter, please submit it on the XMPP wiki.

News

A design job has been posted at the Open Source Design community for Compliance Suite badges. It is an initiative that aims to create graphic elements for developers to use on their clients and servers to be able to identify compliance levels of such software.

A wiki page to keep track of integrations has been created at the XSF's Wiki. The idea behind it is to take a look at projects or services that support features like notifications but do not include XMPP as part of the integrations they support. This page can be used to follow integrations on all kind of software and make it visible so other people can work or promote them.

Subscribe to receive the next edition in your inbox

Software releases

Servers

• Tigase 8.0.0
• MongooseIM 3.3.0
• Metronome 3.12.x
• Openfire CallbackOnOffline plugin 1.2.1 and REST API 1.3.9

Libraries

• Smack 4.3.3

There are new Mac and iOS betas. Barring any issues this will be the next release. This primarily has OMEMO fixes and a few UI improvements. There will be more updates.

Starting today, yaxim is switching its protocol foundation from the deprecated exchange of clumsy and inefficient XML streams to the modern and elegant combination of HTTP and JSON/REST, the Matrix protocol.

Protocol History and Comparison

The XMPP protocol celebrated its 20th birthday early this year. The Matrix followed two months later and is currently in the middle of its own celebration. Some fifteen years later, a small company decided to use the strong brand value of the Matrix name to reinvent XMPP with a modern facade.

Evil voices claim that MATRIX stands for Monolithic, Awefully Trendy Re-Implementation of XMPP, and there is some truth to this, if we compare the words of the respective founding fathers:

Jabber is a new project I recently started to create a complete open-source platform for Instant Messaging with transparent communication to other IM systems(ICQ, AIM, etc).

I think they missed the bit where Matrix is called Matrix because it bridges (matrixes) the existing networks (Slack, IRC, Telegram, Discord, XMPP, etc) in, rather than needing to convince everyone to join.

However, the Matrix protocol has outgrown Vector Ltd, there is a version 1.0 0.4 specification, and even a Matrix.org Foundation.

This is much superior to XMPP, which is based on some arcane specifications maintained by a bunch of grey beards, plus a separate organisation for protocol extensions. In addition, Matrix supports working over 100 bits per second connections, while XMPP only gives you 75 bps.

The monolithic protocol is another huge advantage compared to many hundreds of optional extensions, and the rumors of Matrix fragmentation are a blatant lie.

Enter the Matrix

Therefore, the yaxim developers have decided to take the blue pill, to move forward, and to use the better and more modern and mobile friendly polling based HTTP scheme. Starting with the current beta release, you can enter Matrix chat rooms and talk to users on the Matrix.

The legacy XMPP protocol is remaining in the release for now, but will be removed in the near future to reduce the bloat of yaxim. You will be able to migrate your contacts to your new Matrix account by using the Bifröst bridge.

In parallel, we are working on switching yax.im from prosody to Synapse. The data transition is already completed, and we are only waiting for the data center provider to add 512GB of RAM to the machine before we can switch over.

@gdt wrote:

The Ignite Realtime community is happy to announce the immediate release of version 1.9.2 of the Broadcast plugin for Openfire!

The Broadcast plugin will broadcast messages to all users or to specific groups in Openfire.

This update fixes an issue with support for newer versions of Openfire.

Your instance of Openfire should automatically display the availability of the update in the next few hours. Alternatively, you can download the new release of the plugin at the Broadcast plugin archive page.

For other release announcements and news follow us on Twitter .

Posts: 1

Participants: 1

Read full topic

@guus wrote:

We’re happy to announce the immediate availability of a new plugin for Openfire: the “Random Avatar” plugin!

This plugin adds a webservice to Openfire, from which avatars can be obtained. Adding random data to the end of the request will give you a different avatar. Give it a try on https://www.igniterealtime.org/randomavatar/ to see what’s available!

A sample of the art that’s used:

These, and other, avatars are created by artists Darius Dan and Freepik from Flaticon.

For other release announcements and news follow us on Twitter!

Posts: 1

Participants: 1

Read full topic

@gdt wrote:

The Ignite Realtime community is happy to announce the immediate release of version 2.4.1 of the Hazelcast plugin for Openfire!

The Hazelcast plugin adds support for running multiple redundant Openfire servers together in a cluster.

This update improves the ability for plugins to create their own cluster-wide Caches, as well as a number of other small changes.

Your instance of Openfire should automatically display the availability of the update in the next few hours. Alternatively, you can download the new release of the plugin at the Hazelcast plugin archive page.

For other release announcements and news follow us on Twitter .

Posts: 1

Participants: 1

Read full topic

Today, we are rebranding and expanding our well-received ejabberd SaaS platform!
The new name is Fluux, supporting both XMPP & MQTT, in the cloud, as a single service with a unique and simple business model.

From XMPP to Realtime Standard-based Multi-protocol Service

ejabberd SaaS was launched five years ago. It was one of the first XMPP software-as-services and still is the reference today. Five years later, it runs very reliably in production for many customers, providing service to millions of concurrent online users every day.

When we launched, it was the perfect tool to run a highly reliable, highly scalable mobile chat service for a great price. You received your dedicated servers, managed by ProcessOne, developers of ejabberd, with all the features provided by our Business Edition. You could even develop your own backend API to store most of the data on your own private servers and just enjoy ejabberd, the realtime service, in a “stateless” fashion, no strings attached.

Over time, however, our customers have been using the platform to go further than chat. We have helped them build game services with ejabberd. We have also seen customers using it to connect devices and build large scale Internet of Things projects.

We soon realized that both, our service and ejabberd, needed to look beyond just XMPP. We put a lot of effort and our skills from building a large scale messaging platform, to develop a clustered high performance MQTT server based on the core building bricks of ejabberd. We support geoclustering and the brand new MQTT 5. Our open test server is already widely used for home-made IoT. You can learn more in our MQTT server announcement.

Fluux: Offering XMPP & MQTT as a Service

The next natural step was to offer MQTT as part of our software-as-a-service platform. Today, we are ready to announce that MQTT is available on all our new instances, and upon request for our existing customers.

However, ejabberd name was always associated with XMPP. Jabber is the former name of the XMPP protocol. So, it also made sense to rebrand our ejabberd SaaS platform to a name that shows the support to a wider variety of realtime protocols. That’s why, starting today, ejabberd SaaS is called Fluux.

The business model remains the same one that our customers love. No per-user costs that end up very expensive. You pay for what you consume, measured in “Jabs”. For MQTT, it is even more straightforward, with fewer rules:

• 2 Jabs for authentication
• 1 Jab per MQTT packet published
• 1 Jab per recipient on MQTT publish
• 1 Jab per subscribe or unsubscribe
• 1 Jab per 15 minutes for inactive session.

And the greatest part? You can build hybrid projects using both XMPP & MQTT on the same platform and the same pricing plan. This is great for both gaming, IoT and mobile projects, using the best of both protocols to fit a use case.

Join and Build with Fluux

Fluux is feature-full and future-proof:

• Relies on standards, with no lock-in. We want you to stay with us because your are happy with the service, not because you now have no other choice.
• Enables innovative use cases through the availability of multiple protocols on a single platform.
• Takes away the pain of running a highly reliable and scalable service and lets you focus on your product.
• Keeps the costs predictable with a single unit of usage – the Jab.

You can start using MQTT, XMPP or both on Fluux today. Welcome to your new standard-based realtime platform!

I watch a lot of YouTube videos. So much, that it starts to annoy me, how much of my free time I’m wasting by watching (admittedly very interesting) clips of a broad range of content creators.

Logging out of my Google account helped a little bit to keep my addiction at bay, as it appears to prevent the YouTube algorithm, which normally greets me with a broad set of perfectly selected videos from recognizing me. But then again I use Google to log in to one service or another, so it became annoying to log in and back out again all the time. At one point I decided to delete my YouTube history, which resulted in a very bad prediction of what videos I might like. This helped for a short amount of time, but the algorithm quickly returned to its merciless precision after a few days.

Today I decided, that its time to leave Google behind completely. My Google Mail account was used only for online shopping anyways, so I figured why not use a more privacy respecting service instead. Self-hosting was not an option for me, as I only have a residential IP address on my Raspberry Pi and also I heard that hosting a mail server is a huge pain.

A New Mail Account

So I created an account at the Berlin based service mailbox.org. They offer emails plus some cloud stuff like an office suite, storage etc., although I don’t think I’ll use any of the additional services (oh, they offer an XMPP account as well :P). The service is not free as in free beer as it costs 1€ per month, but that’s a fair price in my opinion. All in all it appears to be a good replacement for all the Google stuff.

As a next step, I went through the long list of all the websites and shops that I have accounts on, scouting for those services that are registered on my Google Mail address. All those mail settings had to be changed to the new account.

Mail Extensions

Bonus Tipp: Mailbox.org has support for so called Mail Extensions (or Plus Extensions, I’m not really sure how they are called). This means that you can create a folder in your inbox, lets say “fsfe”. Now you can change your mail address of your FSFE account to “username+fsfe@mailbox.org”. Mails from the FSFE will still go to your “username@mailbox.org” mail account, but they are automatically sorted into the fsfe inbox. This is useful not only to sort mails by sender, but also to find out, which of the many services you use messed up and leaked your mail address to those nasty spammers, so you can avoid that service in the future.

This trick also works for Google Mail by the way.

Deleting (most) the Google Services

The last step logically would be to finally delete my Google account. However, I’m not entirely sure if I really changed all the important services over to the new account, so I’ll keep it for a short period of time (a month or so) to see if any more important mails arrive.

However, I discovered that under the section “Delete Services or Account” you can see a list of all the services which are connected with your Google account. It is possible to partially delete those services, so I went ahead and deleted most of it, except Google Mail.

Additional Bonus Tipp: I use NewPipe on my phone, which is a free libre replacement for the YouTube app. It has a neat feature which lets you import your subscriptions from your YouTube account. That way I can still follow some of the creators, but in a more manual way (as I have to open the app on my phone, which I don’t often do). In my eyes, this is a good compromise

I’m looking forward to go fully Google-free soon. I de-googled my phone ages ago, but for some reason I still held on to my Google account. This will be sorted out soon though!

De-Googling your Phone?

By the way, if you are looking to de-google your phone, Mike Kuketz has a great series of blog posts about that topic (in German though):

• Android ohne Google: Take back control! Teil 1
• LineageOS: Take back control! Teil 2

Happy Hacking!

I have closed a few ui and Muc related issues. You should start to see regular iOS and Mac public betas rolling out again this week. I hope to have the next Monal release next week. OMEMO was a big update and but I am planning on getting back into the bi-weekly release schedule. Most of the changes are small tweaks and bug fixes related to the UI and UX. This will be my focus for a while rather than new technical features.

@gdt wrote:

The Ignite Realtime community is happy to announce the immediate release of version 1.3.9 of the REST API plugin for Openfire!

The REST API plugin provides the ability to manage Openfire by sending an REST/HTTP request to the server.

This update fixes an issue with incompatibility when installed on Openfire 4.2.

Your instance of Openfire should automatically display the availability of the update in the next few hours. Alternatively, you can download the new release of the plugin at the REST API plugin archive page.

For other release announcements and news follow us on Twitter .

Posts: 1

Participants: 1

Read full topic

@gdt wrote:

The Ignite Realtime community is happy to announce the immediate release of version 1.2.1 of the CallbackOnOffline plugin for Openfire!

The CallbackOnOffline plugin will detect when messages are sent to offline users, and instead sends an HTTP POST request to a pre-defined address with the details of the message as a JSON body.

This update fixes an issue with incompatibility when installed on Openfire 4.3.

Your instance of Openfire should automatically display the availability of the update in the next few hours. Alternatively, you can download the new release of the plugin at the CallbackOnOffline plugin archive page.

For other release announcements and news follow us on Twitter.

Posts: 1

Participants: 1

Read full topic

We recently announced our intention to become a Facebook-free business. We are happy to report that all our Facebook Pages have now been permanently deleted, and all Facebook widgets and buttons have been removed from our websites.

But it wasn’t our first step towards minimising corporate surveillance for our users. In fact, for several years now on all our websites we have a rule to avoid external resources (like public CDNs), avoid externally linked scripts and not use excessive tracking technologies.

Our ProcessOne homepage uses a single session cookie. Our WordPress blog doesn’t create any cookies or use local storage at all. Currently, the only external script and tracking technology we use comes from, you guessed it, google-analytics.com. However, even upon implementing Google Analytics (GA), we made a choice to minimise its impact on our users.

We modified the script to disable IP detection, disable cookies and local storage, and distinguish our visitors just by a custom fingerprint that is useless to GA scripts on other sites and domains. This way, our users can’t be tracked from site to site. The customisations are based on this cookieless-google-analytics repo I cooked up back in 2014.

However, as you may imagine, Google goes to some lengths to make this anonymisation process difficult. On our new XMPP, MQTT & SIP realtime platform called Fluux we wanted to implement the latest version of the GA script. However, the so called gatag.js has removed some of the useful customisation options I mentioned earlier.

For example, you can’t fully disable cookies or local storage. This decision was probably made by the same person who decided that YouTube embeds from youtube-nocookie.com domain store all of the old-school cookies inside the local storage. Technically, it’s indeed nocookie, but in fact it’s still tracking users, with values named like yt-remote-device-id…

In the end, we decided to keep the previous, cookie-less version of the GA script. Additionally, we removed the YT embed in favour of an inline base64-encoded JPG. We are also testing Vimeo embeds in our blog posts. They do use cookies, but they increase overall decentralisation.

When browsing ProcessOne sites and services, you can be sure we spent a significant amount of time to minimise what we collect, or do not track you at all. Because we care about your privacy as we care about our own.

It’s very hard to stay untracked while browsing the web. Google, Facebook and many others have their business models based on extensive tracking. But a lot of fault lies on us, the Webmasters, for using seemingly helpful and time-saving technologies that are brilliant and free – but are they? In the end, if we are not careful, we pay with our data, and the data of our users. And all the nasty consequences that come with it.

We can change that state, one site at a time, with just a little bit of effort and cooperation. We can minimise the exposure of our users. We can switch to locally stored code and customised scripts. We can do more. We can do better.

Marek Foss
Webmaster at ProcessOne

ProcessOne curates two monthly newsletters – tech-focused Real-time Stack and business-focused Real-time Enterprise. Here are the articles concerning tech aspects of real-time development we found interesting in Issue #21. To receive this newsletter straight in your inbox on the day it’s published, subscribe here.

Releasing ejabberd 19.02: the MQTT Edition

This new ejabberd 19.02 release includes new major features, but also several improvements and bug fixes. The biggest news is the introduction of MQTT support.

Introducing Fluux: XMPP & MQTT as a Service

Today, we are rebranding and expanding our well-received ejabberd SaaS platform! The new name is Fluux, supporting both XMPP & MQTT, in the cloud, as a single service with a unique and simple business model.

Introduction to MQTT with IoT Studio

MQTT is one of the main protocol designed to power the Internet of Things. In this talk, we introduce the concepts being MQTT protocol design. If you have any questions regarding MQTT or suggestions of topics for the next IoT Studio videos, please let us know in the comments.

HTTP Recorder and Mock Library

If you need to write tests for code involving lot of API and web page scrapping, you often end up saving pages as fixtures and loading those fixtures to try injecting them in your code to simulate the query.

Install Ejabberd on CentOS 7

In today’s age, the age of security, many organizations like to have control of their communications. Fortunately, they have an open, secure and reliable protocol like XMPP.

MQTT Client Load Balancing With RabbitMQ and Spring Cloud

MQTT is a machine-to-machine (M2M), IoT connectivity protocol. It was designed as an extremely lightweight publish and subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium.

No, the Problem Isn’t “Bad Coders”

In this article, the author is going to look at a recent bug that was caught by the Rust compiler, which shows that not only is this assertion unreasonable but virtually impossible for reasons that haven’t been discussed.

@Flow wrote:

The Ignite Realtime developer community is happy to announce the availability of Smack 4.3.3.

Please have a look at the Changelog for a list of fixes and improvements. As with all patchlevel releases of Smack, it can act as drop in replacement for previous Smack 4.3 release because there are no API changes.

We like to thank everyone who contributed by reporting bugs or giving suggestions. Special thanks goes to people who contributed code:

$ git shortlog -sn 4.3.1..4.3.2
    19  Florian Schmaus
     1  Georg Lukas

More information about how to use the 4.3 release series can be found one the Smack 4.3 README.

Posts: 1

Participants: 1

Read full topic

Have you ever tried to use Mnesia with datasets larger than your transient memory? Are you confused with Erlang data types stored within Mnesia? We know some of you were. That is why we answered these problems by introducing something that is familiar to mainstream developers and also efficient with larger datasets at the same time - i.e. RDBMS backends for our PubSub plugin. Now, lets bundle that up with full support for XEP-0178, as well as, a RabbitMQ backend for our Event Pusher for a more complete package. Ladies and gentlemen welcome MongooseIM version 3.3.0.

Relations meet PubSub

The Publish-Subscribe plugin was one of our main focal points for recent weeks thanks to our friends from Safaricom Alpha, who have sponsored this work. The rationale behind implementing it is simple, and yet not obvious, so let’s dig in.

The XMPP protocol features an extension called Personal Eventing Protocol which is a special case of PubSub, where any entity (e.g. a user) may become a PubSub node. In turn, this can be used for many purposes. For instance, a client may publish information to its contacts about the music currently being played. Alternatively, it may announce microblog entries (e.g. personal Twitter). Nevertheless, end to end encryption is the thing of importance to us and many of MongooseIM users in regard to PEP.

End to end encryption is often a selling point for many pieces of modern IM software. For example, the popular TLS encryption ensures that nobody will be able to eavesdrop on your communication with a bank website. Of course, there are many more properties but the crucial fact remains: it secures the data you exchange with the server. Therefore, when you connect to MongooseIM with TLS enabled your transmission is safe. You have to bear in mind that everything you write is still readable on the server side and if you would like to improve your privacy even further - you need to encrypt your message content as well.

There are several protocols you can use to achieve it, but in most cases, you will need a way to announce public data (e.g. public key) that others may use to establish a secure session with your device. This is where PEP comes in. It provides a facility for storage and distribution that can hold, for instance, OMEMO keys and metadata to your contacts.

Since this data is retrieved and updated fairly often, we have realised that a classic Mnesia backend is no longer sufficient for this purpose and we have put a lot of work into developing an efficient RDBMS backend for PubSub. Besides performance in high volume scenarios, it allows developers to use databases other than Mnesia, ones they are more familiar with.

During the development, we have been also able to pinpoint bottlenecks in the core PubSub code. That is why we have the parallelised distribution of PubSub messages.The pre-3.3 extension used only a single Erlang process to handle all requests, as well as, the broadcasts triggered by them. Currently, the notification distribution is done by a new, short-lived process for every request. Requests themselves are still processed in a single queue - since parallel execution led to transaction deadlocks in Mnesia - but the extension may be configured to process them in several queues. That fixes the issue we have observed on several occasions, where the PubSub process was simply overwhelmed with messages with no reasonable overflow control and back pressure what greatly impaired user experience.

Standardised PKI

Password-based authentication is still a basic method in many places. Why? Try remembering your RSA 4096-bit public key, not to mention the whole public+private key pair. (yes, we know about Keepass; but you most probably have it configured to use a master password, right?)

The PKI authentication is the current industry standard though. It is more secure than a private-public key pair and, if implemented properly, much more convenient for the average user to use. Support for this method debuted in MongooseIM 2.2.x and received a batch of improvements in every subsequent release. This one adds one of the last important pieces i.e. compliance with XEP-0178, which is an official PKI authentication method specification for XMPP. It describes how the SASL EXTERNAL mechanism should behave. In other words, it describes which certificate fields should be used in the process and how.

Pre-3.3 implementation verified only the Common Name field, while now it verifies xmppAddr fields (there may be more than one such field) with CN optionally used as a fallback. What is more, a full JID is verified instead of just the username.

Integration with RabbitMQ

The Event Pusher extension emerged in MongooseIM 2.1.x as a unification of several channels that our server used to deliver data to external endpoints - e.g. delivering user messages to a push notifications service. It has been extended over time and in MIM 3.3 it receives yet another backend: RabbitMQ.

It is especially beneficial for developers who need to digest IM events in an asynchronous manner. Since AMQP is a popular, powerful and pretty easy to learn protocol, it may be used to build a spam detection component. Events published via RabbitMQ may also be consumed for big data analysis (finding patterns in user preferences, behaviour etc.). These are only 2 examples. We imagine that every application may find innovative uses for a stream of events coming in from MongooseIM. What is more, using another Erlang-based piece of software ensures the reliability and performance of this tandem.

Consider the demo of the spam detection mechanism below to be an inspiration for you.

Changelog

Please feel free to read the detailed changelog. Here, you can find a full list of source code changes and useful links. Contributors Special thanks to our contributors: Test our work on MongooseIM 3.3.0 and share your feedback

1. Help us improve the MongooseIM platform:
2. Star our repo: esl/MongooseIM
3. Report issues: esl/MongooseIM/issues
4. Share your thoughts via Twitter
5. Download Docker image with new release
6. Sign up to our dedicated mailing list to stay up to date about MongooseIM, messaging innovations and industry news.
7. Check out our MongooseIM product page for more information on the MongooseIM platform.