There’s a new space to watch on our website – ProcessOne Careers! We are expanding and at the moment we have an exciting new job opportunity for a Go Backend Developer.

Video of my talk on philosophy as a foundation for success is now available, in two parts: the talk itself and the question-and-answer session. Thanks to Prof. Mitzi Lee for making the recording and posting it to YouTube!...

There’s a new space to watch on our website – ProcessOne Careers! We are expanding and at the moment we have an exciting new job opportunity for a Javascript / Typescript Frontend Developer.

I have updated the privacy policy. I am working on other changes to make the iOS client GDPR compliant. The other apps should return to EU this week. 

@Flow wrote:

The Ignite Realtime developer community is happy to announce the availability of Smack 4.3.1.

This is a patchlevel release which contains mostly bugfixes. Please have a look at the Changelog. As with all patchlevel releases of Smack, it can act as drop in replacement for previous Smack 4.3 release because there are no API changes.

We like to thank everyone who contributed by reporting bugs or giving suggestions. Special thanks goes to people who contributed code:

$ git shortlog -sn 4.3.0..4.3.1
    12  Florian Schmaus
     1  damencho
     1  spslinger

More information about the 4.3 release can be found one the release’s Readme.

Posts: 1

Participants: 1

Read full topic

In Book Two, Chapter Three of the Nicomachean Ethics, Aristotle distinguishes three forms of the good: what is pleasurable or enjoyable (τό ἡδύ), what is useful or advantageous (τό συμφέρων), and what is right or beautiful (τό καλόν). In other places in his writings on both ethics and biology, he distinguishes three forms of reaching or desire (ὄρεξις): craving or appetite (ἐπιθυμία), feeling or passion (θυμός), and decision or will (βούλησις). Various interpreters (such as John Cooper in his book Reason and Emotion) have attempted to align these two sets of three, whereas other interpreters (such as Giles Pearson in his book Aristotle on Desire) have criticized such attempts as forced, misguided, and not supported by the textual evidence. Although I tend to agree with the integrators because I believe Aristotle was a fairly systematic thinker, I think Cooper's connection of θυμός with τό καλόν isn't right. Instead, it seems to me that craving or appetite is for the pleasant or enjoyable, feeling or passion is for the useful or advantageous, and decision or will is for what's right or beautiful....

I like to have control over my stuff. For that reason I’ve been running my own XMPP server and my own little Nextcloud instance for a long time. The next logical step would be to self-host my blog.

For the last two years my blog was hosted by the FSFE and it was a great, smooth experience. However, I figured that hosting my blog myself would give me even more control. I can install my own plugins, have a wider range of themes available and so on and so forth. For that reason I chose to migrate my blog.

From now on, you can find my postings on blog.jabberhead.tk! I hope to see you all over there

Happy Hacking!

The new version of our messaging platform, MongooseIM 2.1.0 has been recently released. MongooseIM 2.1.0, as a stronger platform for better chat experience is all about quality and building trust in the consistency and performance of our solution. It introduces tools and improvements making access to MongooseIM features easier.

This tutorial will take you through the details of MongooseICE - help your ops deploy and your devs build a STUN and TURN server for NAT traversal and stream relay. This “How To” will help you keep your infrastructure coherent and perfectly integrated, in order to deliver new features faster and with more confidence over time.

Introduction

Who is this document for?

This tutorial presents our TURN/STUN server in action. You get to see how to set up and configure MongooseICE and examine a system utilising its many talents.

Are you in need of an application requiring NAT traversal? Want to see how a TURN and STUN server would handle it? Or maybe you just like to tinker with interesting technologies and experience setting them up first hand?

If that’s the case, this tutorial is for you.

What is the end result of this tutorial?

At the end of the tutorial you will have a working environment with two peers, one sending a live video to another. The peer-to-peer communication will not be obstructed by any NATs that may occur in the background. The live video stream is only an example here - there are many possible use cases for peer-to-peer communication with NAT traversal. We chose to build an example application that shows video streaming, because it’s vivid, catchy and fun.

What do I need to begin?

Before you begin you have to prepare an environment for setting up the components used in this tutorial. Here’s a list of things you’ll need:

1. One Android phone (or at least an Android emulator). The video player in this tutorial is available only as an Android application.
2. RaspberryPi or any other device that is able to run Elixir code. Oh, and also has ffmpeg installed. We are going to use use RaspberryPi 3, to give this tutorial a hint of IoT.
3. At least one machine with a public IPv4 address. It is necessary, because both MongooseIM and MongooseICE servers need to be accessible by all devices that are used in this demo system. You could use a private, local IP address, but then you would need to ensure that your phone and the RaspberryPi are behind some kind of a NAT relative to this IP address.

Note: the demo will probably work without the NAT, but then there is no point in setting up a TURN server.

We are going to use 2 VPS (Virtual Private Server) that are located somewhere far far away, both having public IPv4 address. Let’s say MongooseICE is bound to 1.1.1.1, and MongooseIM to 2.2.2.2.

General architecture of the environment built with this tutorial

This is the architecture of the system we are building:

As we know by now, MongooseIM is bound to 2.2.2.2/myxmpp.com and MongooseICE to 1.1.1.1. We also have a RaspberryPi that is connected to a private network (so is behind some NAT) and an Android phone that is connected to an LTE network and also is behind the carrier’s NAT.

ICE notes

The end result of this tutorial not only uses MongooseICE and [MongooseIM] servers but also uses custom version of Mangosta-Android and [DemoStreamerICE]. Both projects are custom modified and custom made respectively in order to showcase the video streaming using the data relay capabilities provided by MongooseICE. The streaming itself, along with the signalling protocol, were prepared only for the case of this demo and are not a part of the platform. Those components exist only to visualize what can be achieved with MongooseICE and what can be built on top of it.

Setting up MongooseIM (signalling)

The ICE is nothing without signalling. The signalling protocol itself can be designed specifically for the application that is being deployed or can be implemented based on some standards, e.g. Jingle. Here, we chose to implement the simplest signalling possible, i.e. sending relay addresses via XMPP messages. No matter if we decide to go with this approach or with Jingle, we can use the MongooseIM XMPP server as a transport layer for the signalling. In order to enable signalling we need an instance of MongooseIM running with the simplest configuration, since the only thing we need from it is to provide us with means to communicate between two peers.

Configuration

You can find MongooseIM installation instructions on this page. Once you have cloned the repository and compiled the project, you need to modify the ejabberd.cfg config file (you can find this file at $REPO/_build/prod/rel/mongooseim/etc/ejabberd.cfg, where $REPO is a top-level directory of cloned repo). You can use this configuration file and modify the relevant part:

%%%% ICE DEMO %%%%
{hosts, ["localhost", "myxmpp.com"] }.

This sets the virtual hostname of the XMPP server, so that you can register users in this domain. After that, you can start MongooseIM with

$REPO/_build/prod/rel/mongooseim/bin/mongooseimctl start

Users

After we finish setting up [MongooseIM], we need to register some users. For this demo we need two users: movie@myxmpp.com and phone@myxmpp.com, for RaspberryPi and the Android phone respectively. In order to do that, type:

$REPO/_build/prod/rel/mongooseim/bin/mongooseimctl register phone myxmpp.com xmpp_password
$REPO/_build/prod/rel/mongooseim/bin/mongooseimctl register movie myxmpp.com xmpp_password

on the machine that has [MongooseIM] installed.

As you can see here, we have created those two users, both with the password xmpp_password for simplicity.

Setting up MongooseICE (TURN/STUN server)

Now, since MongooseIM handles the signalling, we need the TURN relay and the STUN server to send peer-to-peer data. For that we are going to use the star of this tutorial - MongooseICE.

How to get and configure

The whole documentation that describes all options and deployment methods, can be found on the project’s github page. Let’s get to it! (this command assumes that we are on the server for MongooseICE and that it has Docker installed):

docker run -it --net=host -e "MONGOOSEICE_UDP_RELAY_IP=1.1.1.1" -e "MONGOOSEICE_STUN_SECRET=secret" -e "MONGOOSEICE_UDP_REALM=myrelay" mongooseim/mongooseice:0.4.0

This command starts the MongooseICE server in the Docker container, attaching its virtual network interface to the network interface of the host machine the Docker deamon is running on. There are three important configuration options we have to set via environment variables:

• MONGOOSEICE_UDP_RELAY_IP - This is the IP address that MongooseICE provides data relay on. This should be set to public IPv4 address.
• MONGOOSEICE_STUN_SECRET - This is a secret password that TURN clients need to provide to connect to this server.
• MONGOOSEICE_UDP_REALM - This is just a name for your TURN relay.

And that’s it! MongooseICE is now ready to roll!

Setting up Mangosta-Android

How to get and install

The source code of the video-stream-demo-enabled Mangosta-Android can be found on the ice_demo_kt branch. If you want to tinker with it and compile it yourself, you can do that. All you need is Android Studio 2.3+. The compilation is pretty straightforward, so I’m not going to explain it here. If you are interested in how it works, most of the code is in the inaka.com.mangosta.videostream package. If you don’t want to compile this application from source, you can just install this .apk on your phone and that’s it.

How to configure

Right after you start Mangosta-Android for the first time, you will need to login to your XMPP server. In order to do that, just enter the JID you have created for the phone (phone@myxmpp.com), the password (xmpp_password) and the server address (2.2.2.2 or myxmpp.com if you’ve set up the domain to actually point to this IP address), and then confirm by clicking “Enter”.

After we log in, we can start setting up the connection to the MongooseICE server we set up before. The process is shown on the screenshots below.

Mangosta_ICE_settings Mangosta test ICE connection Mangosta save ICE settings

On the “Configure ICE” screen we have to set up 5 fields:

1. TURN server address - IPv4 address of our MongooseICE
2. TURN Server port - since we did not set the port while configuring MongooseICE it uses a default one - 3478
3. TURN Realm - Realm name we have set via MONGOOSEICE_UDP_REALM variable. In our case it’s “myrelay”.
4. TURN username - Current version of MongooseICE ignores this, so you may leave it as is.
5. TURN password - The password that we have set via MONGOOSEICE_STUN_SECRET variable. In our case it’s “secret”

And that would be all. Now you can click “TEST CONNECTION” to, well…, test the connection. If everything works, you can “SAVE” the settings. Now your Mangosta-Android is ready to play streamed video, but we still need the source…

Setting up RaspberryPi

Let’s configure the video source now. In our case it will be a RaspberryPi with Elixir and ffmpeg installed running our ICE demo application.

The software

For this demo we provide a simple XMPP client that also is able to send live video stream using ffmpeg whenever other peer asks for it via XMPP. This client is written in Elixir, so we can run it from source quite easily.

How to get and configure

You can get the client’s sources here. For now we only need to run it, so let’s get to it (on our RaspberryPi):

git clone https://github.com/esl/ice_demo.git
cd ice_demo
mix deps.get
iex -S mix

After a while we should get into Elixir shell. In order to enable the streamer, we need to start it, providing some configuration options (in the Elixir shell):

opts = [
  jid: "movie@myxmpp.com",
  password: "xmpp_password",
  host: "myxmpp.com",
  turn_addr: "1.1.1.1:3784"
  turn_username: "username",
  turn_secret: "secret",
  video_file: "/home/pi/sintel.h264"
]
ICEDemo.start_movie(opts)

The first 3 options are all about connecting to the XMPP server - we use “movie@myxmpp.com” user that we created earlier. Next 3 options are about connecting to the MongooseICE server. Those are similar to ones we set in Mangosta-Android. The last one points to the video file that will be streamed on request. This file has to be raw, H.264-encoded, video-only file. If you are not sure how to get one, you can just use [this one]https://drive.google.com/file/d/0B48g-HBQ5xpxeVlUSks2bVRlLTQ/view) (pre-rendered Sintel, OpenBlender project). With this configuration, out RaspberryPi is ready to stream!

[### The end result

Playing the video

Now we finally can get out phone and start streaming the video! In order to do that, we have to click the “New video stream” button as shown on the screenshoots below, enter the JID of the RaspberryPi and confirm with the “Stream!” button.

1. Mangosta start streaming
2. Mangosta start streaming
3. Mangosta start streaming

Hopefully, now you can see the video on your own mobile device.

Participate!

Share your biggest pain points in your journey through app and business building, so that we can work together on how to solve them efficiently.

1. Subscribe to our newsletter
2. Check out our MongooseIM product page for more information on the MongooseIM platform.
3. Read our “Boost your engine of growth with chat and social value”
4. Star us on GitHub and follow us on Twitter

The new version of our messaging platform, MongooseIM 2.1.0 has been recently released. MongooseIM 2.1.0, as a stronger platform for better chat experience is about quality, accessibility and building trust in the consistency and performance of our solution. It introduces new tools and improvements making access to MongooseIM features easier.

In this tutorial, we will dive deep into the details of MongoosePush - help your ops deploy and your devs build push notifications in your chat system. This “How To” will help you keep your infrastructure coherent and perfectly integrated, in order to deliver new features faster and with more confidence over time.

How to set up Push Notifications with MongoosePush

MongooseIM server supports push notifications using FCM (Firebase Cloud Messaging) and APNS (Apple Push Notification Service) providers. Server side push notification support is fully compliant with XEP-0357 Push Notifications, which defines several components that need to work together in order to provide clients with working push notifications. The following list shows those components as defined in XEP-0357 and MongooseIM components that correspond to those entities:

1. XMPP Server in MongooseIM is enabled by module mod_push
2. App Server in MongooseIM is enabled by adding a push node type to mod_pubsub’s configuration
3. XMPP Push Service is implemented as a MongoosePush application
4. All these entities have to be enabled and properly configured in order to use push notifications. So let’s get to it, shall we?

Overall component architecture

The components that make push notifications possible in MongooseIM add up to the following architecture:

The diagram lists three domains in total - two for MongooseIM and one for MongoosePush. Note that this separation is not required, all three components can be on the same host with the same domain.

Configuring MongooseIM components

Firstly, let’s configure all the required MongooseIM components, step by step.

mod_push a.k.a. ‘XMPP Server’

The first component that we need to configure in MongooseIM is the mod_push module. This module communicates with XMPP clients directly in order to enable/disable notifications on per-client basis. The mod_push module is very easy to enable - just paste the following to your MongooseIM configuration file:

{mod_push, [
    {wpool, [{workers, 100}]}
]}.

And that’s basically it. You have just enabled the push notification support with 100 asynchronous workers that will handle all push notification related work. You can also control the format of the “sender” of the push notification (which ultimately becomes the title of push notification) and filter which messages will trigger the notification. In that case you need to create a plugin module that implements the mod_push_plugin behaviour and enable this plugin as specified in the mod_push documentation.

mod_pubsub with mod_push_service_mongoosepush a.k.a. 'App Server’

The next component to configure consist of two modules:

1. mod_pubsub with a push node type enabled that will act as a sink for push notifications generated by mod_push
2. mod_push_service_mongoosepush - a connector to MongoosePush application

mod_pubsub’s push node

According to the XEP-0357 Push Notifications, all notifications generated via the module we have just enabled (i.e. mod_push) have to be send to a push enabled publish-subscribe node. In order to allow clients to allocate such a node, we need to enable it in our mod_pubsub on the MongooseIM server that will communicate with the XMPP Push Service.

The minimal mod_pubsub’s configuration looks as follows:

{mod_pubsub, [
    {plugins, [<<"push">>]}}
]}.

Such configuration will enable the mod_pubsub with only one node type available: push. Please note that if you want use mod_pubsub as a 'normal’ publish-subscribe service, you just need to append the <<“push”>> node type to the plugins list. Also, it’s important to note, that the first node type on the plugins list. will be the default one (allocated when the client does not provide a node type in the node create stanza).

mod_push_service_mongoosepush

This module acts as a bridge between mod_pubsub that receives notifications from mod_push and passes those to MongoosePush which sends them to FCM and/or APNS. To enable this module type in your configuration:

{http_connections, [{mongoose_push_http,
    [{server, "https://localhost:8443"}]
}]}.

{mod_push_service_mongoosepush, [
    {pool_name, mongoose_push_http}
    {api_version, "v2"}
]}.

First, we create the HTTP pool for communicating with MongoosePush. Here, we assume that MongoosePush will be available on the localhost on port 8443 which is the default one. Next we enable mod_push_service_mongoosepush. First option is the name of the HTTP pool to use and the second one is the version of MongoosePush’s API (currently only “v2” is supported).

And that’s it, we’ve just completed the entire MongooseIM configuration. All we need to do now is to set up MongoosePush.

Starting MongoosePush

The easiest way to start MongoosePush is using its docker image. But before you can set MongoosePush up, you need a FCM application token and/or an APNS application certificate. You can get the FCM token here and the easiest way of getting an APNS application certificate is by running this script (please note that you need the certificate in pem format). After you get the FCM application token and/or the APNS application certificate, you can prepare to start MongoosePush. Firstly, prepare the following files structure:

If your FCM app token is MY_FCM_SECRET_TOKEN and you have the priv directory with all ceriticates in the current directory, start MongoosePush with the following command:

docker run -v `pwd`/priv:/opt/app/priv \
  -e PUSH_FCM_APP_KEY="MY_FCM_SECRET_TOKEN" \
  -e PUSH_HTTPS_CERTFILE="/opt/app/priv/ssl/rest_cert.pem" \
  -e PUSH_HTTPS_KEYFILE="/opt/app/priv/ssl/rest_key.pem" \
  -it --rm mongooseim/mongoose-push:latest

If you don’t want to use either APNS or FCM, you simply need to pass PUSH_APNS_ENABLED=0 or PUSH_FCM_ENABLED=0 respectively as additional env variables in your docker run command. For more advanced options and configuration please refer to “Quick start / Configuring” in MongoosePush’s README.md. When your MongoosePush docker is up and running, Push Notifications can be used in your MongooseIM instance.

Using push notifications on client side

There are just a few things the XMPP client application needs to receive the push notifications. See the diagram to examine the process described in this section along with the example notification flow:

Registering with a Push Service provider

Firstly, the client application has to get a device-specific token from the Push Service Provider (FCM or APNS). This process is different, depending on the platform, so please consult your Push Service Provider’s manual to see how to get this token. For example, here you can learn about setting up FCM on Android platform and here you can learn about setting up APNS on iOS platform. After this step, your application shall be able to receive FCM or APNS token - it will be required in the next step of this tutorial.

Setting up an XMPP pubsub node

Please note this first step is specific to the app-server your client application uses. In case of MongooseIM, you just need to allocate (create) a special PubSub node that will act as a gateway for all notifications sent by the XMPP chat server. Without any further ado, let’s configure the PubSub’s push node. In this example mypubsub.com is a domain of the MongooseIM server that has mod_pubsub enabled with the push node support. The client sends the following stanza to the server:

<iq type='set'
    to='pubsub.mypubsub.com'
    id='create1'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <create node='punsub_node_for_my_private_iphone' type='push'/>
  </pubsub>
</iq>

The pubsub.mypubsub.com will be used as a gateway for all notifications and will pass them through to the APNS and/or FCM.

The most important and only distinction from the standard node creation is the type='push' part of the create element. This denotes that you need a node that will handle your push notifications. Here we create a node called punsub_node_for_my_private_iphone. This node should be unique to the device and you may reuse nodes already created this way.

After this step, you need to have the pubsub host (here pubsub.mypubsub.com) and the node name (here: punsub_node_for_my_private_iphone).

Enabling push notifications

The next and the last step is to enable push notifications on the server that handles your messages (and have mod_push enabled). Let’s assume this server**** is available under the mychat.com domain. To enable push notifications in the simplest configuration, just send the following stanza:

<iq type='set' id='x43'>
  <enable xmlns='urn:xmpp:push:0' jid='pubsub.mypubsub.com' node='punsub_node_for_my_private_iphone'>
    <x xmlns='jabber:x:data' type='submit'>
      <field var='FORM_TYPE'><value>http://jabber.org/protocol/pubsub#publish-options</value></field>
      <field var='service'><value>apns</value></field>
      <field var='device_id'><value>your_pns_device_token</value></field>
      <field var='silent'><value>false</value></field>
      <field var='topic'><value>some_apns_topic</value></field>
    </x>
  </enable>
</iq>

We have now enabled push notifications to be send to the pubsub.mypubsub.com to the node punsub_node_for_my_private_iphone created in previous paragraph. In publish-options we have passed the service name that we are using (apns or fcm) and the device token (here: your_pns_device_token) that you received from you push notification service provider (as described in Registering with Push Service provider).

Those two options are the only ones required, but there are some others that are optional:

1. mode - which may be either prod or dev (default to prod). Decides which connection pool type on MongoosePush shall be used. This may be used when APNS on MongoosePush is configured to work with both production and development certificate.
2. click_action - action to perform when notification is clicked on the device. activity on Android and category on iOS. Please refer to your platform / push notification service provider for more info.
3. topic - currently only used with APNS. the value is passed to APNS as topic header. For more information please refer to APNS documentation.
4. silent - if set to true, all notifications will be “silent”. This means that only data payload will be send to push notifications provider with no notification. The data payload will contain all notification fields as defined in XEP-0357.

Disabling push notifications

Disabling push notifications is very simple. Just send the following stanza to your XMPP chat server:

<iq type='set' id='x44'>
  <disable xmlns='urn:xmpp:push:0' jid='pubsub.mypubsub.com' node='punsub_node_for_my_private_iphone'/>
</iq>

You may skip the node='punsub_node_for_my_private_iphone' to globally disable push notifications on all nodes that are registered from your JID. This may be used to disbale push notifications on all your devices.

Participate!

Share the biggest pain points in your journey through app and business building, so that we can work together on how to solve them efficiently.

1. Subscribe to our newsletter
2. Check out our MongooseIM product page for more information on the MongooseIM platform.
3. Read our “Boost your engine of growth with chat and social value”
4. Star us on GitHub and follow us on Twitter

Escalus 4.0.0 is the newest version of our XMPP client library for Erlang, a component of our MongooseIM platform of tools and services. Created as a tool for convenient testing of XMPP servers, it can also be used as a standalone Erlang application. In contrast to other XMPP clients, Escalus provides a rich testing API (assertions, stanza building…) and a high degree of flexibility that allows to completely redefine its behaviour.

It is used by Erlang Solutions for integration, load and stress testing of MongooseIM XMPP server. The newest version, Escalus 4.0.0, now supports Erlang 20 and offers the following features and improvements:

New, RapidXML-based XML parser

Escalus 4.0.0 includes a new version of our exml Erlang XML parser! The parser’s processing layer was rewritten from scratch, extensively profiled and optimised; as a result, the new parser is on average 5 times faster than before when encoding and decoding XML elements.

Included XML viewer

Files containing traced XMPP stanzas now include a powerful XML viewer by Sergey Chikuyonok. The viewer shows elements in a readable manner and supports collapsing, searching by name, XPath support, and more. Inspecting your XMPP traffic has never been easier!

Message pipelining

The new XML parser also supports message pipelining, which means that multiple stanzas can be safely sent and received at once by the client. This can greatly improve efficiency of many use cases and in the future will be used to speed up Escalus’ XMPP connection times by an order of magnitude.

Transport-level metadata for stanzas

The transport process used to receive and deliver XMPP stanzas can now pass on metadata that will be included with elements delivered to the Escalus’ client. Currently the metadata set by existing transport implementations is limited to a receive timestamp, but can be easily overridden and extended by a custom solution.

Other improvements

TCP connections are now set up with a nodelay option which decreases communication latency between Escalus and XMPP server. If you don’t like the change, Escalus 4.0.0 also supports easy overriding of TCP options that allows you to disable it, and more - including the nitty-gritty settings of buffer sizes or ToS flag.

A host of refactoring changes, subtle API improvements, bug fixes, performance enhancements, and an increased number of useful functions to inspect and construct your stanzas all lead to a smoother experience while testing all of your most important features.

Test our work on Escalus 4.0.0 and share your feedback

Check out our full release notes for Escalus 4.0.0 & 3.1.0 on GitHub. Play around and let us know what you think!

Help us improve the MongooseIM platform:

1. Star our repo: esl/escalus
2. Report issues: esl/MongooseIM/issues
3. Share your thoughts via Twitter: twitter.com/MongooseIM
4. Download Docker image with new release
5. Sign up to our dedicated mailing list to stay up to date about MongooseIM, messaging innovations and industry news.
6. Check out our MongooseIM product page for more information on the MongooseIM platform.

This summer the MongooseIM team have not had a second to be lazy - the development of MongooseIM 3.1 release took over what was supposed to be our downtime, when all major activities driven throughout the year slow down. We’re happy to say that in this time of leisure, the new release is packed with important features, improvements and excitement. Take a look at the version highlights and be part of this major step in creating a world-class communication server.

This time, a large part of the changes are focused on development efficiency, but rest assured, we’ve added new items you are going to appreciate.

In the “Test Runner” section, you get to learn all about a little tool we provided so that you can easily set the development environment up and run tests locally without Travis CI.

Our Inbox extension has got three big updates that push the boundaries of what’s currently offered in XMPP world. As a rule, we want to lead by example; less talk and more action. That is why working in cooperation with Forward, we decided to put forward an implementation of a popular use case as a unique feature in this software sector.

This release is also an important lesson for us. A lesson about edge cases and concurrency in integration testing. You don’t necessarily have to be an Erlang developer to benefit from the third section, but reading it allows you to learn with us.

The “Honorable Mentions” section may seem minor, but for some projects the items listed there can indeed make a difference! It’s a candy mix of different changes, so read carefully not to miss your favourite flavours!

Obviously, a single blog post is too small a space to tell a profound story about every new item in the changelog, so we encourage you to check it out. You can find a link at the bottom of this blog post.

Test Runner

The Travis CI is our main verification tool for the Pull Requests we create and merge. Whilst being convenient and critical for our process, it is not very handy for day-to-day development. It is very common to frequently execute limited subset of tests to ensure that a new piece of code we wrote works perfectly. However, waiting for Travis results every time would extend implementation time excessively as it executes multiple presets, while being a shared resource at the same time.

The test runner is a script that helps to set the development environment up, and run tests on a developer machine locally. The test runner shares a lot of code with our Travis CI build scripts, which ensures that test results are consistent between local and CI runs.

The test runner allows to choose which tests to run. It can be useful, when one of the tests is failing and we want to rerun it.

Since MongooseIM supports different database backends, the test runner is able to set a selected database up for you and configure MongooseIM. Simply put, it prepares the local environment before executing actual test suites.

The test runner supports a lot of options to customise your build and test execution scenario. It also supports shell autocompletion for option names and values.

We’ve prepared a recording for you that presents a basic usage of the script.

.

Please note that Asciinema allows you to pause and copy any text fragment you like, so it would be very easy for you to repeat the same steps.

New Inbox Features

Originally sponsored by and created for Forward, Inbox has been available as an open source extension for the last two months already. In MongooseIM 3.1, it returns with a bundle of fresh goodies. The first feature is MSSQL support.

Despite being less frequently used with MongooseIM compared to MySQL or PostgreSQL, it’s still an important piece of the architecture for many projects, especially those running in Azure cloud. We don’t want you to feel excluded, dear Microsoft users!

The second one is the support for classic MUC group chats. MUC Light was never intended as a complete replacement for original XEP-0045 solution. It means that numerous projects exists where mod_muc is still a better match than its younger sibling, and they may now benefit from inbox features as well!

Last but not least is the timestamp support. First of all, they are stored in DB and returned in Inbox query results. For those using mod_inbox from MIM 3.0: you’ll need to update your schemas but don’t worry - it isn’t very complicated. What’s more, a client may now request conversations from a certain time period and sort them by timestamp, both ascending and descending.

This is not our final word on this matter. You may expect further improvements to this extension in upcoming MongooseIM versions!

We’ve prepared a demo of the Inbox feature. It shows both the backed and the frontend side of it. The application used in the demo has been designed by Forward.

.

Lessons learnt regarding CI

OK, these are short and sweet but nevertheless important:

1. Avoid RPC in integration tests. They tend to time out in slow CI environments (such as Travis).
2. When test users exchange messages, always wait until they are received to ensure proper synchronisation.
3. On a slow machine, MSSQL SELECT query may return more than one row (even when retrieving by the exact primary key value) as a consequence of the transaction deadlock.
4. When you can’t use any other means of server state synchronisation, don’t use hardcoded sleep periods; replace them with an incremental backoff and verification in a loop. Sometimes you can’t predict whether a server state is updated properly in 500ms, 1000ms or 3000ms. Adding 5s waits everywhere may cause test suites to run veeery long.
5. Be careful about leaking presences between cases. This applies to XMPP testing. Best practice is to generate fresh user accounts for every scenario.
6. Some databases don’t support transactions so the new data may not be instantly available. For example, in the case of Riak (its Search function in particular) a delay between data insert and query is required.
7. Sometimes creating a schema in a DB may fail for the first time due to timing issues, so implement a retry loop in a DB provisioning scripts. This also applies to Riak.
8. Did I mention creating new user accounts for every test case? It actually applies not only to XMPP. With this practice, you won’t have to worry about possible leftovers of a user’s state.

Honorable mentions

ElasticSearch backend for Message Archive

Almost every MongooseIM module supports more than one type of backend. Usually it’s Mnesia, RDBMS and sometimes Riak. Message Archive Management is a noteworthy exception, as we’ve implemented RDBMS, Riak and Cassandra support for this module. Or “modules” actually, as it consists of over 30 Erlang files already.

It is a very special feature as it processes a vast amount of data and sometimes executes expensive queries. In order to ensure performance and match a project’s architecture, wide range of supported DB backends is essential.

It is our pleasure to announce that yet another backend has joined the family: ElasticSearch.

OTP 21 support

OTP 21.0 has been released ~1 month ago and we’ve added support for this version less than a week after! This is great news for all projects sticking to the most recent Erlang technology as pioneers in BEAM world. The new platform version brought not only improvements in regards to performance but also some incompatibilities that we’ve resolved, so MongooseIM still remains at a technological peak.

As a tradeoff, we’ve dropped official support for OTP 18.x. It should still be possible to compile 3.1 with this version with some minor code modifications, but we’re definitely moving forward. It has allowed us to get rid of non-typed maps specifications as an example. As a reminder, bear in mind, that MongooseIM always supports two most recent, stable OTP branches (currently these are 20.x and 19.x and one being under an active development, 21.x).

Jingle/SIP tutorial

SIP is a common choice for VoIP applications but certain XMPP features may be a very good match for such software. MongooseIM is able to liaise between these two worlds and now it’s easier than ever with significantly extended documentation (compared to the level in 3.0) and a tutorial on mod_jingle_sip usage.

Worker pool unification

Every developer writes a custom worker pool at some point of their career. Everyone. Certain MongooseIM components (the ones that use connection pools) were created with different preferred library in mind. As a result, we’ve ended up with many kinds of worker pools: cuesport, worker_pool and poolboy. It wasn’t only a matter of maintenance difficulty, but performance as well. As an example, cuesport supports only a simple round-robin job assignment algorithm, which is not optimal in every case. It also lacks inspection of any kind.

Given our experience gathered over the years, we’ve selected worker_pool as our primary library. It is very flexible and exposes a dedicated stats API. It was originally created by Inaka and it is actively maintained by its former developers at this present time.

For now, the changes are purely internal. Some projects may observe better performance but the primary goal was to prepare for a second round of unification. Stay tuned for more details in near future.

Changelog

Please feel free to read the detailed changelog. Here, you can find a full list of source code changes and useful links.

Contributors Special thanks to our contributors: @SamuelNichols @Beisenbek @GalaxyGorilla!

Test our work on MongooseIM 3.1 and share your feedback

Help us improve the MongooseIM platform:

1. Star our repo: esl/MongooseIM
2. Report issues: esl/MongooseIM/issues
3. Share your thoughts via Twitter: @MongooseIM
4. Download Docker image with new release
5. Sign up to our dedicated mailing list to stay up to date about MongooseIM, messaging innovations and industry news.
6. Check out our MongooseIM product page for more information on the MongooseIM platform.

Who will find this interesting

If you’re considering XMPP for your project but you are unsure if it can provide the functionality you need, you’ll eventually end up here:

http://xmpp.org/extensions/

I’m pretty sure you’ll be quite intimidated by such a long list of extensions. In some cases it will be pretty easy to find what you need. If you look for PubSub functionality, you’ll quickly notice “Publish-Subscribe”. Sometimes it’s not so obvious though. XMPP developers already know that in order to synchronise outgoing messages between several devices, they have to enable “Message Carbons”. Not very intuitive, isn’t it?

The aim of this blog post is to guide you towards proper XMPP technologies and solutions, given your specific use cases. I’ve worked with and deployed solutions powered by XMPP, such as MongooseIM, for years; so let me be your personal Professor Oak, providing a perfect “companion(s)” to work with and begin your journey in XMPP world. There are almost 400 XEPs, will you catch them all? ;)

The length of this article is caused not by a complexity of descriptions but by a count of use cases and features. :)

All numbers and information on implementation status are valid for March 2017.

What can you expect here?

For every use case, I will list XMPP features you definitely should consider using. Each one of them will be briefly described. The goal here is to understand the usefulness without reading whole specification. Besides that, each item will include MongooseIM’s module name providing discussed extension and example client implementations.

What you won’t find in this post

This post won’t cover any XMPP basics. It assumes you either know them already (what are JID, C2S, S2S, IQ, stanzas, stream etc.) or you intend to learn them from some other guide, like the excellent (iOS) tutorial written by Andres Canal Part 1, Part 2). It’s more of a cookbook, not Cooking For Dummies.

ToC

1. I’m creating …
   1.1 … a mobile application.
   1.2 … a desktop application.
   1.3 … a web application.
   1.4 … an application that just can’t speak XMPP.
2. I need my application to …
   2.1 … show message status like Facebook does.
   2.2 … provide message archive to end users.
   2.2.1 I’d like to have a full text search feature.
3. … display inbox (a list of conversations with unread count and a last message).
4. … allow file transfers and media sharing between users.
   4.1 P2P
   4.2 File Upload
5. … support groupchats …
   5.1 … and I need precise presence tracking in each group.
   5.2 … and I don’t need to broadcast presence information in each group.
6. … be compatible with other public XMPP setups.
7. … present the same view of each conversation on every user’s device.
8. … allow users to block each other.
9. … support end-to-end encryption.
10. … be a part of Internet of Things.
11. … receive push notifications.
12. … publish messages to groups of subscribers.

1. Creating …

Before we proceed to more specific requirements, it’s important to identify crucial standards based on your application type.

1.1 … a mobile application.

Smartphones are omnipresent nowadays. It’s a fact. The whole software market considered, mobile apps are an important medium between various companies and their customers. Some of them are the actual products (games, communicators, car navigations, etc.), not only a “channel”. If you’re going to develop a mobile application, you will need…

XEP-0198 Stream Management

It’s an extension that provides two features actually. One of them is stanza delivery confirmation (both server and client side), what allows early detection of broken connections or malfunctioning network layer. The other one is stream resumption. It makes reconnection faster by reducing the round-trip count and relieves the client of fetching message archive as pending, unacknowledged messages will be retransmitted from server buffer.

It is enabled by default in MongooseIM and supported by major client libs like Smack or XMPPFramework. From a client developer perspective, it’s pretty transparent because the whole extension is enabled with a single flag or method call.

MUC Light, MIX, Push notifications, HTTP File Upload

These extensions are especially useful in the mobile environment. Why? With MUC Light and MIX you gain control over presence broadcasting - you can spare your users frequent radio wakeups and bandwidth usage. These extensions are a significant improvement over traditional presence-driven group chats.

Virtually every app on our smartphones uses push notifications. Some are useful and some are just annoying commercials. It doesn’t matter - it’s almost certain you’ll want them integrated with your XMPP service.

HTTP File Upload allows asynchronous media sharing, which is much more convenient in the case of group chats and doesn’t require both parties to stay online during the transfer.

These are just brief summaries. You can find more details further in this post.

1.2. … a desktop application.

Despite mobile phones’ expansion and software products exclusive for them (Instagram, Snapchat, Tinder, etc.), nobody can deny the comfort of UI operated with a mouse, keyboard, or tablet. Some apps simply require processing power that portable devices can’t provide. If your code is going to be executed on desktops PCs and laptops, you’ll appreciate…

There are no extensions that are strictly essential for desktop apps. Everything depends on specific applications. Just bear in mind that the standards important for mobile apps are generally useful for desktop ones too, only less critical.

1.3. … a web application.

As the days of heavy browser incompatibility (thank you, standardisation!) and Flash technology abuse are long gone, web applications are a great way to provide cross-platform solutions. It’s not only easier to reach more platforms but also to ensure the users are always running the most up-to-date version.

If you’re a web developer, you’re going to connect to the XMPP server via BOSH or Websockets.

Websockets

Websockets technology allow to upgrade an HTTP connection to an asynchronous, full-duplex, binary one (a bit of simplification but it’s the essence). It means that XMPP stanzas can be exchanged almost as efficiently as over a raw TCP connection (Websockets add small overhead of integer with packet size). It’s the recommended protocol for single-page apps.

Note: You can combine Stream Management’s resumption with Websockets, although it will still be slower than BOSH’s session pause.

Warning: Websockets are not implemented by old browsers. If you have to support any outdated clients, take a look at this table first.

BOSH

Defined in XEP-0124: Bidirectional-streams Over Synchronous HTTP (BOSH) and XEP-0206: XMPP Over BOSH. This protocol encapsulates XMPP stanzas in HTTP requests. It also simulates asynchronous, bidirectional communication by issuing long polling requests from client to the server to retrieve live data. What does it mean in practical terms?

A persistent connection may be maintained but in general BOSH is designed to deal with interrupted connections. It’s a perfect solution for web apps that trigger browser navigation. On such event, all connections made by e.g. JavaScript from browser are closed but the BOSH session survives it on the server side (not forever of course) and the client can quickly and efficiently resume the session after page reload.

The protocol is pretty verbose though, so if you don’t need this feature, go for Websockets.

1.4. … an application that just can’t speak XMPP.

You probably think that I’m crazy; why use XMPP with XMPP-less clients? Let’s change the way we think about XMPP for a moment. Stop considering XML the only input data format the XMPP server accepts. What if I told you that it’s possible to restrict XML to the server’s routing core and just make REST calls from any application? Tempting?

It’s a non-standard approach and it hasn’t been documented by XSF (yet), but MongooseIM already exposes most important functionalities via REST. Check out this and this document to find out more.

2. I need my application to …

Now we continue to more specific use cases.

2.1. … show message status like Facebook does.

By message status we mean following states (plus live notifications):

1. Not sent to server yet.
2. Acknowledged by the server.
3. Delivered to the recipient.
4. Displayed by the recipient.
5. User is composing a message.
6. User has stopped composing a message.

(1) and (2) are handled by Stream Management. It’s pretty obvious - before receiving an ack from the server, you are in (1); and ack confirms the message entered state (2).

We can deal with (3) and (4) by using XEP-0333: Chat Markers. These are special stanzas sent by a recipient to the original sender. There are dedicated markers for received and displayed events.

(5) and (6) are provided by XEP-0085: Chat State Notifications. It is up to a client to send updates like <composing/> and <paused/> to the interlocutor.

2.2. … provide message archive to end users.

Virtually every modern chat application maintains conversation history both for 1-1 communication and group chats. It can remind you of a promise you’ve made, be evidence in a divorce case, or help in police investigation.

XMPP provides two protocols for accessing message archives. The older one, XEP-0136 Message Archiving is used by hardly anyone, because it’s difficult to implement and overloaded with features. It has been superseded by more modern XEP-0313 Message Archive Management, which is the current standard.

There is one caveat though - its syntax changed significantly between versions, so it’s common for libraries and servers to explicitly state what versions are supported by the specific piece of software. These are 0.2, 0.3 and 0.4(.1) and 0.5. MongooseIM supports all of them in mod_mam module. If you choose another server, make sure its MAM implementation is compatible with your client library. Smack and XMPPFramework use 0.4 syntax.

2.2.1. I’d like to have a full text search feature.

Although standard Message Archive Management doesn’t specify any queries for full text search, it remains flexible enough to create such requests on top of the existing ones.

In MongooseIM this feature is still in experimental phase and has been recently merged into master branch. It’s not supported in any client library yet, so you have to construct a custom MAM query to do full text searches. Take a look at the PR description, It’s not that difficult. :)

2.3. … display inbox (a list of conversations with unread count and a last message).

Unfortunately there are no open solutions providing this feature. XMPP community is in the process of discussing and creating the specification of Inbox functionality. Erlang Solutions is designing a XEP proposal, which you can view here.

A quasi-inbox is available as a part of experimental standard Bind 2.0. It doesn’t cover all possible use-cases but a list of unread messages is what you actually need for optimal UX after establishing a connection. This feature is already under development in MongooseIM project.

In the meantime, you can build an inbox view by persisting last known archived message ID or timestamp and query Message Archive Management for all messages that came later. When you fetch them all, you can build an inbox. Unfortunately this is not very efficient and that’s why the community needs a new standard.

2.4. … allow file transfers and media sharing between users.

Almost everyone loves to share cat pictures and every modern IM solution provides means to do this. Various file transfer techniques in the XMPP world can be grouped in two categories: P2P connections and file upload.

The former involves establishing a direct connection between two clients, sometimes with a bit of a help from a TURN server. It ensures that data won’t get stored on any intermediate servers. Obviously, it requires less effort from the service provider because it’s easier and cheaper to set up a TURN service than to maintain a proper media server (or pay for storage in the cloud).

File upload is much more efficient when sharing media with a group. It doesn’t require both parties to remain online for the transfer duration.

2.4.1. P2P

Now, you DO have a choice here. There are a couple of XEPs, describing various P2P transfer initiation methods. XEP-0047 In-Band Bytestreams (IBB) is guaranteed to work in every network, because it sends data (Base64-encoded) via IQs. So if you can reach the XMPP service, you can transfer files. It may be slow and not very convenient but it will work.

Let’s carry on. You can transfer media via bytestreams external to XMPP. The P2P session is negotiated via XMPP but it’s only the “signalling” part. There are quite a few XEPs describing various negotiation and transmission protocols, so I will highlight specific implementations rather than listing all of the names which would only confuse readers who just want to send some bytes.

• XMPPFramework: Look for XMPPIncomingFileTransfer and XMPPOutgoingFileTransfer. They support SOCKS5 and In-Band Bytestreams.
• Smack: Everything begins with FileTransferManager. It supports SOCKS5 and In-Band Bytestreams as well.
  
  

2.4.2. File Upload

Unless you already have a dedicated media server that exposes an API to perform uploads and downloads, you should definitely take a look at XEP-0363 File Upload. It defines standard stanzas to request upload slots and respective download links. It is XMPP server’s responsibility to allocate the slots and return the links to the client.

Unfortunately this extension is not widely supported yet. You can find it in XMPPFramework but not in Smack yet. In the case of MongooseIM, it’s already available with Amazon S3 backend (with more storage plugins to come!).

2.5. … support group chats …

A couple of years ago it was really simple - there was only one kind of group chat supported in the XMPP world. Today we have three standards, two of them being maintained by XSF and one published by Erlang Solutions. MIX (XEP-0369), doesn’t have any implementations yet and as a standard it changes very frequently, so it is not described in this post.

2.5.1. … and I need precise presence tracking in each group.

If you need IRC-like experience where users have certain roles in a room and client disconnection triggers leaving the room, then classic XEP-0045 Multi-User Chat will work for you. It has its disadvantages (frequent presence broadcast may impact UX and consume processing power or connection throughput) but fits the use case, where accurate presence information is important. It is provided by MongooseIM’s mod_muc (other major servers implement it as well) and is supported by all mainstream client libs.

2.5.2. … and I don’t need to broadcast presence information in each group.

Erlang Solutions’ Multi-User Chat Light is a protocol derived from real world use cases, where groups doesn’t care about presences and full member list is always available to room members. It has some strong assumptions (like only 2 types of affiliation or rooms being joinable only by invite) but is designed to reduce round-trips, expose powerful API (e.g. room creation + configuration + adding new members in one request) and be easy to work with. Check it out and see if it fits in your application. Server implementation is currently exclusive to MongooseIM (mod_muc_light) and respective plugins are available in Smack and XMPPFramework.

2.6. … be compatible with other public XMPP setups.

Even some proprietary installations do integrate with open XMPP world (like GTalk and Facebook at some point), so if this is your use case as well, the first important thing to remember is that no custom stanzas may leave your cluster. By custom I mean anything that is not covered by any XSF-approved XEP. Additionally, you will really benefit from using XEP-0030 Service Discovery protocol a lot, because you can never be sure what is the supported feature set on the other end. It is used to query both clients and servers. Virtually every client and server supports it. In case of MongooseIM, the base module is mod_disco.

2.7. … present the same view of each conversation on every user’s device.

I use Facebook messenger on multiple devices and I really expect it to display the same shopping list I got from my wife on both my desktop and my mobile phone. It usually breaks message order but anyway - at least the list is there.

The problem is actually a bit more complex, because you have to take care of synchronising both online and offline devices.

Online devices can ask the server to forward all incoming/outgoing messages, even if they originate from or are addressed to some other resource of the same user. It is achieved by enabling XEP-0280 Message Carbons. On the client side it’s easy - just enable the feature after authenticating and the server will do the rest. It’s supported by MongooseIM in mod_carboncopy module. You can find respective implementations in Smack, XMPPFramework, Stanza.io and many others, since it’s a very simple, yet powerful extension.

If you want to fetch everything that happened while a specific device was offline for a while, just query XEP-0313 Message Archive Management (see “… provide message archive to end users.” section).

2.8. … allow users to block each other.

You just can’t stand your neighbour nagging you via IM to turn down the volume while Kirk Hammett is performing his great solo? Block him. Now. XMPP can help you with it. In two ways actually.

Yes, XMPP features two standards that deal with blocking: XEP-0016 Privacy Lists and the simpler XEP-0191 Blocking Command. The former allows users to create pretty precise privacy rules, like “don’t send outgoing presences to JID X” or “accept IQs only from JIDs in my roster”. If you need such a fine grained control, take a look at MongooseIM’s mod_privacy. On the client side it is supported by the likes of Smack and XMPPFramework.

Blocking Command is much simpler but most setups will find it sufficient. When a client blocks a JID, no stanza will be routed from the blockee to the blocker. Period. MongooseIM (mod_blocking), Smack and XMPPFramework have it.

2.9. … support end-to-end encryption.

When Alice wants to send a message to Bob… no, we’ve all probably seen this classic example too many times already. :)

There is no “one size fits all” when it comes to E2E encryption. The first tradeoff you’ll have to make is to decide whether you want new users devices to be able to decrypt old messages, or do you prefer to have a property of forward secrecy. For a full comparison between available encryption methods, let’s take a look at the table published by OMEMO authors:

	Legacy Open PGP	Open PGP	OTR	OMEMO
Multiple Devices	Yes	Yes	No	Yes
Offline Messages	Yes	Yes	No	Yes
File Transfer	Yes	Non-standard	Non-standard	Yes
Verifiability	No	Yes	Yes	Yes
Deniability	Yes	No	Yes	Yes
Forward Secrecy	No	No	Yes	Yes
Server Side Archive	Yes	Yes	No	No
Per Message Overhead	High	High	Low	Medium

It’s difficult to find an open library that supports any of these methods. Gajim communicator has an OMEMO plugin. Smack and XMPPFramework don’t support E2E encryption in their upstream versions. If you’re going to use E2E encryption in your application, most probably you’ll have to implement it on your own. Good thing is there are standards you can base your code on.

2.10. … be a part of Internet of Things.

We are a peculiar bunch. We use semiconductors to build machines that do heavy number crunching for us, deliver messages in a blink of an eye and control robotic arms with precision far beyond ours. A desire has awoken in us to go even deeper and augment everything with electronics. To let everything communicate with each other.

If you’re designing a fridge microcontroller that is supposed to fetch results from bathroom scales and lock the door for 8h for every excessive BMI point, you’ll need…

• XEP-0323 Internet of Things - Sensor Data
• XEP-0324 Internet of Things - Provisioning
• XEP-0325 Internet of Things - Control
• XEP-0326 Internet of Things - Concentrators
• XEP-0347 Internet of Things - Discovery
  
  

Unfortunately there are no public implementations of these standards. I wish it was easier but it seems you just can’t avoid reading these XEPs, picking the most suitable parts and creating your own implementation.

To find out more and become an active member of XMPP IoT community, check out IoT Special Interest Group.

2.11. … receive push notifications.

Push Notifications are (usually) doing a great service to mobile devices’ battery life. It is great indeed that a single TCP connection is maintained by OS, while apps can remain hibernated in the background. It is natural to every chat application to deliver notifications to the end user, even when a smartphone is resting in the pocket. How does XMPP cooperate with popular services like APNS or GCM?

It depends.

Although it’s not difficult to find XEP-0357 Push Notifications, it deserves some explanation. This specification is very generic. It assumes the existence of another XMPP-enabled “App server” that handles push notifications further. Although implementations could be found (e.g. in MongooseIM or Prosody server), it is very common for commercial installations to use custom protocols to provide push tokens and send PN packets directly to the respective services (APNS, GCM, SNS…)

2.12. … publish messages to groups of subscribers.

Publish-subscribe is an ancient and extremely useful pattern. XMPP got its own PubSub specification quite early (first versions were published in 2003) and even though the protocol is pretty verbose (and a bit complicated), for the basic usage you’ll need to learn only the most important aspects: there are nodes in the PubSub service where publishers can publish data. Nodes can group other nodes or remain plain leaves. That’s the whole story. The rest is about configuration, access control, etc.

XEP-0060 Publish-Subscribe is implemented in every major XMPP piece of software. In case of MongooseIM, it’s handled by mod_pubsub. You can find in popular client libraries as well: Smack, XMPPFramework or Stanza.io.

Set sail!

Now, if you feel brave enough, you can dive into this looong, official list of XEPs. These documents are designed to provide a precise information for libs and server developers. In your daily routine you’ll probably won’t care about every server-side edge case or whether some information is encapsulated in element X or Y. These are perfect reference guides for if you’re stuck somewhere or need to tap into lib’s plugin internals.

I’d recommend one more intermediate step though. Browse servers’ and clients’ feature lists published by virtually every project on their webpages. They usually skip (or enumerate them in separate, more detailed docs) minor items and highlight the features that may be important to you. This way you’ll expand your XMPP vocabulary and knowledge, while postponing the stage where reading XEPs is unavoidable, thus making the learning curve less steep.

It won’t be long until you realise that XMPP is your friend, I promise. :)

Stay tuned for Part 2! In the meantime:

1. Brush up on your XMPP basics with our guide to building an iOS app from scratch using XMPPFramework (parts 1 and 2)
2. Learn more about MongooseIM, our XMPP based open source mobile messaging platform.

First steps: XMPPFramework

Build a complete iOS messaging app using XMPPFramework is a tutorial that shows you how to build a fully functional instant messaging iOS app using the very cool XMPPFramework protocol and Swift3. In this part, we are going to get our hands dirty! To recap on the theory, or if you just landed here randomly, have a quick read through the first part, then get your Xcode ready and let’s start!

In this issue we are going to be integrating the library to our project, creating a connection with the server and authenticating. The XMPPFramework library is the most used XMPP library for iOS and macOS. At the beginning it may be a little bit overwhelming but after a few days working with it you will learn to love it.

Installing the library

Let’s create a brand new Xcode project and install the library. In this tutorial we are going to be using Swift 3. The easiest way to integrate XMPPFramework to the project is using CocoaPods.

Let’s create our Podfile using the pod init command in the folder where our .xcodeproj lives. There are thousands of forks but the maintained one is the original: robbiehanson/XMPPFramework.

So let’s add the pod to our Podfile and remember to uncomment the use_frameworks!.

use_frameworks!

target 'CrazyMessages' do
    pod 'XMPPFramework', :git=> 'git@github.com:robbiehanson/XMPPFramework.git', :branch => 'master'
end

 

Then pod install and CocoaPods is going to do its magic and create a .xcworkspace with the library integrated. Now we just need to import XMPPFramework in the files we want to use the library and that’s it.

 

Starting to build our Instant Messaging app

The most important thing in an XMPP application is the stream, that’s where we are going to “write” our stanzas, so we need an object that is going to hold it. We are going to create an XMPPController class with an XMPPStream:

import Foundation
import XMPPFramework

class XMPPController: NSObject {
    var xmppStream: XMPPStream

    init() {
        self.xmppStream = XMPPStream()  
    }

}

 

We are dealing with a highly asynchronous library here. For every action we are going to have a response some time in the future. To handle this XMPPFramework defines the XMPPStreamDelegate. So implementing that delegate is going to help us answer lots of different questions like: “How do I know when XMPP has successfully connected?”, “How do I know if I’m correctly authenticated?”, “How do I know if I received a message?”. XMPPStreamDelegate is your friend!

So we have our XMPPController and our XMPPStream, what do we need to do now? Configure our stream with the hostName, port and ourJID. To provide all this info to the controller we are going to make some changes to the init to be able to receive all these parameters:

enum XMPPControllerError: Error {
    case wrongUserJID
}

class XMPPController: NSObject {
    var xmppStream: XMPPStream

    let hostName: String
    let userJID: XMPPJID
    let hostPort: UInt16
    let password: String

    init(hostName: String, userJIDString: String, hostPort: UInt16 = 5222, password: String) throws {
        guard let userJID = XMPPJID(string: userJIDString) else {
            throw XMPPControllerError.wrongUserJID
        }

        self.hostName = hostName
        self.userJID = userJID
        self.hostPort = hostPort
        self.password = password

        // Stream Configuration
        self.xmppStream = XMPPStream()
        self.xmppStream.hostName = hostName
        self.xmppStream.hostPort = hostPort
        self.xmppStream.startTLSPolicy = XMPPStreamStartTLSPolicy.allowed
        self.xmppStream.myJID = userJID

        super.init()

        self.xmppStream.addDelegate(self, delegateQueue: DispatchQueue.main)
    }
}

 

Our next step is going to actually connect to a server and authenticate using our userJID and password, so we are adding a connect method to our XMPPController.

func connect() {
    if !self.xmppStream.isDisconnected() {
        return
    }

   try! self.xmppStream.connect(withTimeout: XMPPStreamTimeoutNone)
}

 

But how do we know we have successfully connected to the server? As I said earlier, we need to check for a suitable delegate method from XMPPStreamDelegate. After we connect to the server we need to authenticate so we are going to do the following:

extension XMPPController: XMPPStreamDelegate {

    func xmppStreamDidConnect(_ stream: XMPPStream!) {
        print("Stream: Connected")
        try! stream.authenticate(withPassword: self.password)
    }

    func xmppStreamDidAuthenticate(_ sender: XMPPStream!) {
        self.xmppStream.send(XMPPPresence())
        print("Stream: Authenticated")
    }
}

 

We need to test this. Let’s just create an instance of XMPPController in the AppDelegate to test how it works:

try! self.xmppController = XMPPController(hostName: "host.com",
                                     userJIDString: "user@host.com",
                                          password: "password")
self.xmppController.connect()

If everything goes fine we should see two messages in the logs but of course that’s not happening, we missed something. We never told to our xmppStream who was the delegate object! We need to add the following line after the super.init()

self.xmppStream.addDelegate(self, delegateQueue: DispatchQueue.main)

If we run the app again:

Stream: Connected
Stream: Authenticated

 

Success! We have our own XMPPController with a fully functional and authenticated stream!

Something that may catch your attention is how we are setting our delegate, we are not doing:

self.xmppStream.delegate = self

 

Why not? Because we can “broadcast” the events to multiple delegates, we can have 10 different objects implementing those methods. Also we can tell what’s the thread where we want to receive that call, in the previous example we want it in the main thread.

Getting a Log In

Our app is super ugly, let’s put on some makeup! We have nothing but an XMPPController and a hardcoded call in the AppDelegate. I’m going to create a ViewController that is going to be presented modally as soon as the app starts, that ViewController will have the neccesary fields/info to log in to the server.

I’m going to create a LogInViewControllerDelegate that is going to tell to our ViewController that the Log in button was pressed and that’s it. In that delegate implementation we are going to create our XMPPController, add the ViewControlleras delegate of the XMPPStream and connect!

extension ViewController: LogInViewControllerDelegate {

    func didTouchLogIn(sender: LogInViewController, userJID: String, userPassword: String, server: String) {
        self.logInViewController = sender

        do {
            try self.xmppController = XMPPController(hostName: server,
                                                     userJIDString: userJID,
                                                     password: userPassword)
            self.xmppController.xmppStream.addDelegate(self, delegateQueue: DispatchQueue.main)
            self.xmppController.connect()
        } catch {
            sender.showErrorMessage(message: "Something went wrong")
        }
    }
}

 

Why are we adding ViewController as a delegate of XMPPStream if our XMPPController alreay has that delegate implemented? Because we need to know if this connection and authentication was successfull or not in our ViewController so we are able to dismiss the LogInViewController or show an error message if something failed. This is why being able to add multiple delegates is so useful.

So as I said I’m going to make ViewController to comform to the XMPPStreamDelegate:

extension ViewController: XMPPStreamDelegate {

    func xmppStreamDidAuthenticate(_ sender: XMPPStream!) {
        self.logInViewController?.dismiss(animated: true, completion: nil)
    }

    func xmppStream(_ sender: XMPPStream!, didNotAuthenticate error: DDXMLElement!) {
        self.logInViewController?.showErrorMessage(message: "Wrong password or username")
    }

}

 

And that’s it! Our app can log in to our server as I’m showing here:

Logging!

We’ve been talking a lot about XMPP, stanzas and streams… but is there a way I can see the stream? Yes SR! XMPPFramework got us covered!

XMPPFramework ships with CocoaLumberJack, a pretty well known logging framework. We just need to configure it, set the logging level we want and that’s it. Logs are going to start showing up!

Configuring CocoaLumberjack

This is a really simple task, you just need to add to your func application(application: UIApplication, didFinishLaunchingWithOptions ... method the following line (remember to import CocoaLumberjack):

DDLog.add(DDTTYLogger.sharedInstance(), with: DDLogLevel.all)

I’m not going to paste here all the connection process log because it makes no sense to try to understand what’s going on at this stage of our learning. But I think showing what some stanzas look like is a good idea. To do this I’m going to be sending messages from Adium.

I’m going to send this <message/>:

<message to="test.user@erlang-solutions.com">
    <body>This is a message sent from Adium!</body>
</message>

 

Let’s see how it looks like when it reaches our app:

<message xmlns="jabber:client" from="iamadium@erlang-solutions.com/MacBook-Air" to="test.user@erlang-solutions.com">
   <body>This is a message sent from Adium!</body>
</message>

 

Let’s send a <presence/> from Adium:

<presence>
    <status>On vacation</status>
</presence>

 

We are receiving:

<presence xmlns="jabber:client" from="iamadium@erlang-solutions.com/MacBook-Air" to="test.user@erlang-solutions.com">
   <status>On vacation</status>
</presence>

 

No doubts at all right? We send something and we receive it on the other end! That’s it!

Test Time!

I want to be sure that you are understanding and following everything and not just copy and pasting from a tutorial (as I usually do 🙊). So if you are able to answer these questions you are on a good track!

• Why am I sending a presence after successfully authenticating? What happens if I don’t send it?
• What happens if I write a wrong server URL in the Log In form? How do I fix this problem if there is a problem…
• How do I detect if suddenly the stream is disconnected from the server? (maybe a network outage?)
• How do I detect if the user/password was wrong?

If you need help leave a message!

The sample project is on Github!

The next part is going to be on Roster, and if I will have space I would also like to add sending and receiving messages. I’ve been kind of super busy lately so I’m not sure when I’m going to be able to deliver the next issue but I’ll try to work on it as soon as I have some free minutes to spare!

PS: Also take a look at MongooseIM, our XMPP based open source mobile messaging platform. 

 

YAXT??! Yet another XMPP tutorial?

 

Well, this is going to be another tutorial, but I’m going to try to make it a little bit different. This is an XMPP tutorial from an iOS developer’s perspective. I’ll try to answer all the questions I had when I started working in this area. This journey is going to go from no XMPP knowldege at all to having a fully functional instant messaging iOS app using this cool protocol. We are going to be using the super awesome (yet overwhelming at the beginning…) XMPPFramework library, and the idea is also to also mix in some iOS concepts that you are going to need for your app.

What’s XMPP?

 

From Wikipedia: Extensible Messaging and Presence Protocol (XMPP) is a communications protocol for message-oriented middleware based on XML.

This basically means XMPP is a protocol for exchanging stuff. What kind of stuff? Messages and presences. We all know what messages are, but what about presences? A presence is just a way of sharing a “status”, that’s it. You can be ‘online’, 'offline’, 'having lunch’, or whatever you want. Also there’s another important word: Extensible meaning it can grow. It started as an instant messaging protocol and it has grown into multiple fields for example IoT (Internet of Things). And last, but not least: every piece of information we are going to exchange under this protocol is going to be XML. I can heard you complaining but… Come on, it’s not that bad!

Why do we need XMPP? Why not just REST?

 

Well what other options do we have? On the one hand, a custom solution means building everything from scratch, that takes time. On the other hand, we have XMPP, a super tested technology broadly used by millions of people every day, so we can say that’s an advantage over a custom approach.

Everytime I talk about XMPP, someone asks me 'Why not just REST?’. Well, there is a misconception here. REST is not a protocol, it’s just a way of architecting a networked application; it’s just a standarized way of doing something (that I love btw). So let’s change the question to something that makes more sense: “Why not just build a custom REST chat application?”. The first thing that comes to my mind is what I already explained in the previous paragraph, but there is something else. How do I know when someone has sent me a message? For XMPP this is trivial: we have an open connection all the time so, as soon as a message arrives to the server, it will send us the message. We have a full-duplex. On the other hand, the only solution with REST is polling. We will need to ask the server for new messages from time to time to see if there is something new for us. That sucks. So, we will have to add a mechanism that allows us to receive the messages as soon as they are created, like SSE or WebSockets.

There is one more XMPP advantage over a custom REST chat application. REST uses HTTP, an application level protocol that is built on top of a transport level protocol: TCP. So everytime you want to use your REST solution, you will need HTTP, a protocol that is not always available everywhere (maybe you need to embed this in a cheap piece of hardware?). Besides, we have XMPP built on top of TCP that’s going to be always available.

What’s the basic stuff I need to know to get started?

 

Well, you know a lot already but let’s make a list. Lists are always good:

• XMPP is built on top of TCP. It keeps an open connection all the time.
• Client/Server architecture. Messages always go through a server.
• Everything we send and receive is going to be XML and it’s called Stanza.
• We have three different types of stanzas: iq, message and presence.
• Every individual on the XMPP network is univocally identified by a JID (Jabber ID).
• All the stanzas are cointained in a Stream. Let’s imagine the Stream as a white canvas where you and the server write the stanzas.
• Stream, iq, message and presence are the core of XMPP. You can find everything perfectly detailed in RFC6120
• XMPP can be extended to accomplish different stuff. Each extension is called XEP (XMPP Extension Protocol).

 

What’s a JID?

Jabber ID (JID) is how we univocally identify each individual in XMPP. It is the address to where we are going to send our stanzas.

This is how a JID looks like:

• localpart: This is your username.
• domainpart: Server name where the localpart resides.
• resourcepart: This is optional, and it identifies a particular client for the user. For example: I can be logged in with andres@erlang-solutions.com on my iPhone, on my Android and on my mac at the same time… So all these will be the same localpart + domainpart but different resourcepart

I’m sure you have already noticed how similar the JID looks to a standard email address. This is because you can connect multiple servers together and the messages are rooted to the right user in the right server, just as email works. Pretty cool, right?

Sometimes you will see we have a JID with just the domain part. Why?! Because it’s also possible to send stanzas to a service instead of a user. A service? What’s a service?! Services are different pieces of an XMPP server that offer you some special functionality, but don’t worry about this right now, just remember: you can have JIDs without a localpart.

What’s a Stanza?

Stanza is the name of the XML pieces that we are going to be sending and receiving. The defined stanzas are: <message/>, <presence/> and <iq/>.

 

<message/>

This is a basic <message/> stanza. Everytime you want to send a message to someone (a JID), you will have to send this stanza:

<message from='andres@erlang-solutions.com/iphone' to='juana@erlang-solutions.com' type='chat'>
    <body>Hey there!</body>
</message>

 

<iq/>

It stands for Info/Query. It’s a query-action mechanism, you send an iq and you will get a response to that query. You can pair the iq-query with the iq-response using the stanza id.

For example, we send an iq to the server to do something (don’t pay attention to what we want to do… you just need to know there is an iq stanza and how the mechanism works):

<iq to='erlang-solutions.com' type='get' id='1'>
  <query xmlns='http://jabber.org/protocol/disco#items'/>
</iq>

And we get back another iq with the same id with the result of the previous query:

<iq from='erlang-solutions.com' to='ramabit@erlang-solutions.com/Andress-MacBook-Air' id='1' type='result'>
    <query xmlns='http://jabber.org/protocol/disco#items'>
        <item jid='muc.erlang-solutions.com'/>
        <item jid='muclight.erlang-solutions.com'/>
        <item jid='pubsub.erlang-solutions.com'/>
    </query>
</iq>

 

<presence/>

Used to exchange presence information, as you could have imagined. Usually presences are sent from the client to the server and broadcasted by it. The most basic, yet valid presence, to indicate to the server that a user is avaiable is:

<presence/>

After a sucessfull connection, you are not going to receive any <message/> until you make yourself available sending the previous presence.

If you want to make yourself unavailable, you just have to send:

<presence type="unavailable"></presence>

If we want to make the presences more useful, we can send something like this:

<presence>
      <status>On vacation</status>
</presence>

 

What’s a Stream?

Before answering this, let’s refresh our mind. What’s a Unix socket? From Wikipedia: A socket is a special file used for inter-process communication. These allows communication between two processes. So a socket is a file that can be written by two processes (in the same computer or in different computers in the same network). So the client is going to write to this file and server too.

Ok, but how is a socket related to a Stream? Well, we are going to be connected to a server using a socket, therefore we are going to have a 'shared file’ between the client and the server. This shared file is a white canvas where we are going to start writting our XML stanzas. The first thing we are going to write to this file is an opening <stream> tag! And there you go… that’s our stream.

Perfect, I understand what a stream is, but I still don’t understand how to send a message to the server. Well, the only thing we need to do to send a message is writting a <message/> stanza in our shared file. But what happens when the server wants to send me a message? Simple: it will write the message in the 'shared file’.

Are we ok so far?

 

I’m sure at this point you have questions like:

• “What?! An active TCP connection open all the time? I’m used to REST! How am I going to do that?!” 

​           Easy, you don’t have to care about that any more! That’s why we are going to use the library, and it will take care of that.

• “You said nothing about how to connect to the server!”

           Believe me, you don’t have to care about this either. If we start adding all this info, we are going to get crazy. Trust me, I’ve been there.

• “What about encrypted messages? We need security! How are we going to handle this?”

          Again, you don’t have to care about this at this point. Baby steps!

 

You just need to be able to answer: “What’s XMPP?”, “How do you send a message?”, “How do you change your status in XMPP?”, “How do you ask something to the server?”, “What’s a Stream?”. If you can answer all that, you are WAY better than me when I started.

All the concepts we described so far are the core of XMPP.  To find out how to get started with XMPPFramework, how to connect to the server and authenticate a user, go to PART 2!

Also, check out MongooseIM, our XMPP based open source mobile messaging platform.

Welcome to the XMPP newsletter.

It is also available in French.

If you have an article, tutorial or blog post you'd like us to include in the newsletter, please submit it on the XMPP wiki.

News

Paul Schaub penned some thoughts on the future of OMEMO where he discusses some of its current drawbacks and proposes futher improvements.

After nearly two years of running a Matrix server, Disroot have explained why they chose to refocus on XMPP instead. Their reasons include better resource consumption and a privacy-respecting architecture.

Daniel Gultsch visited Mailbox.org where he held an XMPP workshop and helped them to update their XMPP service. Here's the original blog post in German.

Salut à Toi now has one-on-one OMEMO support and when D-Bus is available it can be used as a remote-control.

Cisco, through their acquisition of Broadsoft, use Tigase XMPP Server in their products under AGPLv3 license.

Dino, the GNOME team chat client, now has a message search feature.

Ejabberd Business Edition now supports another protocol, MQTT 5.

JC Brand wrote a blog post about the Converse 4 release in which he talks about the history of the project and his plans for the future.

Subscribe to receive the next edition in your inbox

Tutorials

• XMPP real-time chat in React Native

Software releases

• Ejabberd 18.09

Clients

• Conversations 2.3.0
• Converse 4.0.0 and 4.0.1.
• JSXC 4.0.0 alpha
• Monal 2.2 alpha (with OMEMO)

Libraries

• Strophe.js version 1.2.16

Other software

• XMPPFrameworkLogger An iOS jailbreak tweak to log XMPP communication

The summer holidays season is often a time of great inspiration. This year we used that opportunity to focus on important groundwork on ejabberd codebase. However, we also decided to work on making ejabberd more accessible to newcomers.

For all these reasons, ejabberd is equally important for our growing community of users, for our customers, and for people that always wanted to give ejabberd a try but were afraid to make the jump.

Easier kickstart for newcomers

ejabberd, being written in Erlang has the reputation to be highly robust, highly scalable, clusterable, but also somewhat mysterious and, in turn, difficult to operate.

This release focus on making ejabberd more approchable, thanks to two efforts:

ejabberd binary installers polish

We polished our installers and fixed minor issues that users reported. This is often the first contact of our users with ejabberd. Please keep on reporting issues and suggesting improvements.

Default configuration file simplification

We simplified our default configuration file and made our default value better for production in the majority of cases. Tweaking the configuration file is the first task of a newcomer to ejabberd. It has to be done whether you are using our installers or are installing ejabberd from your Linux distribution package.

We now have a clean, no-bloat default config that has all the features from the Compliance Suite 2018 (XEP-0387) enabled by default. It’s a great base to build up your configuration with the help from the ejabberd docs.

Improved logging

We started an effort to improve the logging of ejabberd. Some entries changed the log level in which they are publishing. Other entries now comes in a more timely manner. All in all, this version is a first step to make ejabberd logs more useful and less intimidating for newcomers.

Up to date / increased security

OpenSSL 1.1.1 support

When operating a messaging / communication service, security is a top concern for sysadmins. We already had simplified access to TLS certificates with support for Let’s Encrypt. Now, we have adopted latest brand new TLS version.

OpenSSL 1.1.1 has been officially released in September 2018. It introduces support for TLS 1.3.
ejabberd 18.09 is already able to support OpenSSL 1.1.1 and as such you can use ejabberd with TLS 1.3.

TLS 1.3 is a major update for TLS and brings many benefits: you get increase speed in handshakes and improved security. If you want to learn more about TLS 1.3, I recommend you watch Filippo Valsorda presentation on TLS 1.3.

There is a catch though. Installers do not yet include OpenSSL 1.1.1. If you are planning to use ejabberd with TLS 1.3, you will need to build it yourself against your local OpenSSL 1.1.1. We are already working on making the next installers able to package the newest OpenSSL. This is quite a lengthy process, given the fact that our installer supports many operating systems (Linux, macOS and Windows), but we will get it done.

New external_secret option in HTTP Upload

Another interesting addition is a new external_secret option in the HTTP Upload. Thanks to this, you can offload all HTTP Upload processing to a separate HTTP server, for example nginx. The secret key is here to ensure that the upload service can verify that the upload comes from your HTTP upload module. It prevents random strangers to upload content to your server. This is a much needed addition to make this module more usable.

Sample configuration:

modules:
  ...
  mod_http_upload:
    ...
    put_url: "http://separate.http.server/upload"
    external_secret: "foo bar baz"
  ...

Please, make sure your secret key is unpredictable and long enough.

Performance improvements: getting ready to support Erlang OTP 21

This new ejabberd release is compliant with Erlang OTP 21. As such, it brings significant performance improvements to ejabberd. Erlang OTP 21 removed some IO locks in Erlang VM and thus enhance application IO scalability.

We package our installers with OTP 21 already. Try it and report your findings!

Modular ejabberd core

ejabberd code base has been in deep refactoring for many months now. We reached a point where our ejabberd core is extremely modular, thanks to our flexible hook based API.

This can lead to many innovations, with ejabberd becoming a great base for building high performance messaging platform. It means that this core is now the basis for our newest release of ejabberd Business Edition (4.0), without having to do any patching to ejabberd. It makes ejabberd suitable to implement other protocols by leveraging the clustering, networking and routing components of ejabberd. This leads to support of MQTT 5.0 protocol in ejabberd Business Edition.

This is a major milestone for ejabberd code base itself.

Download and install ejabberd 18.09

The source package and binary installers are available at ProcessOne.

As usual, the release is tagged in the Git source code repository on Github. If you suspect that you’ve found a bug, please search or fill a bug report in Issues.

---

On October 10th I'll give a talk entitled "You're Majoring in What?!? Why Philosophy is a Foundation for Success" at the University of Colorado Boulder (Hellems Arts and Sciences building, room 199, 6 PM). Although I don't believe it will be recorded, at the least I'll post the slides afterward. And if you live in the vicinity, feel free to stop by - I'll use my typical "performance art" presentation style, so it might even be mildly entertaining. :-)...

Recently I've revisited a field that's intrigued me for some time: philosophical counselling, or more broadly philosophical practice. Most of those active in this field are almost anti-method, and what methods they do employ are derived from hermeneutics, phenomenology, existentialism, Stoicism, and Socratic questioning - some of which I find interesting but none of which I find deeply congenial. Philosophical practice emerged in Europe around 1980 among thinkers influenced by hermeneutics and existentialism, and even the American philosophers in this area are mostly existentialists (associated with the American Philosophical Practitioners Association) or Stoics (associated with the National Philosophical Counseling Association) - for instance, Elliot Cohen, the progenitor of logic-based therapy, was a disciple of Albert Ellis (inventor of Rational Emotive Behavior Therapy). By contrast, my philosophical orientation is toward eudaimonia (the ancient Greek term for happiness, flourishing, living well) and the focus of my writing and living is on solving problems that hinder the pursuit of happiness. Upon reflection, I'm starting to see the possibility of developing a more joyous approach to philosophical practice that takes its inspiration from positive thinkers - especially the six I'm writing books about: Rand, Nietzsche, Aristotle, Thoreau, Epicurus, and Lao Tzu....

ProcessOne is proud to announce that ejabberd Business Edition now supports MQTT 5. This is a whole new protocol added – as an addition to XMPP (and SIP) – to our server. Our enterprise customers will be able to use both MQTT and XMPP in their real-time solutions.

ejabberd is the world’s leading XMPP server, both in performance and market share. With MQTT support, ejabberd brings equally incredible performance to this popular protocol, and true flexibility to real-time solutions.

What is MQTT?

MQTT stands for MQ Telemetry Transport. It is a publish/subscribe, extremely simple and lightweight messaging protocol, designed for constrained devices and low-bandwidth, high-latency or unreliable networks. The design principles are to minimise network bandwidth and device resource requirements while also attempting to ensure reliability and some degree of assurance of delivery.

These principles also turn out to make the protocol ideal for the emerging “machine-to-machine” (M2M) or “Internet of Things” world of connected devices, and for mobile applications where bandwidth and battery power are at a premium.

Why is it a big deal?

It is the first time that a single platform is able to support the major reference real-time standard protocols.

XMPP is an IETF standard for instant messaging communication. It is also used for Internet of Things in use cases where powerful devices need to be connected to the internet, especially with their own presence signal and control interface. XMPP offers more features than MQTT, but the extra flexibility and complexity comes at a cost. Some less powerful devices cannot leverage XMPP and need to resort to MQTT, which is designed with that small devices use cases in mind. MQTT is a standard from OASIS.

Supporting both standards opens a whole set of new use cases, making it possible to connect people, applications, small IoT devices and larger IoTs on the same platform.

Moreover, from the very beginning ejabberd Business Edition provides full support for the brand new MQTT 5, being ready to support the most advanced use cases offered by this protocol. It is one of the very first production-ready MQTT 5 servers, with compliance to the brand new specification released in May 2018.

MQTT and XMPP

Thanks to combined support of XMPP and MQTT, ejabberd Business Edition is the first truly multi-protocols real-time messaging servers. Your solutions can use the best of both worlds, and your implementations can be flexible and scalable at the same time.

By using a multi-protocol server, your deployment can benefit from the ecosystems of both protocols, and be compatible with vast number of standard tools and software.

Does it mean, we consider XMPP obsolete? Absolutely not. XMPP and MQTT are designed with different use cases in mind. However, we think that in many cases, a real-time project will need to leverage both XMPP and MQTT during its lifetime, making humans, things and application collaboration possible. This is our vision.

What are the main benefits?

ejabberd Business Edition has been much refined during recent years. ProcessOne has patiently engineered a server framework that is stable and highly available thanks to a fast and flexible clustering engine, and versatile thanks to its plugin API.

MQTT protocol in ejabberd Business Edition has been reimplemented from scratch, but the server infrastructure leverages the brand new modular architecture of eBE. It means it uses the same highly scalable clustering engine, the same authentication infrastructure, can provide virtual hosting, reuse the logging modules, and use the same API.

For the users, the benefits are clear: you can reuse the same server, infrastructure, authentication mechanism, security policy for your XMPP clients and MQTT clients. You can invest in a single platform and be sure to be future-proof and address both your current and future needs in term of real-time technologies.

You can have a part of your devices connected to XMPP and another part connected on MQTT, without the need to manage two separate platforms. And most of all, you can design your system for use cases mixing XMPP strong points (human collaboration, chat bots for automation) with MQTT (connection to millions of very lightweight devices).

The bottom line is simple:

1. You can switch between XMPP and MQTT as you wish, even use both protocols on the same infrastructure.
2. You will save on infrastructure, given the high-performance of the platform.
3. You get support on solution design for real-time infrastructure and can get help choosing between XMPP and MQTT, from a vendor that has no interest in selling one protocol more than another.

How can I test it?

If you are a developer and would like to try connecting your client to our MQTT server, we got you covered.

ProcessOne has deployed a public MQTT broker with anonymous access. This is one of the first MQTT 5 public brokers available.

The broker is deployed at mqtt.fluux.io on port 1883 and 8883 (TLS).

You can, for example, test our broker with mosquitto client. On macOS, you can install mosquitto via Terminal using Homebrew with:

brew install mosquitto

From there, you can subscribe to the topic test/topic with:

mosquitto_sub -h mqtt.fluux.io -p 1883 -V mqttv311 -v -t "test/topic" -q 2

and then (in another Terminal tab or window) you can publish with:

mosquitto_pub -h mqtt.fluux.io -p 1883 -V mqttv311 -t "test/topic" -m "message" -q 2

You should see your message received and displayed below the mosquitto_sub command in your previous Terminal tab or window.

Note: In this example, Mosquitto client is using MQTT v3.1.1, as it doesn’t support version 5.0 at the moment of writing.

Don’t hesitate to send your feedback and findings through our contact page. Happy hacking!

What is next?

From our initial tests ejabberd Business Edition MQTT performance is very impressive, and we will be publishing our benchmarks soon.

Regarding MQTT 5, shared subscriptions are still being developed and will be available in a next release.

Regarding protocol version coverage, our platform supports MQTT 3.1.1 and we plan to add support for MQTT 3.1 as well in a future update.

Most of all, we are planning to increase the interoperability between MQTT and XMPP. The plan is to have both XMPP and MQTT clients be able to leverage the same services provided by ejabberd Business Edition: push notifications, groupchat through Muc/Sub, etc.

The goal is to leverage use cases based on real-time communication between humans, things, and applications. This can only be done by building an ambitious platform that is protocol agnostic and leverages the best of each technology.

We believe the IoT revolution will fuel the growth of both XMPP and MQTT protocols. For this to happen, the development community needs flexible, scalable and – most importantly – trusted tools. For more than 15 years ejabberd has been the best XMPP server. We now want it to be the best MQTT server as well.

ProcessOne curates two monthly newsletters – tech-focused Real-time Stack and business-focused Real-time Enterprise. Here are the articles concerning business aspects of real-time enterprise we found interesting in Issue #14. To receive this newsletter straight in your inbox on the day it’s published, subscribe here.

Companies are migrating to the cloud for the IoT

Developers’ evolving demands and needs will push IoT platform providers to reshape their products and services in the coming years. Developers’ needs, concerns, and demands will change as their companies build out their IoT projects.

The Internet of Things explained by a professor

The IoT becomes a subject of the degree course on Production and Process Management at Heilbronn University of Applied Sciences in Germany. Prof. Dr. Thomas Pospiech talks about the “IoT box”, an invention he has developed to better convey the concept.

IBM teams with Maersk on new blockchain shipping solution

IBM and shipping giant Maersk having been working together for the last year developing a blockchain-based shipping solution called TradeLens. Today they moved the project from Beta into limited availability.

Risky business? How to manage IoT security in the enterprise

Nearly every week, we hear about a new cyberattack or security breach. In fact, more than 1.7 billion identities have been exposed in data breaches in the past eight years. As the world becomes increasingly interconnected, organizations become susceptible to more risk.

IoT boom will change how data is analysed

As more devices come online, each generating data, the way information is analysed and used to facilitate machine learning will have to change.

Five IoT predictions for 2019

The IoT is growing at an exponential rate. Vehicles, wearable gadgets, RFID sensors and software are advancing past basic function and the network is growing to include even more advancements each day. What will the future bring?