Planet Jabber

The XMPP Standards Foundation (XSF) will hold its 23th XMPP Summit in Brussels, Belgium, on Thursday January 31st and Friday February 1st 2019. These are the two days preceding FOSDEM 2019.

The XSF invites you all to attend, and discuss all things XMPP!

If you're interested in attending, please make yourself known by filling out your details on the wiki page for Summit 23 (to edit the page, you'll need a wiki account, which we'll happily provide for you. Find us either in the jdev@conference.jabber.org chatroom, or via the Summit mailling list).

Please note that, although we welcome everyone to join, you must announce your attendance beforehand, as the venue is not publicly accessible (and we need badges printed for you).

If you haven't already, make sure that you're signed up to the Summit mailling list, which is where you can expect most updates to be announced.

See you there!

Swift for Linux packages are officially provided for Ubuntu. If you want to run Swift on another Linux server flavor, you are on your own.

Fortunately, building Swift on Linux is pretty straightforward once you know how to do it.

Please, note that the build process is quite heavy and your likely need more than 16 GB or RAM to build Swift (especially if you prepare a debug build). In my example, I used a server with 32 GB. At the end of the article, you will find a link to download the result of the build. You may want to use it to avoid rebuilding it on your own on a smaller server.

Now, let’s check the build process on Linux Debian Stretch.

Preparing the server

Install Swig 3.12 on Debian

First, let’s fix a specific Debian issue.

The default version of Swig provided with Debian Stretch is Swig 3.0.10. The problem is that it is not compliant with LLDB. See LLDB source code for details.

We first need to ensure Swig is not already installed:

$ sudo apt-get -y remove swig3.0 swig

You can then install Swig a newer version of Swig from packages compiled for Ubuntu. I used the package for Swig and Swif3.0 for Ubuntu 18.04 LTS, as found on pkgs.org. You may want to select other packages, depending on your need.

Let’s download and install Swig 3.12 packages:

$ wget http://archive.ubuntu.com/ubuntu/pool/universe/s/swig/swig_3.0.12-1_amd64.deb
$ wget http://archive.ubuntu.com/ubuntu/pool/universe/s/swig/swig3.0_3.0.12-1_amd64.deb
$ sudo dpkg -i swig*.deb

Other dependencies

Other dependencies required for Swift are described in Swift README.

You can install them all, with the following command:

$ sudo apt-get install -y git cmake ninja-build clang python uuid-dev libicu-dev icu-devtools libbsd-dev libedit-dev libxml2-dev libsqlite3-dev libpython-dev libncurses5-dev pkg-config libblocksruntime-dev libcurl4-openssl-dev systemtap-sdt-dev tzdata rsync swig

You should be ready to download and compile Swift from Github repository.

Building Swift

You can type the following commands to build Swift from source.

First, you need to clone Swift repository in a subdir of swift-source new directory:

$ mkdir swift-source
$ cd swift-source
$ git clone https://github.com/apple/swift.git

Then, you need to download all the repositories it will build in the process:

$ ./swift/utils/update-checkout --clone

Finally, you need to launch Swift build itself. The following command is the easiest one to use to install Swift on your Linux server:

$ swift/utils/build-script --preset=buildbot_linux,no_test install_destdir=~/swift-install installable_package=~/swift.tgz

It will build Swift Swift in swift-install dir and prepare a .tgz archive with the content of the installation.

Have a long break, it will take a while :)

Once it is done, you can simply copy the resulting structure in /usr/local/:

$ sudo cp -R ~/swift-install/usr/* /usr/local/

Testing Swift a try

The first thing you can test to get started is the REPL.

The swift command can be used to launch the REPL. You can then type interactive Swift commands:

$ swift
Welcome to Swift version 4.2-dev (LLVM cbe8d5e28f, Clang 3452631569, Swift 0d2787fb31).
Type :help for assistance.
  1> print("Hello, Swift!")
Hello, Swift!
  2> 

You can also put that print statement in a hello.swift file and compile it with swiftc:

$ echo "print(\"Hello, Swift\")" > Hello.swift
$ swiftc Hello.swift
$ ./Hello
Hello, Swift

Your Swift environment is now ready and you can now start developing in Swift on Linux.

Next steps

From there, the next step is to explore Swift on the server-side, using a server framework like Swift-NIO, Vapor, Kitura or Perfect.

Enjoy!

Downloading the resulting build

If you would like to try Swift on Debian to learn the language on a smaller instance, you can try downloading our prebuilt archive: swift-4.2-git.tgz.

Once downloaded, you can download the signature file swift-4.2-git.tgz.asc and check the signature as follows:

$ gpg --keyserver pgp.mit.edu --recv-key CF34B813
$ gpg --verify swift-4.2-git.tgz.asc 
gpg: Signature made Thu 15 Nov 2018 11:03:00 AM CET using RSA key ID CF34B813
gpg: Good signature from "Mickaël Rémond <mremond@p*.net>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 58F4 6916 C32F 20EE 648F  9475

It is signed with my GPG key (Mickaël Rémond).

There was a bug that came up recently about Monal disconnecting in Mojave when backgrounded which really confused me.  It seems a feature added to OSX in Mavericks called App Nap that suspends apps to save battery life might be the culprit.  I am putting out a new Mac beta today Bonus: With improved dark mode support, better dark mode correct fonts and icons. 

There is new Mac beta. I have tried to fix the dark mode visibility as well as some possible networking issues. 

There is a new iOS beta. Testflight users should update or grab it from the beta  link to the side. 

In order to reduce the pain for most users, I have added a new login screen that will pop up if no accounts are set up (e.g. first run)

I have made the latest iOS Beta available to the public.  This link is also permanently on the left of this site.  This is a beta so you will see debugging messages as well as a debug log at the bottom of settings titled ‘Log’  this will let you view everything the app is doing.  If you encounter any issues feel free to look there and send me the last few lines (make sure not to send anything involving authentication)

In this short post, I will describe how to a read provisioning profile from iOS mobile app to discover some apps metadata.

The mobile provisioning profile is a file embedded by XCode when you build and package your application. It contains several pieces of information that can be useful for your app. For example, you can get the Apple push environment to use, either sandbox for development apps or production for store or Testflight production builds.

Principles

Getting and reading the mobile provisioning file

The mobile provision file is embedded in the app. You can read it from the main bundle. You first need to read the path of that file from the main bundle:

let profilePath: String? = Bundle.main.path(forResource: "embedded",
                                            ofType: "mobileprovision")

And then you can try reading the file into an string:

let profileString = try? NSString.init(contentsOfFile: profilePath,
                                       encoding: String.Encoding.isoLatin1.rawValue)

Extracting the metadata plist

The provisioning profile contains a plist file that exposes some app metadata. However, the plist string is embedded in a larger binary file. The easiest way to extract the plist is to read only the XML part of the file. This can be done as follows:

// Skip binary part at the start of the mobile provisionning profile
let scanner = Scanner(string: profileString as String)
guard scanner.scanUpTo("<plist", into: nil) != false else { return nil }

// ... and extract plist until end of plist payload (skip the end binary part.
var extractedPlist: NSString?
guard scanner.scanUpTo("</plist>", into: &extractedPlist) != false else { return nil }

guard let plist = extractedPlist?.appending("</plist>").data(using: .isoLatin1) else { return nil }

Parsing the plist string

You can finally parse the plist strong to get the plist content into a MobileProvision struct, using Swift decodable protocol, introduced in Swift 4:

let decoder = PropertyListDecoder()
do {
   let provision = try decoder.decode(MobileProvision.self, from: plist)
   // Do something with the data
} catch let err {
   // Handle parsing error
}

Putting it all together

Here is the full code with the MobileProvision struct implementing the Decodable protocol:

You can use the code from your mobile app as follows:

var sandbox: Bool
if let provision = MobileProvision.read() {
   print("We need to use Sandbox")
   sandbox = provision.entitlements.apsEnvironment == .development ? true : false
}

That’s it. Your application is now able to discover if it should use APNS in development or production mode automatically, without having to rely on defining a DEBUG build environment variable.

It’s been a while since the last release. I am planning out what will be in the next Mac and iOS clients.   I was hoping to have OEMEO but it hasn’t been sufficiently  tested yet, so I am turning it off until the following release.   I will have an open beta for iOS 3.1 soon. This will be a GDPR update  but along with it comes the new rich links, a more compressed chat ui as well as the share sheet.  Mac clients will see the Mojave dark mode and a share sheet.

OMEMO is, like any other encryption protocol based on trust. The user has to make sure, that the keys they are trusting belong to the right users. Otherwise a so called Man-in-the-Middle attack is possible. An attacker might pretend to be your contact and secretly steal all the messages you thought were encrypted. They are, just to the wrong recipient.

To counteract such attacks, OMEMO encourages the user to verify their contacts fingerprints. A fingerprint can be considered the name of a contacts key. The user has to make sure, that the key A he is presented with really belongs to contact C by checking, if the fingerprints match. As those fingerprints are usually long, random series of characters, this is a tedious task. Luckily there are techniques like QR codes, which make our lifes easier. Instead of comparing two long strings character by character, you just scan a code and are done.

The QR-Code contains the Jabber-ID of the owner, as well as all of their fingerprints. So by scanning your code, a friend might automatically add you to their contact list and mark your devices as trusted. You can also put the QR-Code on your personal website, so people who want to reach out to you can easily establish a secure connection to you.

I spent the last few days looking into JavaFX (I have no real UI designing experience in Java, apart from Android), designing a small tool that can generate OMEMO fingerprint QR-Codes. This is what I came up with:

The tool logs into your XMPP account and fetches all your published keys. Then it presents you with a list in which you can select, which fingerprints you want to include in the QR-Code.

There are still a lot of features missing and I consider the tool in no means as completed. My plans are to add the possibility to export QR-Codes to file, as well as to copy the text content of the code to clipboard. You see, there is a lot of work left to do, however I wanted to share my thoughts with you, so maybe client developers can adopt my idea.

Something people have requested for a while has been to have wallpapers on chat. This will be in the 3.1 release.  I am also updating the bubbles to contain delivery and time stamp info so you don’t have to worry about text visibility with your background.

Welcome to the XMPP newsletter.

It is also available in French.

If you have an article, tutorial or blog post you'd like us to include in the newsletter, please submit it on the XMPP wiki.

In this newsletter you will find information about an XMPP sprint happening this month in Germany, an article about XMPP use cases, many software releases and much more.

News

The Berlin XMPP meetup was held, where Holger Weiß continued his series about the perfect XMPP server setup. The next Berlin XMPP meetup will be on the 14th of November. Daniel Gultsch will talk about a brand new feature for Conversations he is working on.

On Saturday the 17th and Sunday the 18th of November an XMPP sprint will be held in Düsseldorf. It is still possible to sign up for participation.

Daniel Pocock made a call for participation in the Real Time Communications track at FOSDEM. Great opportunity for anyone interested in giving an XMPP talk.

The XSF is holding Board and Council Elections. Candidates can apply until the 4th of November.

Subscribe to receive the next edition in your inbox

Tutorials

Erlang solutions published some tutorials listed below:

Part one of an XMPP use cases guide titled 21 XMPP use-cases and the best ways to achieve them where one will find an article explaining which XEPs can help software developers when building applications that need to support some XMPP features like groupchats, receive push notifications, etc.

How to set up Push Notifications with MongoosePush.

How to set up MongooseICE (ICE/TURN/STUN server).

Software releases

Tigase XMPP Server v7.1.4 has been released.

ProcessOne, the company behind ejabberd, wrote about improvements made for ejabberd Business Edition and ejabberd SaaS, including HTTP file transfer using S3 and Minio, unread inbox support and more.

RTB (Real-Time Benchmark) can now be used for XMPP benchmarks and MQTT. It supports MQTT 3.1.1 and 5.0.

Clients

October has been an active month for Converse.js. Three releases went out. All of them are available in Github, with 4.0.4 being the latest release.

Libraries

The Smack library released its 4.3.1 version.

JaXMPP (Tigase client library) reaches version 3.2.0 with many fixes and new features including support for XEP-0363 HTTP File Upload and other interesting ones.

Two XMPP libraries for JavaScript got a new release in October. Check the latest versions for Strophe.js and xmpp.js

Escalus 4.0.0 comes with a new XML parser and some other features like an included XML viewer.

Other software

JSXC released the latest version for the Nextcloud authentication hub called xmpp-cloud-auth (also known as xcauth).

I’ve been focusing some of my efforts as of late on things that make Monal feel more modern. There are parts of the app I haven’t touched in 5-6 years and expectations have changed in those years.  The inline images in 3.0 was very well received and I am following that up with inline link previews.  If you have used any other modern messaging client this will be very familiar to you.  XMPP doesn’t need to feel like its form the 90s.

Impasses along the path toward the goals of life can occur in three dimensions:...

There are many ways to categorize and divide up the various areas and activities of life. Here is a six-fold scheme I'm exploring:...

This is more of a question for other open source developers.  I recognize there are payment structures in the Mac App Store that do not work for paid software. However, I’ve always wondered why other open source apps do not publish there.

My general experience publishing  Monal there has only been positive.   While I still push out beta clients on the web the App Store has been the primary way that users have discovered Monal.  It also takes care of several things that are pain points for developers such as upgrades and hosting/bandwidth costs.  I honestly think more people use XMPP because I have made the software readily available in a trusted location. The Mac App Store has a lot of good software but also a lot of junk. The ratio is a lot worse than on iOS and I think we should change that. 

If the problem is the $99 developer fee, would you be more interested in publishing software there if it were to go away (or at least not be something you need to worry about)?

We have been adding many improvements to ejabberd Business Edition and are rolling them out for new ejabberd SaaS customers in October.

HTTP file transfer using S3 and Minio

ejabberd Business Edition was previously supporting HTTP file transfer using Amazon AWS S3 backend. We now created and documented the process where our customers can host their files themselves using Minio, an S3 compliant implementation.

If you are an ejabberd SaaS customer, you can keep on using your own Amazon account by providing specific S3 credentials for your own S3 bucket. This allows you to manage your own storage and define your own retention period. This approach is the best way to also be compliant with GDPR, as ejabberd SaaS service is stateless and does not have any way to access the files your users are transferring.

Improvements in mobile MUC and unread inbox support

MucSub is a popular extension to ejabberd Business Edition and ejabberd SaaS. This is a way to support standard MUC chatroom, while providing and extra protocol to interact with it that is mobile-friendly.

The first improvement we made is that your own MAM archive will now archive the MucSub messages. This makes it possible to manage them as “conversations”, like one-to-one messages, when it comes to Message Archive Management.

Once we stored MucSub message in MAM, we had to go further. We also made some improvements to manage your MAM download efficiently, by providing a way to download an unread inbox summary. This packet allows you to get the list of conversations that have unread messages, either MucSub conversations or one-to-one discussions. The client can then download the message archive containing unread messages on-demand. This avoids having to download all your MAM messages at once on connect to display unread counters. This is a huge optimisation and an improvement for mobile users.

Finally, we expanded the range of MUC Admin functions you can perform as an admin without needing to join the room.

Push module improvements

Push can now be fully customised, with all keys being editable and settable through API.

That said, we also improved the standard behaviour. We now support notification grouping on iOS 12. Notifications sent will be automatically grouped per conversation. The user can still opt for a global app grouping in the iOS notification setting for that application.

MySQL 8 support

ejabberd now supports MySQL 8 new authentication mechanism. You can start testing it today.

Send IQ API call

We exposed and documented an API call that allows you to send an IQ using the API, then wait for the response to that IQ and return it as a result of the HTTP call.

Note for ejabberd SaaS existing customers

If you are already an ejabberd SaaS customers, and need access to the new features immediately, please contact ejabberd SaaS support to plan an upgrade with our team.

There was a push server outage on push.monal.im for a few hours this weekend. It was  my mistake while upgrading components. I will be more careful in the future. Obviously no messaging traffic goes through this server so there was no data loss. Some people may not have reliably received new message notifications this weekend. 

I’ve completed my server changes. Monal for iOS and Mac should both be available in EU stores.  I will update the next release of both with links to the privacy policy. 

In my recent philosophy talk, I mentioned that thinking philosophically has helped me better understand people. This might seem counterintuitive - after all, don't big thinkers believe they have all the answers? Yes, I used to have that kind of arrogance, too. Yet over time I started to wonder: why do people have so many different opinions about things? Why have these various philosophies of life persisted for so long (Stoicism, Platonism, Buddhism, Taoism, etc.)? Why do I think what I think? Is it really in my control? At some level can you really "blame" people for the contents of their minds?...

The new Mac beta for OSX should support dark mode. I am still tweaking it so there may be places where colors are less than ideal.  This build also supports OMEMO.

We are happy to announce that RTB now fully supports MQTT 3.1.1 and 5.0. With this addition, RTB is now a fully-featured real-time infrastructure benchmarking tool.

RTB is capable of testing XMPP and MQTT solutions at the rate of millions of concurrent sessions. You can find RTB on GitHub. In the near future we plan to release detailed benchmarks stress-testing ejabberd on both protocols.

What is RTB?

RTB (Real-Time Benchmark) is an open source benchmarking tool to stress-test real-time protocols. The aim of this project is to define an easy benchmark for XMPP and MQTT servers, with minimal configuration overhead.

RTB was first released by ProcessOne in July 2018. To compare server implementations and be sure they are ready for massive traffic, there is a need for a “golden standard” benchmarking tool that is easy to use and avoids ambiguity due to configuration or overhead – each test should have the same baseline and capabilities for the results to be comparable.

We believe RTB fits such a role, because it has sane defaults (gathered from statistics of real world servers) and is able to cover all the test features defined in the XMPP Compliance Suite 2018.

What is next?

The next step for RTB is to implement support for MQTT version 3.1. Additional work will be put into making RTB simple to use and have friendly display of results.

We encourage the community to use RTB, and provide feedback through issues and pull requests.