Planet Jabber

January 15, 2017

Arnaud Joset

Authentication without password using XMPP on a Django website

This article describes the authentication with XMPP on a Django powered website.

Authentication without password

When you authenticate on a website, the domain validate your identity before letting you access confidential information. They are several ways perform this validation and the use of passwords is the most popular. Another method is the use of a token generator i.e. a small device that generate a secret passphrase that you copy on a website. Today I will present you another authentication method without password using XMPP.

XMPP authentication

XMPP has a nice authentication mechanism. It is normalized in the XMPP extension XEP-0070. It may be used on website. There are 4 steps.

  1. The user visits its favorite website and go to the login section.
  2. The user enter its jid (XMPP address) in a form and click on a button to authenticate.
  3. The website send a XMPP request to the user asking if he wants to login on the website. The request display also a code that must be identical on the website and the XMPP client in order to validate the request.
  4. The user validate the request on its XMPP client and therefore he is login on the website.

There are plenty XMPP clients: Gajim, Salut-à-toi, Movim, Conversation, Poezio, Pidgin, Psi etc. Several of them work on mobile, on webpage or on Desktop. Therefore, it is possible to authenticate easily on a website using your smartphone, Desktop or another platform easily without password.

Note: if the client does not support the XEP-0070, there is a fallback mechanism where the user send back the validation code in a chat window. Therefore, it is possible to authenticate with all XMPP clients.

Examples

Gajim

Gajim XEP-0070

Salut à toi (Primitivus)

Primitivus XEP-0070

The following section presents the implementation of this mechanism on a Django website.

Use XMPP authentification mechanism with Django

Make it easy with HTTPAuthenticationOverXMPP

In this section, the XMPP part is managed by a component written by "Chteufleur‎". This component is easy to use. It manage the XMPP session and the web developeur just have to make a request to the component and it sends a return code:

  • 200 : User accepts the request
  • 400 : One or more mandatory parameter(s) is missing
  • 401 : User denies the request or timeout
  • 520 : Unknown error appends
  • 523 : Server is unreachable

The installation procedure is described in the Readme file of the project (https://git.kingpenguin.tk/chteufleur/HTTPAuthentificationOverXMPP).

Django files

The view manage the form fields and send the jid and validation code (transaction_id) to a module called XmppBackend. The transaction_id is generated when the form is accessed. Its value is kept in memory by using the session mechanism of Django (see section Settings.py).

Several files are needed to obtained the desired result. The following sections describes them.

Forms.py

from django import forms

class AuthForm(forms.Form):
    username = forms.CharField(max_length=100, help_text="(XMPP jid)")

HTML template

{% extends "base.html" %}

{% block content %}

{% if form.errors %}
<p>Your username is invalid. Please try again.</p>
{% endif %}

<form method="post" action="{% url 'login' %}">
   {% csrf_token %}
   <table>
       {{form.as_p}}
    </table>
    <input type="submit" value="Login" id="Login" name="login"/>
</form>
Your validation code: {{ transaction_id|linebreaks }}
<strong>{{ status_msg|linebreaks }}</strong>
{% endblock %}

Views.py

views.py reads the content of the POST and sends the result to xmpp_auth. It also handles the session and the transaction_id generation.

from django.shortcuts import render
from django.contrib.auth import login
from django.http import HttpResponse
from . import xmpp_auth
from .forms import AuthForm


def index(request):
    return render(request, 'index.html')


def xmpp_authentification(request):
    xb = xmpp_auth.XmppBackend()
    transaction_id = None
    status_msg = ""
    if request.method == 'POST':
        try:
            transaction_id = request.session.get('transaction_id')
        except KeyError:
            request.session['user_logged_in'] = False
            return render(request, 'fail.html')
        form = AuthForm(request.POST)
        # check whether it's valid:
        if form.is_valid():
            username = form.cleaned_data['username']
            user, status_code = xb.authenticate(username=username, password=None, transaction_id=transaction_id)
            if user is not None:
                login(request, user)
                # Redirect to a success page.
                request.session['user_logged_in'] = True
                return render(request, 'success.html')
            if status_code == 401:
                request.session['user_logged_in'] = False
                status_msg = "User {} refused to authenticate.".format(username)
        else:
            request.session['user_logged_in'] = False
            return render(request, 'fail.html')
    else:
        request.session['user_logged_in'] = False
        transaction_id = xb.id_generator(6)
        request.session['transaction_id'] = transaction_id
        form = AuthForm()

    return render(request, 'registration/login.html', {'form': form , 'transaction_id' : transaction_id,
                                                       'status_msg': status_msg})

xmpp_auth.py

This module makes the following request to the component:

GET /auth?jid=user%40host%2fresource;domain=example.net;method=POST;transaction_id=what_you_want;timeout=120 HTTP/1.1

The component send back a return code. In case of success, the system try to find the user in the database. If this user does not exist, it is created. The system described here is simple and the code must be adapted for more complex website (profile creation, additionnal data etc).

id_generator is called by views.py and by default, it send a code made of 8 characters (both letters and digits) but it is possible to adapt easily this behavior.

import sys
import requests
import string
import random
from django.contrib.auth.models import User


class XmppBackend(object):
    """
    Authenticate with the XMPP 00-70 XEP
    """
    def __init__(self):
        self.transaction_id = None

    def get_transaction_id(self):
        return self.transaction_id
    def set_transaction_id(self, transaction_id):
        self.transaction_id = transaction_id


    def authenticate(self, username=None, password=None, transaction_id = None):
        # Check the token and return a user.
        timeout = 300
        payload = {'jid': username, 'domain': 'agayon.be', 'method': 'POST', 'timeout': timeout,
                   'transaction_id': transaction_id}
        r = requests.get('https://auth.agayon.be/auth', params=payload)
        if r.status_code == 200:
            try:
                user = User.objects.get(username=username)
            except User.DoesNotExist:
                # Create a new user. There's no need to set a password
                user = User(username=username)
                user.is_staff = False
                user.is_superuser = False
                user.save()
            return user, r.status_code
        if r.status_code == 401:
            print("User {} refused to authenticate".format(username), file=sys.stdout)
            return None, r.status_code
        return None, r.status_code

    def id_generator(self, size=8, chars=string.ascii_letters + string.digits):
        self.transaction_id = ''.join(random.choice(chars) for _ in range(size))
        return self.transaction_id

Settings.py

The setting of the website must be adapted to your needs. In this simple example, the sessions must be enabled (it is the case by default). Our example use cached session but you can use cookies or even databases. See the excellent documentation of Django for additional information.

LOGIN_URL = '/path/to/login/'

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': 'unix:/tmp/memcached.sock',
    }
}

Links

Credits

  • The image comes from the post on Linuxfr (by Chteufleur).
  • The description of the XMPP component coms from its repository (by Chteufleur).

by Arnaud at January 15, 2017 18:30

January 14, 2017

Peter Saint-Andre

Monadnock Valley Press Annual Report 2016

In 2016 I again spent very little time on posting public-domain texts at the Monadnock Valley Press - only a few works by Henry David Thoreau, Virginia Woolf, and Blaise Pascal. Once I finish my forthcoming book on Thoreau, I plan to clear out a backlog of books, essays, and plays by Ibsen, Confucius, Emerson, Montaigne, Spinoza, Petrarch, Lucretius, Epictetus, Seneca, and Cicero. With revenue of $248.62 and expenses of $181.58, we earned only $67.04 in 2016, too. It's a good thing I don't write philosophy books for the money!...

January 14, 2017 00:00

January 13, 2017

Arnaud Joset

Chatty Server

Chatty server is a XMPP bot programmed to run on a server. It can be used

  • as a dynamic toto list.
  • to provide the travel time by car, foot and public transport between two adresses.
  • to give useful information about the load of the server.
  • to give the status of the server. The status of the account change depending on the load and memory usage.

The following sections describes the usage of chatty_server.

When it start

At start, the program set a status base on the load and the memory usage.

‎18:40:29 ‎server‎: Agayon.be en ligne
‎18:40:30 ‎server‎: How you doin' ? Load: 0.12 0.05 0.05 Memory (Go): 1.08GiB/1.96GiB (35.2%)
‎18:40:30 ‎server is now Available (How you doin' ? Load: 0.12 0.05 0.05 Memory (Go): 1.08GiB/1.96GiB (35.2%))
Get the list of available commands

The list of command is send whenever the user send a message to the bot.

‎18:40:58 ‎jnanar‎: help
‎18:40:58 ‎server‎: Available Commands : status, reminder, travel

Calculate a travel time

Calculate a travel time is easy and fast. You need to define a Google Maps API key to use this function. You can also, define

  • your address,
  • the summary key: it is the summary of this travel( a highway, a long road, etc),
  • the shortest distance between work and home,
  • as well as the duration of this particular travel.

When the distance differs or time differs too much from these data, the bot will remind you to take another road.

18:22:22 jnanar: travel
18:22:23 server: Usage: travel car start % stop
conveyance = car, bus, metro, tramway etc.
Example: travel car rue du parc 4000 Liège % Place saint Lambert 15 4000 Liège

Examples

18:42:08 jnanar: travel car rue du parc 4000 Liège % Place saint Lambert 15 4000 Liège
18:42:09 server: travel car
travel rue du parc 4000 liège to place saint lambert 15 4000 liège
Summary: Quai Orban
Distance: 2.46 km
Duration in traffic: 9.22 min
Duration (normal): 8.05 min

In order to use the travel time function you need to define an API key, a work location and a home location in the file named "libs/credentials.py". Some shortcuts are available: h2w and w2h for home and work location.

18:49:10 jnanar: travel h2w
18:49:11 server: home address to work address
Summary: E42
Distance: 7.81 km
Duration in traffic: 11.03 min
Duration (normal): 11.13 min

Obtain the status of the server

18:34:13 jnanar: status
18:34:13 server:
Load: 2.12 2.26 2.31 Memory (Go): 3.94GiB/11.23GiB (17.6%)
CPUS : ('14.5%', '11.9%', '19.0%', '10.9%', '35.7%', '7.6%')
Memory (Go): 3.94GiB/11.23GiB (17.6%)

Use it as a reminder or a TODOlist.

‎‎18:42:45 ‎jnanar‎: reminder list
‎18:42:45 ‎server‎: Todo list
‎18:42:57 ‎jnanar‎: reminder write a blog article 5min
‎18:42:57 ‎server‎: Reminder: write a blog article at 2017-01-04 18:48:04.797349
‎18:43:23 ‎jnanar‎: reminder 2h program my personal robot
‎18:43:23 ‎server‎: Reminder: program my personal robot at 2017-01-04 20:43:30.599113
‎18:43:29 ‎jnanar‎: reminder list
‎18:43:29 ‎server‎: Todo list
todo: due write a blog article at 2017-01-04 18:48:04.797349
todo: due program my personal robot at 2017-01-04 20:43:30.599113
‎18:43:57 ‎jnanar‎: reminder 1min do a test
‎18:43:57 ‎server‎: Reminder: do a test at 2017-01-04 18:45:04.779812
‎18:44:05 ‎jnanar‎: reminder list
‎18:44:05 ‎server‎: Todo list
todo: due write a blog article at 2017-01-04 18:48:04.797349
todo: due program my personal robot at 2017-01-04 20:43:30.599113
todo: due do a test at 2017-01-04 18:45:04.779812
‎18:45:01 ‎server‎:
---
Reminder: do a test
---
‎18:45:10 ‎jnanar‎: reminder list
‎18:45:10 ‎server‎: Todo list
todo: due write a blog article at 2017-01-04 18:48:04.797349
todo: due program my personal robot at 2017-01-04 20:43:30.599113
todo: done do a test at 2017-01-04 18:45:04.779812

chatty_server is written in python v3 and is licensed under the GPLv3 licence.

Install

Install the dependencies

Clone the repository

  • git clone git@gitlab.com:r1dScripts/chatty_server.git
  • Modify libs/credentials.example.py and rename it as libs/credentials.py.

by Arnaud at January 13, 2017 17:00

January 12, 2017

Erlang Solutions

Build a complete iOS messaging app using XMPPFramework - Part 2

First steps: XMPPFramework

Build a complete iOS messaging app using XMPPFramework is a tutorial that shows you how to build a fully functional instant messaging iOS app using the very cool XMPPFramework protocol and Swift3. In this part, we are going to get our hands dirty! To recap on the theory, or if you just landed here randomly, have a quick read through the first part, then get your Xcode ready and let's start!

In this issue we are going to be integrating the library to our project, creating a connection with the server and authenticating. The XMPPFramework library is the most used XMPP library for iOS and macOS. At the beginning it may be a little bit overwhelming but after a few days working with it you will learn to love it.

Installing the library

Let's create a brand new Xcode project and install the library. In this tutorial we are going to be using Swift 3. The easiest way to integrate XMPPFramework to the project is using CocoaPods.

Let's create our Podfile using the pod init command in the folder where our .xcodeproj lives. There are thousands of forks but the maintained one is the original: robbiehanson/XMPPFramework.

So let's add the pod to our Podfile and remember to uncomment the use_frameworks!.

use_frameworks!

target 'CrazyMessages' do
    pod 'XMPPFramework', :git=> 'git@github.com:robbiehanson/XMPPFramework.git', :branch => 'master'
end

 

Then pod install and CocoaPods is going to do its magic and create a .xcworkspace with the library integrated. Now we just need to import XMPPFramework in the files we want to use the library and that's it.

 

Starting to build our Instant Messaging app

The most important thing in an XMPP application is the stream, that's where we are going to "write" our stanzas, so we need an object that is going to hold it. We are going to create an XMPPController class with an XMPPStream:

import Foundation
import XMPPFramework

class XMPPController: NSObject {
    var xmppStream: XMPPStream

    init() {
        self.xmppStream = XMPPStream()  
    }

}

 

We are dealing with a highly asynchronous library here. For every action we are going to have a response some time in the future. To handle this XMPPFramework defines the XMPPStreamDelegate. So implementing that delegate is going to help us answer lots of different questions like: "How do I know when XMPP has successfully connected?", "How do I know if I'm correctly authenticated?", "How do I know if I received a message?". XMPPStreamDelegate is your friend!

So we have our XMPPController and our XMPPStream, what do we need to do now? Configure our stream with the hostNameport and ourJID. To provide all this info to the controller we are going to make some changes to the init to be able to receive all these parameters:

enum XMPPControllerError: Error {
    case wrongUserJID
}

class XMPPController: NSObject {
    var xmppStream: XMPPStream

    let hostName: String
    let userJID: XMPPJID
    let hostPort: UInt16
    let password: String

    init(hostName: String, userJIDString: String, hostPort: UInt16 = 5222, password: String) throws {
        guard let userJID = XMPPJID(string: userJIDString) else {
            throw XMPPControllerError.wrongUserJID
        }

        self.hostName = hostName
        self.userJID = userJID
        self.hostPort = hostPort
        self.password = password

        // Stream Configuration
        self.xmppStream = XMPPStream()
        self.xmppStream.hostName = hostName
        self.xmppStream.hostPort = hostPort
        self.xmppStream.startTLSPolicy = XMPPStreamStartTLSPolicy.allowed
        self.xmppStream.myJID = userJID

        super.init()

        self.xmppStream.addDelegate(self, delegateQueue: DispatchQueue.main)
    }
}

 

Our next step is going to actually connect to a server and authenticate using our userJID and password, so we are adding a connect method to our XMPPController.

func connect() {
    if !self.xmppStream.isDisconnected() {
        return
    }

   try! self.xmppStream.connect(withTimeout: XMPPStreamTimeoutNone)
}

 

But how do we know we have successfully connected to the server? As I said earlier, we need to check for a suitable delegate method from XMPPStreamDelegate. After we connect to the server we need to authenticate so we are going to do the following:

extension XMPPController: XMPPStreamDelegate {

    func xmppStreamDidConnect(_ stream: XMPPStream!) {
        print("Stream: Connected")
        try! stream.authenticate(withPassword: self.password)
    }

    func xmppStreamDidAuthenticate(_ sender: XMPPStream!) {
        self.xmppStream.send(XMPPPresence())
        print("Stream: Authenticated")
    }
}

 

We need to test this. Let's just create an instance of XMPPController in the AppDelegate to test how it works:

try! self.xmppController = XMPPController(hostName: "host.com",
                                     userJIDString: "user@host.com",
                                          password: "password")
self.xmppController.connect()

If everything goes fine we should see two messages in the logs but of course that's not happening, we missed something. We never told to our xmppStream who was the delegate object! We need to add the following line after the super.init()

self.xmppStream.addDelegate(self, delegateQueue: DispatchQueue.main)

If we run the app again:

Stream: Connected
Stream: Authenticated

 

Success! We have our own XMPPController with a fully functional and authenticated stream!

Something that may catch your attention is how we are setting our delegate, we are not doing:

self.xmppStream.delegate = self

 

Why not? Because we can "broadcast" the events to multiple delegates, we can have 10 different objects implementing those methods. Also we can tell what's the thread where we want to receive that call, in the previous example we want it in the main thread.

Getting a Log In

Our app is super ugly, let's put on some makeup! We have nothing but an XMPPController and a hardcoded call in the AppDelegate. I'm going to create a ViewController that is going to be presented modally as soon as the app starts, that ViewController will have the neccesary fields/info to log in to the server.

I'm going to create a LogInViewControllerDelegate that is going to tell to our ViewController that the Log in button was pressed and that's it. In that delegate implementation we are going to create our XMPPController, add the ViewControlleras delegate of the XMPPStream and connect!

extension ViewController: LogInViewControllerDelegate {

    func didTouchLogIn(sender: LogInViewController, userJID: String, userPassword: String, server: String) {
        self.logInViewController = sender

        do {
            try self.xmppController = XMPPController(hostName: server,
                                                     userJIDString: userJID,
                                                     password: userPassword)
            self.xmppController.xmppStream.addDelegate(self, delegateQueue: DispatchQueue.main)
            self.xmppController.connect()
        } catch {
            sender.showErrorMessage(message: "Something went wrong")
        }
    }
}

 

Why are we adding ViewController as a delegate of XMPPStream if our XMPPController alreay has that delegate implemented? Because we need to know if this connection and authentication was successfull or not in our ViewController so we are able to dismiss the LogInViewController or show an error message if something failed. This is why being able to add multiple delegates is so useful.

So as I said I'm going to make ViewController to comform to the XMPPStreamDelegate:

extension ViewController: XMPPStreamDelegate {

    func xmppStreamDidAuthenticate(_ sender: XMPPStream!) {
        self.logInViewController?.dismiss(animated: true, completion: nil)
    }

    func xmppStream(_ sender: XMPPStream!, didNotAuthenticate error: DDXMLElement!) {
        self.logInViewController?.showErrorMessage(message: "Wrong password or username")
    }

}

 

And that's it! Our app can log in to our server as I'm showing here:

Logging!

We've been talking a lot about XMPP, stanzas and streams... but is there a way I can see the stream? Yes SR! XMPPFramework got us covered!

XMPPFramework ships with CocoaLumberJack, a pretty well known logging framework. We just need to configure it, set the logging level we want and that's it. Logs are going to start showing up!

Configuring CocoaLumberjack

This is a really simple task, you just need to add to your func application(application: UIApplication, didFinishLaunchingWithOptions ... method the following line (remember to import CocoaLumberjack):

DDLog.add(DDTTYLogger.sharedInstance(), with: DDLogLevel.all)

I'm not going to paste here all the connection process log because it makes no sense to try to understand what's going on at this stage of our learning. But I think showing what some stanzas look like is a good idea. To do this I'm going to be sending messages from Adium.

I'm going to send this <message/>:

<message to="test.user@erlang-solutions.com">
    <body>This is a message sent from Adium!</body>
</message>

 

Let's see how it looks like when it reaches our app:

<message xmlns="jabber:client" from="iamadium@erlang-solutions.com/MacBook-Air" to="test.user@erlang-solutions.com">
   <body>This is a message sent from Adium!</body>
</message>

 

Let's send a <presence/> from Adium:

<presence>
    <status>On vacation</status>
</presence>

 

We are receiving:

<presence xmlns="jabber:client" from="iamadium@erlang-solutions.com/MacBook-Air" to="test.user@erlang-solutions.com">
   <status>On vacation</status>
</presence>

 

No doubts at all right? We send something and we receive it on the other end! That's it!

Test Time!

I want to be sure that you are understanding and following everything and not just copy and pasting from a tutorial (as I usually do 🙊). So if you are able to answer these questions you are on a good track!

  • Why am I sending a presence after successfully authenticating? What happens if I don't send it?
  • What happens if I write a wrong server URL in the Log In form? How do I fix this problem if there is a problem...
  • How do I detect if suddenly the stream is disconnected from the server? (maybe a network outage?)
  • How do I detect if the user/password was wrong?

If you need help leave a message!

The sample project is on Github!

The next part is going to be on Roster, and if I will have space I would also like to add sending and receiving messages. I've been kind of super busy lately so I'm not sure when I'm going to be able to deliver the next issue but I'll try to work on it as soon as I have some free minutes to spare!

PS: Also take a look at MongooseIM, our XMPP based open source mobile messaging platform. 

 

January 12, 2017 10:37

January 10, 2017

Prosodical Thoughts

Prosody 0.9.12 released

We are pleased to announce a new minor release from our stable branch.

This release fixes a few minor issues, and fixes valid certificates failing to verify correctly when Prosody 0.9.x is used with LuaSec 0.6.

A summary of changes in this release:

Minor changes

  • Dependencies: Fix certificate verification failures when using LuaSec 0.6 (fixes #781)
  • mod_s2s: Lower log message to 'warn' level, standard for remotely-triggered protocol issues
  • certs/Makefile: Remove -c flag from chmod call (a GNU extension)
  • Networking: Prevent writes after a handler is closed (fixes #783)

Download

As usual, download instructions for many platforms can be found on our download page

If you have any questions, comments or other issues with this release, let us know!

by The Prosody Team at January 10, 2017 19:57

January 08, 2017

Peter Saint-Andre

Thoreau on Magnanimity

Reading is one thing; understanding is another. Part of what I do while "working through" the writings of a thinker like Thoreau is to delve deeply into key themes, concepts, and even words. Consider, for example, this sentence from Economy, the first chapter of Walden: "To be a philosopher is not merely to have subtle thoughts, nor even to found a school, but so to love wisdom as to live according to its dictates, a life of simplicity, independence, magnanimity, and trust." When reading on the surface, it's easy to gloss over a sentence like this and merely have a pleasant feeling that it's a good thing to live philosophically. Yet what exactly is Thoreau talking about here? What does he mean by the somewhat vague and uncommon word magnanimity? To figure that out can require quite a bit of digging....

January 08, 2017 00:00

January 06, 2017

Peter Saint-Andre

Corporatization

While taking my dog to two different doctors the other day (long story), I discovered that veterinary practices are increasing being bought up by large corporate entities like PetVet and VCA. Your seemingly small, local caregiver just might have sold his or her practice to one of these companies, supposedly while still retaining control over medical decision-making (that little voice in the back of my head is saying "he who pays the piper, calls the tune")....

January 06, 2017 00:00

January 03, 2017

Ignite Realtime Blog

First Release Candidate of Smack 4.2 has been released

The Ignite Realtime community is proud to announce that nearly 2 years after the release of Smack 4.1, the first Release Candidate (RC) of Smack 4.2 has been uploaded to Maven Central. Smack 4.2 marks another milestone in the development of Smack. With the availability of the first RC, the API of Smack 4.2 was sealed. This means that now is the ideal time for Smack users to adopt their codebase to the new Smack 4.2 API, and eventually start using Smack 4.2 in the experimental/development branch of their codebase. Please consult the Smack 4.2 Readme and Upgrade Guide for more information.

 

I'd like to use this post also to draw your attention at a very important topic. The codebase of smack-core and -tcp has grown historically over the past 15 years. This is not an issue per se. Smack was well designed from the beginning, is now modular and not affected by bit rot. But especially on important class, namely XMPPTCPConnection, has come to age. It is based on threads, when it should use NIO. It uses triggers for state changes, when it should be designed as finite state machine. And some more. I know that a lot of people are affected by Smack creating at least 2 threads per connection (instead of using NIO). This all also contributed at some amount to the latest security vulnerability found in Smack (SMACK-739 / CVE 2016-10027).

 

The only solution to tackle this would be to re-implement the affected code from scratch. But needles to say that this also would require funding, as it is not a simple one weekend task. I hope to have the resource to do this at some point in the future. If you think you can help, or know someone who possibly would be interested support the funding, then please contact me.

by Ignite Realtime Blog (igniterealtime@jiveon.com) at January 03, 2017 14:14

January 02, 2017

Fanout Blog

Push and reliability

In push architectures, one of the main challenges is delivering data reliably to receivers. There are many reasons for this:

  • Most push architectures use the publish-subscribe messaging pattern, which is unreliable.
  • TCP’s built-in reliability is not enough, as modern network sessions span multiple connections.
  • Receivers can’t tell the difference between data loss and intentional silence.
  • There is no one size fits all answer.

The last point trips up developers new to this problem space, who may wish for push systems to provide “guaranteed delivery.” If only it were that simple. Like many challenges in computer science, there isn’t a best answer, just trade-offs you are willing to accept.

Below we’ll go over the various issues and common practices around reliable push, plus a unique approach we developed at Fanout.

...

by justin at January 02, 2017 19:12

January 01, 2017

Peter Saint-Andre

Philosophy and Ideology

I know someone who only listens to the music of Bob Dylan. Now, I respect Dylan's songwriting immensely and I play a number of his songs, but I don't think you can call someone who only listens to Bob Dylan a lover of music. That requires what jazz musicians call big ears: an appreciation for a wide variety of musical styles, composers, and performers....

January 01, 2017 00:00

December 31, 2016

Ignite Realtime Blog

Openfire 4.1.1 Release

The Ignite Realtime Community is pleased to announce the release of Openfire 4.1.1.  This release contains a few database related bugfixes that only impacted those who upgraded to 4.1.0 and particularly those using MySQL or SQL Server databases.

 

OSsha1sumFilename
Linux RPM (+32bit JRE)1a71272fbc29fb1239170f878ad803f3136082e1openfire-4.1.1-1.i686.rpm
Linux RPM (No JRE)d56d83eaeca9c1cfe750e9fdbf1b1b0ef9cda74fopenfire-4.1.1-1.noarch.rpm
Linux RPM (+64bit JRE)d17abe02887e991a442d2906e67dbf821ec82fdfopenfire-4.1.1-1.x86_64.rpm
Linux Debianab16f19cbf56fdf592c3cb367e867079c7bbf9b4openfire_4.1.1_all.deb
Mac DMGc5d09b44e4bba74e04b0029799f4b6e6f1b190fdopenfire_4_1_1.dmg
Windows EXE762bbb2d2aa38ecb00c9b50841a9055d52ebd9deopenfire_4_1_1.exe
Binary (tar.gz)36157c519b323007acd2067739e27fe242b58465openfire_4_1_1.tar.gz
Binary (zip)ae67a8763b0a4c7bf64b97becd22f07ee5233369openfire_4_1_1.zip
Source (tar.gz)01b060137c44e215acfcb30f77dae159cc4b5a06openfire_src_4_1_1.tar.gz
Source (zip)28cb1739d0c2bdc1aacfc8d7639a7a9ee38532d9openfire_src_4_1_1.zip

 

You can download it from here:

Ignite Realtime: Downloads

As a reminder, Openfire's source code and development can be found on Github and an active developer discussion occurs within our open_chat@conference.igniterealtime.org groupchat.  Please consider helping out with Openfire's development!

 

As always, please report issues in the Community Forums and thanks for your interest in Openfire.

by Ignite Realtime Blog (igniterealtime@jiveon.com) at December 31, 2016 19:17

Peter Saint-Andre

The Irrelevance of God

While doing research for my forthcoming book on Henry David Thoreau, I've been exploring the philosophies of ancient India (Thoreau was a great admirer of the Bhagavat-Gita, the Rig-Veda, and the Laws of Manu). In so doing, I recently came across the Samkhya school of thought, which is an atheistic strand of Hinduism closely related to Yoga philosophy (apparently the Yoga Sutras of PataĂąjali grew out of the Samkhya tradition). Interestingly, Samkhya is more nontheistic than anti-theistic: in essence it treats god or gods as irrelevant to life and ethics. This approach is in line with my attitude; although I have been a non-believer since the age of nine (as described in my very first journal post from 1989), I am not militantly anti-theist and the underlying theme of my lifelong philosophy project is how to be good without god....

December 31, 2016 00:00

December 30, 2016

Peter Saint-Andre

The Irrelevance of God

For students of American culture, one of the most fascinating books of the year was Hillbilly Elegy by J.D. Vance. The author's descriptions of his troubled upbringing in the rural hills of Kentucky and a fading industrial town in Ohio have spurred casual observers to note that these are just the kinds of places that switched political allegiances in 2016. Yet cries of "Trump!" uncover only a minuscule percentage of the story. To truly understand what's going on here requires a long-term historical perspective....

December 30, 2016 00:00

December 28, 2016

Peter Saint-Andre

The Borderer's Lament

When I quit Twitter back in early November, I promised to post a journal entry every Friday night. Although I've kept that up for eight weeks, there's a slight problem: I have too much to say! Already I have quite the backlog of entries to write, and there are times when something topical cries out for a post on a day other than Friday. Case in point: today the RFC Editor published my latest RFC, on SIP-XMPP interoperability with respect to presence functionality. Here's the backstory....

December 28, 2016 00:00

December 23, 2016

Ralph Meijer

Changes

For me, Christmas and Jabber/XMPP go together. I started being involved with the Jabber community around the end of 2000. One of the first things that I built was a bot that recorded the availability presence of my online friends, and show this on a Christmas tree. Every light in the tree represents one contact, and if the user is offline, the light is darkened.As we are nearing Christmas, I put the tree up on the frontpage again, as many years before.

Over the years, the tooltips gained insight in User Moods and Tunes, first over regular Publish-Subscribe, later enhanced with the Personal Eventing Protocol. A few years later, Jingle was born, and in 2009, stpeter wrote a great specification that solidifies the relationship between Christmas and Jabber/XMPP.

Many things have changed in those 16 years. I've changed jobs quite a few times, most recently switching from the Mailgun team at Rackspace, to an exciting new job at VimpelCom as Chat Expert last April, working on Veon (more on that later). The instant messaging landscape has changed quite a bit, too. While we, unfortunately, still have a lot of different incompatible systems, a lot of progress has been made as well.

XMPP's story is long from over, and as such I am happy and honored to serve as Chair of the XMPP Standards Foundation since last month. As every year, my current focus is making another success of the XMPP Summit and our presence with the Realtime Lounge and Devroom at FOSDEM in Brussels in February. This is always the highlight of the year, with many XMPP enthousiasts, as well as our friends of the wider Realtime Communications, showing and discussing everything they are working on, ranging from protocol discussions to WebRTC and IoT applications.

Like last year, one of the topics that really excite me is the specification known as Mediated Information eXchange (MIX). MIX takes the good parts of the Multi User Chat (MUC) protocol, that has been the basis of group chat in XMPP for quite a while, redesigned on top of XMPP Publish-Subscribe. Modern commercial messaging systems, for business use (e.g. Slack and HipChat), as well as for general use (e.g. WhatsApp, WeChat, Google's offerings), have tried various approaches on the ancient model of multi-part text exchange, adding multi-media and other information sources, e.g. using integrations, bots, and cards.

MIX is the community's attempt to provide a building block that goes beyond the tradional approach of a single stream of information (presence and messages) to a collection of orthogonal information streams in the same context. A room participant can select (manually or automatically by the user agent) which information streams are of interest at that time. E.g. for mobile use or with many participants, exchanging the presence information of all participants can be unneeded or even expensive (in terms of bandwidth or battery use). In MIX, presence is available as a separate stream of information that can be disabled.

Another example is Slack's integrations. You can add streams of information (Tweets, continuous integration build results, or pull requests) to any channel. However, all participants have no choice to receive the resulting messages, intermixed with discussion. The client's notification system doesn't make any distinction between the two, so you either suffer getting alerts for every build, or mute the channel and possibly miss interesting discussion. The way around it is to have separate channels for notifications and discussion, possibly muting the former.

Using MIX, however, a client can be smarter about this. It can offer the user different ways to consume these information streams. E.g. notifications on your builds could be in a side bar. Tweets can be disabled, or modeled as a ticker. And it can be different depending on which of the (concurrent) clients you are connected with. E.g. the desktop or browser-based client has more screen real-estate to show such orthogonal information streams at the same time, a mobile client might still show the discussion and notifications interleaved.

All-in-all MIX allows for much richer, multi-modal, and more scalable interactions. Some of the other improvements over MUC include persistent participation in channels (much like IRC bouncers, but integrated), better defined multi-device use (including individual addressing), reconnection, and message archiving. I expect the discussions at the XMPP Summit to tie the loose ends as a prelude to initial implementations.

I am sure that FOSDEM and the XMPP Summit will have many more exciting topics, so I hope to see you there. Until then, Jabber on!

by ralphm at December 23, 2016 13:28

ProcessOne

ejabberd 16.12

2016 was year of several major code refactors and improvements for ejabberd. From Elixir support, to test suite and code clean up and modernization.

After ejabberd 16.09 which brings a lot of improvements, ejabberd 16.12 includes a big refactor we have been preparing for long time, and a couple of new features that pursue the central effort on API, commands and web integration started early this year.

ejabberd 16.12 includes: a new API permissions framework for commands, a new more reliable BOSH module, major code refactor, more integration and unit tests, improved support for Erlang/OTP R19, compatibility with rebar3 build system, many fixes and optimizations, and Docker containers.

Thank you to all users who have send back their feedback on ejabberd 16.12 beta version. This is helping a lot make ejabberd better.

We hope you will enjoy this new release and we wish you a merry Christmas ! Stay tuned for even more new features in 2017 !

fb-ejabberd-16.12

New BOSH module

This new BOSH implementation improves BOSH users experience, as it’s more robust and efficient and overall over lower latency.

New API permissions framework

This new framework is one of the big changes in this release. It makes our API very flexible, allowing administrators to fine grain access to some users to a group of API, depending on the path used to call a given command.

While it should make it possible to implement any kind of access control on top of ejabberd management API, it also makes it much easier to configure ejabberd API for simple needs.

ejabberd API operations are organised around the concept of commands. ejabberd modules provide many commands, but the mechanism is generic and any module can provide its own set of commands.

All commands can be exposed through interfaces. Available interfaces are: ejabberdctl command-line tool, ejabberd ReST API and ejabberd XML-RPC API. The XML-RPC API still works but is deprecated in favor of the ReST API. You should migrate to ReST if you are using it.

Finally, ReST API can be accessed through two authentication mechanisms: HTTP Basic Authentication and OAuth 2.

Each command interface can have different restrictions based on how exposed and sensitive the commands are.

Note that the following configuration snippets assume ejabberd API listeners are properly configured, as defined in API listener configuration

By default, when no api_permission option is provided, ejabberd would use the following default permissions:

api_permissions:
 "console commands":
   - from:
     - ejabberd_ctl
   - who: all
   - what: "*"
 "admin access":
   - who:
     - admin
     - oauth:
       - scope: "ejabberd:admin"
       - admin
   - what:
     - "*"
     - "!stop"
     - "!start"

This will grant access to all commands when ejabberdctl is used (this is what “console commands” group does), and additionally each command that is authenticated by user that match acl auth from any source will be able to call all commands except start and stop.

It’s possible to extend this by adding new section like this:

api_permissions:
 ...
 "allow statbot to get server stats"
   - who:
     - user: "statbot@server.example.com"
   - what:
     - connected_users_number
     - num_active_users

This will allow user statbot to execute commands connected_users_number and num_active_users.

You can even relax the need to authenticate on ReST API, for example, if you only want to restrict the API to localhost usage, for admin tasks:

api_permissions:
  "local admin can use all commands"
    - who:
      - ip: "127.0.0.1/8"
    - what:
      - "*"
      - "!stop"
      - "!start"

As you can see, for simple cases, the API permission management is now very simple. Yet, the system is powerful and can accommodate your most advanced needs.

For more details, see full api permission documentation.

You can now rely on ejabberd hundreds of commands to integrate in a precise and secure way with your back ends.

Major code refactor

Finally, the most important change (in terms of line of code and impact on ejabberd code base) is refactor of XMPP packet handling in the entire code base. This improvement makes code simpler, safer, and smaller. ejabberd now uses a dedicated XMPP library, which helps developers packing/unpacking XMPP packets.

Example is always better than words:

Don’t build entire XML element, but rely on common “templates”

Before:

#xmlel{name = <<"identity">>,
       attrs = [{<<"category">>, <<"pubsub">>},
                {<<"type">>, <<"pep">>}]}
IQ#iq{type = error, sub_el = [Error, SubEl]}

After:

#identity{category = <<"pubsub">>, type = <<"pep">>}
xmpp:make_iq_result(IQ, Error);

Match packets efficiently

Before:

normal_state({route, From, <<"">>,
       #xmlel{name = <<"message">>,
              attrs = Attrs,
              children = Els} = Packet},
       StateData) ->
   case is_user_online(From, StateData) of
     true ->
       case fxml:get_attr_s(<<"type">>, Attrs) of
         <<"groupchat">> ->
              ...

After:

normal_state({route, From, <<"">>,
     #message{type = Type,
              lang = Lang} = Packet},
     StateData) ->
   case is_user_online(From, StateData) of
      true when Type == groupchat ->
          ...

Don’t use macros to build error response

Before:

jlib:make_error_reply(Packet, ?ERRT_BAD_REQUEST(Lang, Txt));

After:

xmpp:make_error(Packet, xmpp:err_bad_request(Txt, Lang));

The code is safer, as all XMPP processing in the core of ejabberd can be typed-checked through static analyzer like Dialyzer.

Overall, this change paves the way to improvements of ejabberd API and will make it much more pleasant for contributors to write new modules and extensions. Improving the contribution documentation is next on our list and you are welcome to join the effort.

This big refactor impacts the entire code base. Now that all tests are successful, we provide an early beta before the final 16.12 release. 16.12-beta1 is not yet intended for production use. We are waiting for your feedback to check that special cases did not sleep through our careful refactor. Please, test it and send us feedback in your use case, so that we can catch the tricky bugs before the final version is released.

Docker containers

Ejabberd docker container contributed by Rafael Römhild is now included in ejabberd sources.
We also provide simple containers to build and run ejabberd from sources. Thanks to this image, you can build ejabberd with dependencies provided in our Docker image, without the need to install any specific tools (beside Docker) directly on your own machine. This works on Linux, FreeBSD, MacOSX, Windows.
See more in this blogpost.

External modules

Ejabberd’s ext_mod has been updated to build external modules with the new xmpp library. You should port your contribution to make it compile and work with ejabberd 16.12. While this will improve your code, it will also offer possibility to implement post_install and pre_uninstall directly in your module. By exporting these functions, ejabberd will call them after module has been successfully installed and right before uninstalling it to allow any custom tasks at these stages.

Windows installer

The installer now moves ejabberd data files (configuration, database and logs) in %ProgramData% instead of %APPDATA%. This move fixes running ejabberd as a service. As a result, installer now registers ejabberd service by default. If you’re upgrading ejabberd from an older version, the installer will automatically move your configuration from %APPDATA% to %ProgramData% to keep your existing data without any manual operation.

Changes

This is just a summary of the most relevant ones:

API / integration

  • New API permissions framework

Commands

  • Add configurable weight for ejabberd commands
  • add_rosteritem: Support several groups separated by ;
  • create_rooms_file: Fix reading room jids from file
  • delete_old_messages: Fix command for SQL backends
  • send_message: Don’t duplicate the message
  • Remove obsolete remove_node command (use leave_cluster)
  • Fix reload_config
  • Cleanup mod_admin_extra, add few functions
  • Expose unregister API command

Core XMPP

  • New BOSH module
  • Use fxml_gen XML generator
  • Use our new stand-alone XMPP library instead of jlib.erl
  • Don’t let MAM messages go into offline storage
  • Add xdata generator for XMPP data form
  • Get rid of excessive (io)list_to_binary/1 calls

HTTP

  • Add authentication support to mod_http_fileserver
  • ejabberd_http: Handle missing POST data gracefully
  • Use inets instead of lhttpc in http_p1
  • Add http_p1.erl, rest.erl, and oauth2 ReST backend for OAuth2 tokens

MUC

  • Create room on configuration request as per XEP-0045, 10.1.3
  • Ensure that presence_broadcast room option is stored
  • Fix conference disco#items when running multiple virtual hosts
  • Fix Result Set Management (RSM) for conference disco#items
  • Introduce muc_invite hook
  • Make the constant MAX_ROOMS_DISCOITEMS configurable
  • mod_carboncopy: Don’t copy MUC private messages

MUC/Sub

  • Store the flag “Allow Subscription” room option in database
  • When getting list of subscribed rooms, also check temporary ones
  • Add password support in muc_subscribe
  • When unsubscribes, check if room should get closed

Pubsub

  • Enforce pubsub node removal

Relational databases support

  • Append ; to privacy_list_data exporting lines
  • Improve relational database import

Build

  • Make build system compatible with rebar3
  • Produce ejabberd.service and fix for systemd usage
  • Cleanup ext_mod and fix compilation path
  • Fix compilation of external module with new xmpp lib
  • Add optional post_install and pre_uninstall hooks for external module

Miscelanea

  • Add docker container from Rafael Römhild
  • fast_tls is now compatible with LibreSSL
  • Delete obsolete module mod_configure2
  • Rename #error{} record to #stanza_error{}
  • Bugfix: Ignore offline sessions in ejabberd command statistics
  • Bugfix: Don’t let MAM messages go into offline storage

Feedback

As usual, the release is tagged in the Git source code repository on Github.

The source package and binary installers are available at ProcessOne.

If you suspect that you’ve found a bug, please search or fill a bug report on Github.

by Christophe Romain at December 23, 2016 09:41

Peter Saint-Andre

RFC 8048: SIP-XMPP Presence Revisited

Just about thirteen years ago I posted a list of "desert island dics" that I couldn't do without (yes, this was back before streaming music services took off, so forgive the anachronism - though I still prefer to own and listen to physical albums). Looking at that list again recently reminds me that I've been meaning to post about some other recent musical finds....

December 23, 2016 00:00

December 21, 2016

Ignite Realtime Blog

Openfire 4.1.0 Final Release

Christmas has come early.

 

Our github repository - GitHub - igniterealtime/Openfire: A XMPP server licensed under the Open Source Apache License. - reached a thousand and one stars today, so we've celebrated by releasing Openfire 4.1.0, containing over 150 fixes and new features. I feel I should personally thank Daryl Herzmann and Guus der Kinderen in particular for the gargantuan effort they've both put into this release, but we're also very lucky to have a large number of developers within the community providing us with high-quality patches via Pull Requests. This is, once again, a huge collective effort.

 

Changlog: Openfire Changelog

 

 

Platform
sha1sumFilename
Mac

3e245dc7cd67707847bba605ec4942a81438066d

openfire_4_1_0.dmg
Windowsd590ebf8e3ca429b1d738833e14419c3dd514110openfire_4_1_0.exe
Binary (tar.gz)

074021df4cc60af0f18879421c3b9e11316dad91

openfire_4_1_0.tar.gz
Binary (zip)

2e933881c80d9ded13027679e6a0465f20d812eb

openfire_4_1_0.zip
Source (tar.gz)cf458223b8acf23c84219f343d5d14bebfe9a963openfire_src_4_1_0.tar.gz
Source (zip)a4a637354be17031be64f7cff9443584dde4022dopenfire_src_4_1_0.zip
RPM Installer (32-bit JRE)

8ed5af7cbd41b48c866dc756cd8d927c34b39400

openfire-4.1.0-1.i686.rpm
RPM Installer (64-bit JRE)

ec257c300732d00d323f44f02a6d97d4030068fa

openfire-4.1.0-1.x86-64.rpm
RPM Installer (No JRE)

88d8bb61f2437d0ad1612d6f6464bf5dacd2425d

openfire-4.1.0-1.noarch.rpm
DEB Installer9ea6b83a7aa9229be0358385e02985a9df04418eopenfire_4.1.0_all.deb

 

You can download it from here:

 

Ignite Realtime: Downloads

 

As a reminder, Openfire's source code and development can be found on Github and an active developer discussion occurs within our open_chat@conference.igniterealtime.org groupchat.  Please consider helping out with Openfire's development!  We can use help in about all areas of development, but especially:

  • experienced packaging developers to improve the install/upgrade experience on various platforms
  • developers interested in transitioning Openfire to Maven
  • testing of Bamboo generated builds

 

Thank you for your interest in Openfire!

by Ignite Realtime Blog (igniterealtime@jiveon.com) at December 21, 2016 15:20

December 15, 2016

ProcessOne

ejabberd 16.12 beta1

In 2016, we made several major code refactors and improvements on ejabberd. From Elixir support, to test suite and code clean up and modernization.

After 16.09 which brings a lot of improvements, the last version for 2016 includes a big refactor we have been preparing for long time, and a couple of new features that pursue the central effort on API, commands and web integration started early this year.

ejabberd 16.12 includes: a new API permissions framework for commands, a new more reliable BOSH module, major code refactor, more integration and unit tests, improved support for Erlang/OTP R19, compatibility with rebar3 build system and many fixes and optimizations.

fb-ejabberd-release-1

New BOSH module

This new BOSH implementation improves BOSH users experience, as it’s more robust and efficient and overall over lower latency.

New API permissions framework

This new framework is one of the big changes in this release. It makes our API very flexible, allowing administrators to fine grain access to some users to a group of API, depending on the path used to call a given command.

While it should make it possible to implement any kind of access control on top of ejabberd management API, it also makes it much easier to configure ejabberd API for simple needs.

ejabberd API operations are organised around the concept of commands. ejabberd modules provide many commands, but the mechanism is generic and any module can provide its own set of commands.

All commands can be exposed through interfaces. Available interfaces are: ejabberdctl command-line tool, ejabberd ReST API and ejabberd XML-RPC API. The XML-RPC API still works but is deprecated in favor of the ReST API. You should migrate to ReST if you are using it.

Finally, ReST API can be accessed through two authentication mechanisms: HTTP Basic Authentication and OAuth 2.

Each command interface can have different restrictions based on how exposed and sensitive the commands are.

Note that the following configuration snippets assume ejabberd API listeners are properly configured, as defined in API listener configuration

By default, when no api_permission option is provided, ejabberd would use the following default permissions:

api_permissions:
 "console commands":
   - from:
     - ejabberd_ctl
   - who: all
   - what: "*"
 "admin access":
   - who:
     - admin
     - oauth:
       - scope: "ejabberd:admin"
       - admin
   - what:
     - "*"
     - "!stop"
     - "!start"

This will grant access to all commands when ejabberdctl is used (this is what “console commands” group does), and additionally each command that is authenticated by user that match acl auth from any source will be able to call all commands except start and stop.

It’s possible to extend this by adding new section like this:

api_permissions:
 ...
 "allow statbot to get server stats"
   - who:
     - user: "statbot@server.example.com"
   - what:
     - connected_users_number
     - num_active_users

This will allow user statbot to execute commands connected_users_number and num_active_users.

You can even relax the need to authenticate on ReST API, for example, if you only want to restrict the API to localhost usage, for admin tasks:

api_permissions:
  "local admin can use all commands"
    - who:
      - ip: "127.0.0.1/8"
    - what:
      - "*"
      - "!stop"
      - "!start"

As you can see, for simple cases, the API permission management is now very simple. Yet, the system is powerful and can accommodate your most advanced needs.

For more details, see full api permission documentation.

You can now rely on ejabberd hundreds of commands to integrate in a precise and secure way with your back ends.

Major code refactor

Finally, the most important change (in terms of line of code and impact on ejabberd code base) is refactor of XMPP packet handling in the entire code base. This improvement makes code simpler, safer, and smaller. ejabberd now uses a dedicated XMPP library, which helps developers packing/unpacking XMPP packets.

Example is always better than words:

Don’t build entire XML element, but rely on common “templates”

Before:

#xmlel{name = <<"identity">>,
       attrs = [{<<"category">>, <<"pubsub">>},
                {<<"type">>, <<"pep">>}]}
IQ#iq{type = error, sub_el = [Error, SubEl]}

After:

#identity{category = <<"pubsub">>, type = <<"pep">>}
xmpp:make_iq_result(IQ, Error);

Match packets efficiently

Before:

normal_state({route, From, <<"">>,
       #xmlel{name = <<"message">>,
              attrs = Attrs,
              children = Els} = Packet},
       StateData) ->
   case is_user_online(From, StateData) of
     true ->
       case fxml:get_attr_s(<<"type">>, Attrs) of
         <<"groupchat">> ->
              ...

After:

normal_state({route, From, <<"">>,
     #message{type = Type,
              lang = Lang} = Packet},
     StateData) ->
   case is_user_online(From, StateData) of
      true when Type == groupchat ->
          ...

Don’t use macros to build error response

Before:

jlib:make_error_reply(Packet, ?ERRT_BAD_REQUEST(Lang, Txt));

After:

xmpp:make_error(Packet, xmpp:err_bad_request(Txt, Lang));

The code is safer, as all XMPP processing in the core of ejabberd can be typed-checked through static analyzer like Dialyzer.

Overall, this change paves the way to improvements of ejabberd API and will make it much more pleasant for contributors to write new modules and extensions. Improving the contribution documentation is next on our list and you are welcome to join the effort.

This big refactor impacts the entire code base. Now that all tests are successful, we provide an early beta before the final 16.12 release. 16.12-beta1 is not yet intended for production use. We are waiting for your feedback to check that special cases did not sleep through our careful refactor. Please, test it and send us feedback in your use case, so that we can catch the tricky bugs before the final version is released.

Changes

This is just a summary of the most relevant ones:

API / integration

  • New API permissions framework

Commands

  • Add configurable weight for ejabberd commands
  • add_rosteritem: Support several groups separated by ;
  • create_rooms_file: Fix reading room jids from file
  • delete_old_messages: Fix command for SQL backends
  • send_message: Don’t duplicate the message

Core XMPP

  • New BOSH module
  • Use fxml_gen XML generator
  • Use our new stand-alone XMPP library instead of jlib.erl
  • Don’t let MAM messages go into offline storage
  • Add xdata generator for XMPP data form
  • Get rid of excessive (io)list_to_binary/1 calls

HTTP

  • Add authentication support to mod_http_fileserver
  • ejabberd_http: Handle missing POST data gracefully
  • Use inets instead of lhttpc in http_p1
  • Add http_p1.erl, rest.erl, and oauth2 ReST backend for OAuth2 tokens

MUC

  • Create room on configuration request as per XEP-0045, 10.1.3
  • Ensure that presence_broadcast room option is stored
  • Fix conference disco#items when running multiple virtual hosts
  • Fix Result Set Management (RSM) for conference disco#items
  • Introduce muc_invite hook
  • Make the constant MAX_ROOMS_DISCOITEMS configurable
  • mod_carboncopy: Don’t copy MUC private messages

MUC/Sub

  • Store the flag “Allow Subscription” room option in database
  • When getting list of subscribed rooms, also check temporary ones

Relational databases support

  • Append ; to privacy_list_data exporting lines
  • Improve relational database import

Build

  • Make build system compatible with rebar3
  • Produce ejabberd.service and fix for systemd usage

Miscelanea

  • Bugfix: Don’t let MAM messages go into offline storage
  • Delete obsolete module mod_configure2
  • Bugfix: Ignore offline sessions in ejabberd command statistics
  • Rename #error{} record to #stanza_error{}

Feedback

As usual, the release is tagged in the Git source code repository on Github.

The source package and binary installers are available at ProcessOne.

If you suspect that you’ve found a bug, please search or fill a bug report on Github.

by Christophe Romain at December 15, 2016 14:57

December 14, 2016

Ignite Realtime Blog

Openfire 4.1.0 Beta

A Year and a Day - TV Tropes

 

On the 13th of December, 2015, we released Openfire 4.0.0 Beta.

 

Now, a year later, we have a new Beta to try. This one has numerous major security fixes and enhancements, and a huge long Openfire Changelog to read through.

 

Platform
sha1sumFilename
Mac

0a8ac2adbec60f411ff959a967d9635173da0ef2

openfire_4_1_0_beta.dmg
Windows1834ff1278e0249f4fe8662536683aabd6b839c5openfire_4_1_0_beta.exe
Binary (tar.gz)

73e43670a6ceb3bd987f9d0176eb6a8346b2a581

openfire_4_1_0_beta.tar.gz
Binary (zip)

a84e156284bf1d51f8f1cee24e977846b350944f

openfire_4_1_0_beta.zip
Source (tar.gz)625e92e1dd76c99ebde70b3ae7de6a651f08d4d5openfire_src_4_1_0_beta.tar.gz
Source (zip)9d9fe67f5e62f3d31d2ab6d6a4ebf282a1e01254openfire_src_4_1_0_beta.zip
RPM Installer (32-bit JRE)

a3fb24fd3c33fb3f92ab53b8665d028b68c11e32

openfire-4.1.0-0.2.beta.i686.rpm
RPM Installer (64-bit JRE)

e055fd8eae9cf5f418fcfda7da93b8c3cd6a38bc

openfire-4.1.0-0.2.beta.x86-64.rpm
RPM Installer (No JRE)

71855a71dfe1ab4ad9aa472bd1ebb0b7a702bf04

openfire-4.1.0-0.2.beta.noarch.rpm
DEB Installer9cce98aadd53cb3995b355fd54543459ab80e056openfire_4.1.0.beta_all.deb

 

Please do try this, but not in production of course. Unless you're a mad and crazy bunch, like we are. You can download it from here:

 

Ignite Realtime: Beta Downloads

 

As a reminder, Openfire's source code and development can be found on Github and an active developer discussion occurs within our open_chat@conference.igniterealtime.org groupchat.  Please consider helping out with Openfire's development!  We can use help in about all areas of development, but especially:

  • experienced packaging developers to improve the install/upgrade experience on various platforms
  • developers interested in transitioning Openfire to Maven
  • testing of Bamboo generated builds

 

Thank you for your interest in Openfire!

by Ignite Realtime Blog (igniterealtime@jiveon.com) at December 14, 2016 18:52

December 13, 2016

ProcessOne

Build awesome Internet of Things with ProcessOne

Nabaztag SymphonyYou probably don’t know this, but ProcessOne has been delivering enterprise-grade solutions for device communication since before it was called the Internet of Things. IoT market is growing at a rapid pace, and by 2018 there will be more internet-connected devices than PCs, smartphones and tablets combined!

But back in 2008, we already implemented production-ready infrastructure for IoT. Based on XMPP, of course, it powered the community of Nabaztags – ambient electronic devices in the shape of a rabbit that could communicate with text and voice messages, play music and internet radio, as well as speak various news: weather, stocks, rss, email etc. Its community continued to be very live and open, and keeps supporting the rabbits through private servers.

After the Nabaztag project we were approached by Kodak to create an IoT infrastructure for digital, connected photo frames. Through 2010 we developed XMPP-based connection backend for several series of Kodak digital frames.

Another notable example of our long-term commitment to the Internet of Things is the now decade-long cooperation with IMA Téléassistance, a major European player in security and monitoring of individual and commercial property. With the growing popularity of Smart Home solutions, IMA Téléassistance recently entered this market with additional connected products like electricity plugs, light and motion sensors or smoke detectors. With our proven IoT backend, they had the confidence to expand their device pool.

At ProcessOne, we believe connected devices will power the big change in enterprise and infrastructure operations. It is our opinion that your business will need IoT infrastructure to cut costs, empower employees and optimize processes. And we are here to help you do that – See more and contact us today! »

by Marek Foss at December 13, 2016 12:33

Peter Saint-Andre

Musical Finds

Although I don't live or eat in a consistently paleo fashion, I do like some paleo foods. One of my favorites is grain-free granola. It also happens to be very simple to make. Here's how....

December 13, 2016 00:00

December 09, 2016

Peter Saint-Andre

Grain-Free Granola Recipe

In Walden, Thoreau wrote:...

December 09, 2016 00:00

December 05, 2016

ProcessOne

ejabberd Development with Docker

We have released a Docker image to help you get started with ejabberd development easily.

ejabberd_docker

Docker image for ejabberd developers is available from Docker Hub: ejabberd/mix

Thanks to this image, you can build ejabberd with dependencies provided in our Docker image, without the need to install any specific tools (beside Docker) directly on your own machine.

Please note that this image can likely be reused as is to build other Erlang or Elixir software.

Pulling ejabberd development Docker image

You can pull ejabberd image for developer with this Docker command:

docker pull ejabberd/mix

Once done, you are ready to build ejabberd from source and add your custom plugin modules.

Building ejabberd from source

You can build ejabberd from source with all dependencies, with the following commands:

git clone https://github.com/processone/ejabberd.git
docker run --rm -v $(pwd):$(pwd) -w $(pwd) ejabberd/mix do deps.get, deps.compile, compile

Alternatively if you do not have Git installed, you can do retrieve compressed source code archive and then build ejabberd:

wget https://github.com/processone/ejabberd/archive/master.zip
unzip ejabberd-master
cd ejabberd-master
docker run --rm -v $(pwd):$(pwd) -w $(pwd) ejabberd/mix do deps.get, deps.compile, compile

Run ejabberd with mix command-line tool attached

During development, you will need to run ejabberd with debug console attached.

First, you can customize the configuration used by ejabberd in development mode by editing the file config/ejabberd.yml. As a default, it will run ejabberd with console attached on domain “localhost”:

docker run --rm -it -p 5222:5222 -p 5280:5280 -v $(pwd):$(pwd) -w $(pwd) --entrypoint="/usr/bin/iex" ejabberd/mix -S mix

You can then directly create a user from Elixir shell:


Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false] Interactive Elixir (1.3.4) - press Ctrl+C to exit (type h() ENTER for help) iex(1)> :ejabberd_auth.try_register("test", "localhost", "passw0rd") {:atomic, :ok}

Finally, you can connect with user “test@localhost” (password: passw0rd) on server on localhost port 5222 and use those parameters to connect with an XMPP client.

Get into the container

If you want to run Erlang command line, you can do so by opening a shell inside the container and then running the command as desired:

docker run -it -v $(pwd):$(pwd) -w $(pwd) --entrypoint="/bin/sh" ejabberd/mix

Troubleshooting

Clock resync

If you have warning about file timestamp being out of sync (Like ‘Clock skew detected’), you may want to force resync your clock before running the build. Docker on MacOS does not force clock resync of Docker after the laptop went to sleep.

You can force clock resync as follow:

docker run -it  --rm --privileged --entrypoint="/sbin/hwclock" ejabberd/mix -s

You can check if the clock of your laptop is in sync with the one inside Docker with the following command:

docker run --rm --entrypoint="/bin/sh" ejabberd/mix -c date -u && date -u

Send us feedback

This image is under development and improvement, so please do not hesitate to send us your feedback, so that we can improve ejabberd development experience.

The goal is to provide a simple easy way to develop ejabberd plugins from any OS, whether you use Linux, MacOS or Windows.

Enjoy !

by Mickaël Rémond at December 05, 2016 11:31

December 02, 2016

Erlang Solutions

Build a complete iOS messaging app using XMPPFramework - Tutorial Part 1

YAXT??! Yet another XMPP tutorial?

 

Well, this is going to be another tutorial, but I'm going to try to make it a little bit different. This is an XMPP tutorial from an iOS developer's perspective. I'll try to answer all the questions I had when I started working in this area. This journey is going to go from no XMPP knowldege at all to having a fully functional instant messaging iOS app using this cool protocol. We are going to be using the super awesome (yet overwhelming at the beginning...) XMPPFramework library, and the idea is also to also mix in some iOS concepts that you are going to need for your app.

What's XMPP?

 

From Wikipedia: Extensible Messaging and Presence Protocol (XMPP) is a communications protocol for message-oriented middleware based on XML.

This basically means XMPP is a protocol for exchanging stuff. What kind of stuff? Messages and presences. We all know what messages are, but what about presences? A presence is just a way of sharing a "status", that's it. You can be 'online', 'offline', 'having lunch', or whatever you want. Also there's another important word: Extensible meaning it can grow. It started as an instant messaging protocol and it has grown into multiple fields for example IoT (Internet of Things). And last, but not least: every piece of information we are going to exchange under this protocol is going to be XML. I can heard you complaining but... Come on, it's not that bad!

Why do we need XMPP? Why not just REST?

 

Well what other options do we have? On the one hand, a custom solution means building everything from scratch, that takes time. On the other hand, we have XMPP, a super tested technology broadly used by millions of people every day, so we can say that's an advantage over a custom approach.

Everytime I talk about XMPP, someone asks me 'Why not just REST?'. Well, there is a misconception here. REST is not a protocol, it's just a way of architecting a networked application; it's just a standarized way of doing something (that I love btw). So let's change the question to something that makes more sense: "Why not just build a custom REST chat application?". The first thing that comes to my mind is what I already explained in the previous paragraph, but there is something else. How do I know when someone has sent me a message? For XMPP this is trivial: we have an open connection all the time so, as soon as a message arrives to the server, it will send us the message. We have a full-duplex. On the other hand, the only solution with REST is polling. We will need to ask the server for new messages from time to time to see if there is something new for us. That sucks. So, we will have to add a mechanism that allows us to receive the messages as soon as they are created, like SSE or WebSockets.

There is one more XMPP advantage over a custom REST chat application. REST uses HTTP, an application level protocol that is built on top of a transport level protocol: TCP. So everytime you want to use your REST solution, you will need HTTP, a protocol that is not always available everywhere (maybe you need to embed this in a cheap piece of hardware?). Besides, we have XMPP built on top of TCP that's going to be always available.

What's the basic stuff I need to know to get started?

 

Well, you know a lot already but let's make a list. Lists are always good:

  • XMPP is built on top of TCP. It keeps an open connection all the time.
  • Client/Server architecture. Messages always go through a server.
  • Everything we send and receive is going to be XML and it's called Stanza.
  • We have three different types of stanzas: iq, message and presence.
  • Every individual on the XMPP network is univocally identified by a JID (Jabber ID).
  • All the stanzas are cointained in a Stream. Let's imagine the Stream as a white canvas where you and the server write the stanzas.
  • Stream, iq, message and presence are the core of XMPP. You can find everything perfectly detailed in RFC6120
  • XMPP can be extended to accomplish different stuff. Each extension is called XEP (XMPP Extension Protocol).

 

What's a JID?

Jabber ID (JID) is how we univocally identify each individual in XMPP. It is the address to where we are going to send our stanzas.

This is how a JID looks like:

  • localpart: This is your username.
  • domainpart: Server name where the localpart resides.
  • resourcepart: This is optional, and it identifies a particular client for the user. For example: I can be logged in with andres@erlang-solutions.com on my iPhone, on my Android and on my mac at the same time... So all these will be the same localpart + domainpart but different resourcepart

I'm sure you have already noticed how similar the JID looks to a standard email address. This is because you can connect multiple servers together and the messages are rooted to the right user in the right server, just as email works. Pretty cool, right?

Sometimes you will see we have a JID with just the domain part. Why?! Because it's also possible to send stanzas to a service instead of a user. A service? What's a service?! Services are different pieces of an XMPP server that offer you some special functionality, but don't worry about this right now, just remember: you can have JIDs without a localpart.

What's a Stanza?

Stanza is the name of the XML pieces that we are going to be sending and receiving. The defined stanzas are: <message/><presence/> and <iq/>.

 

<message/>

This is a basic <message/> stanza. Everytime you want to send a message to someone (a JID), you will have to send this stanza:

<message from='andres@erlang-solutions.com/iphone' to='juana@erlang-solutions.com' type='chat'>
    <body>Hey there!</body>
</message>

 

<iq/>

It stands for Info/Query. It's a query-action mechanism, you send an iq and you will get a response to that query. You can pair the iq-query with the iq-response using the stanza id.

For example, we send an iq to the server to do something (don't pay attention to what we want to do... you just need to know there is an iq stanza and how the mechanism works):

<iq to='erlang-solutions.com' type='get' id='1'>
  <query xmlns='http://jabber.org/protocol/disco#items'/>
</iq>

And we get back another iq with the same id with the result of the previous query:

<iq from='erlang-solutions.com' to='ramabit@erlang-solutions.com/Andress-MacBook-Air' id='1' type='result'>
    <query xmlns='http://jabber.org/protocol/disco#items'>
        <item jid='muc.erlang-solutions.com'/>
        <item jid='muclight.erlang-solutions.com'/>
        <item jid='pubsub.erlang-solutions.com'/>
    </query>
</iq>

 

<presence/>

Used to exchange presence information, as you could have imagined. Usually presences are sent from the client to the server and broadcasted by it. The most basic, yet valid presence, to indicate to the server that a user is avaiable is:

<presence/>

After a sucessfull connection, you are not going to receive any <message/> until you make yourself available sending the previous presence.

If you want to make yourself unavailable, you just have to send:

<presence type="unavailable"></presence>

If we want to make the presences more useful, we can send something like this:

<presence>
      <status>On vacation</status>
</presence>

 

What's a Stream?

Before answering this, let's refresh our mind. What's a Unix socket? From Wikipedia: A socket is a special file used for inter-process communication. These allows communication between two processes. So a socket is a file that can be written by two processes (in the same computer or in different computers in the same network). So the client is going to write to this file and server too.

Ok, but how is a socket related to a Stream? Well, we are going to be connected to a server using a socket, therefore we are going to have a 'shared file' between the client and the server. This shared file is a white canvas where we are going to start writting our XML stanzas. The first thing we are going to write to this file is an opening <stream> tag! And there you go... that's our stream.

Perfect, I understand what a stream is, but I still don't understand how to send a message to the server. Well, the only thing we need to do to send a message is writting a <message/> stanza in our shared file. But what happens when the server wants to send me a message? Simple: it will write the message in the 'shared file'.

Are we ok so far?

 

I'm sure at this point you have questions like:

  • "What?! An active TCP connection open all the time? I'm used to REST! How am I going to do that?!" 

​           Easy, you don't have to care about that any more! That's why we are going to use the library, and it will take care of that.

  • "You said nothing about how to connect to the server!"

           Believe me, you don't have to care about this either. If we start adding all this info, we are going to get crazy. Trust me, I've been there.

  • "What about encrypted messages? We need security! How are we going to handle this?"

          Again, you don't have to care about this at this point. Baby steps!

 

You just need to be able to answer: "What's XMPP?", "How do you send a message?", "How do you change your status in XMPP?", "How do you ask something to the server?", "What's a Stream?". If you can answer all that, you are WAY better than me when I started.

All the concepts we described so far are the core of XMPP. In part 2 you will find out how to get started with XMPPFramework, connecting to the server and authenticating a user!

December 02, 2016 10:00