Planet Jabber

The new version 3.0 of OneTeam for iPhone has been released, with the push notification feature from Apple Push Notification Service (APNS).

When moving with your iPhone, as a pedestrian, bike rider, car driver or in public transportation, you might have to shutdown the OneTeam for iPhone application while staying connected to your XMPP server. It is convenient for battery saving purposes, or because you want to use another application (iPhone OS 3.0 still does not authorize background applications).

When an event occurs on your XMPP server, it is sent to the APNS (Apple Push Notification Service), which instantly relays it to your iPhone. A push notification is then poped up on your screen, with a sound played. The OneTeam badge (icon) is then changed, showing a new event has arrived, or the number of events waiting for you to read.

You simply have to tap the OneTeam icon to read your new event. The reconnection will be fast and seamless.

This feature is very handy, since you can use your iPhone the way you want (browse the web, use another application) while still being instantly notified as if you were using OneTeam on the background.

This push notification feature is also deployed on OneTeam.im XMPP server, and ejabberd has a component, IMpush, that can send push notifications to the APNS. This makes ejabberd and OneTeam the first XMPP client and server solution that supports the Apple push system.

OneTeam for iPhone has also received fixes and improvements: groupchat, file transfer, image sending... Here are a few screenshots:

An example of Groupchat:

See the Push configuration screen:

And the general OneTeam confguration:

A few comments on the experience after having used OneTeam for iPhone with push and groupchat:

• "Battery life is not impacted by being always connected to my XMPP server on mobile. I have used all mobile XMPP client and this is the first time I can truely stay connected all the time. I feel this is really really the premise of the mobile revolution for mobile instant messaging and mobile XMPP."
• "OneTeam for iPhone client has the type of feature you would expect from a good desktop XMPP client and not a mobile one. You can use groupchat, file transfer. This is really what I need professionally."

OneTeam for iPhone will be available on the Apple's App Store in a few days, after the usual moderation period at Apple.

IMpush is sold as soon as today on the IMstore to prepare ejabberd servers ahead of OneTeam 3.0 with push.

The first presentation by me (Mickaël Rémond) was titled "OneTeam Media Server: Adding Video to Instant Messaging with Erlang".

The second presentation by Geoff Cant was title "Whitelabel Erlang" and give feedback on how Erlang can be used to write scalable hosted application for business.

Psi 0.13-rc2 is here!  Notably, voice calling is now a standard feature of the Windows version.  Please download and test.

Final on July 13th if no showstoppers are found.

New in 0.13
- Voice calls (Jingle RTP).
- Basic XMPP URI handler.
- Ability to permanantly trust certificates at connect time.
- Mini command system (Ctrl+7 in chat window).
- Various bugfixes.

If you are building from source (e.g. on Linux) and want voice calls, you will need to obtain the PsiMedia plugin separately.  On Linux, the plugin file is called libgstprovider.so, and must be put in Psi’s $LIBDIR/psi/plugins directory.  You’ll know it worked if “About GStreamer” appears in the Help menu.

It looks like we have a season for load tests. High number of people contact me asking for some help and advice on the load tests running.

Most of cases are about comparing different server implementations and I am not going to talk about comparing. Comparing different server implementation is very difficult. So many factors and elements must be taken into consideration for a fair comparison. Even generating exactly the same database for each server is quite difficult task.

I would like to give a few hints about running load tests over the Tigase server. What needs to be considered, what settings are important, how to tweak the system to get best possible performance. How to find system bottlenecks and how to deal with them....

All these suggestion are put in a random order and I am writing them from the top of my head. Please don't hesitate to send me your comments about it. I will try to update this guide in time.

read more

Mickaël Rémond, CEO of ProcessOne, was covered by a Datamonitor study: "CEO Viewpoint: Social Networks Mobility = Market Opportunity".

In this Analyst Opinion series by Daniel Okubo, it is observed that social networks and mobile network operators are currently deploying IM and presence features based on XMPP.

In a competitive and recessive context, the leaders and newcomers look to differentiate in order to increase their market shares. Real time communication based on open standard like XMPP has proven a good way to value existing users base.

This document is available at Datamonitor ResearchStore, at the price of $150, or $375 for a multi-user license. ProcessOne is not affiliated with DataMonitor and does not earn any commission on sales of this document, but we thought it could be usefull to know for customers interested in ProcessOne.

Due to Waqas's asking for months now I've finally got around and made some benchmarks on two string prepping libraries. The one being libidn, as far as I know most XMPP servers use this and the other being ICU by IBM.

While libidn is LGPL licensed with ICU you have nearly all freedom with the choice of license for your project, due to MIT license.

Current XMPP standard says that every JID must be string prepped so if you don't want to cache because of memory reasons your string prepping routines are called quite often. It turns out ICU is much faster than libidn. For this benchmark I tested a couple of strings and let them go through nameprep, resourceprep and nodeprep profiles via string prepping.

The result is that ICU is about 60 times faster in string prepping. Version of libidn being 1.15 and of ICU being 4.2.0.1.

The source code package of the benchmark is attached to this node. It falls under GPLv3 license.

Enjoy!


#!/usr/bin/perl -w
#
# check_bosh plugin for nagios
#
# usage:
# check_bosh -U url -H host
#
# Check if bosh service running at specified address
#
# initial version: 3 May 2000 by Truongchinh Nguyen and Karl DeBisschop
#
# current status: $Revision: 1196 $
#
# Copyright Notice: GPL
# $Id: check_bosh.pl 1196 2009-06-17 15:20:46Z s.strigler $
#

use strict;
use lib "/usr/lib/nagios/plugins";
use utils qw($TIMEOUT %ERRORS &print_revision &support);
use vars qw($PROGNAME);
my ($verbose,$host,$response,$state);
my ($opt_V,$opt_H,$opt_h,$opt_U);
$opt_V = $opt_h, $opt_h = $opt_U = '';
$state = 'UNKNOWN';

$PROGNAME = "check_bosh";
sub print_help ();
sub print_usage ();

$ENV{'BASH_ENV'}='';
$ENV{'ENV'}='';
$ENV{'PATH'}='';
$ENV{'LC_ALL'}='C';

use Getopt::Long;
Getopt::Long::Configure('bundling');
GetOptions(
"V" => \$opt_V, "version" => \$opt_V,
"h" => \$opt_h, "help" => \$opt_h,
"U=s" => \$opt_U, "url=s" => \$opt_U,
"H=s" => \$opt_H, "host=s" => \$opt_H,
"v+" => \$verbose, "verbose+" => \$verbose
);

# -h means display verbose help screen
if ($opt_h) { print_help(); exit $ERRORS{'OK'}; }

# -V means display version number
if ($opt_V) {
print_revision($PROGNAME,'$Revision: 1196 $ ');
exit $ERRORS{'OK'};
}

# -U means URL
unless ($opt_U) { print_usage(); exit $ERRORS{'UNKNOWN'}; }

# -H means host name
unless ($opt_H) { print_usage(); exit $ERRORS{'UNKNOWN'}; }

if (! utils::is_hostname($opt_H)){
print "$opt_H is not a valid host name\n";
print_usage();
exit $ERRORS{"UNKNOWN"};
}

# Just in case of problems, let's not hang Nagios
$SIG{'ALRM'} = sub {
print ("ERROR: No response from RPC server (alarm)\n");
exit $ERRORS{"UNKNOWN"};
};
alarm($TIMEOUT);

$state = &check_bosh($opt_U, $opt_H);

if ($state eq 'OK') {
print "$state: BOSH service at $opt_U up and running\n";
}elsif ($state eq 'WARNING') {
print "$state: BOSH service not working correctly. Bad hostname given?\n";
}else{
print "$state: BOSH service at $opt_U is not running\n";
}
exit $ERRORS{$state};

sub check_bosh() {
my $url = shift;
my $host = shift;

use LWP::UserAgent;
my $ua = LWP::UserAgent->new;

my $req = HTTP::Request->new(POST => $url);
$req->content_type('text/xml; charset=utf-8');
my $payload = "“;
print “SEND: “, $payload, “\n” if($verbose);

$req->content($payload);

my $res = $ua->request($req);

if ($res->is_success) {
print “RECV: “, $res->content, “\n” if ($verbose);

if ($res->content =~ /type=’terminate’/) {
return ‘WARNING’;
} elsif ($res->content =~ /sid=’([^']+)’/) {
$req = HTTP::Request->new(POST => $url);
$req->content_type(’text/xml; charset=utf-8′);

$payload = ““;
print “SEND: “, $payload, “\n” if($verbose);
$req->content($payload);

$res = $ua->request($req);

if ($res->is_success) {
print “RECV: “, $res->content, “\n” if ($verbose);
}
else {
print $res->status_line, “\n” if ($verbose);
return ‘CRITICAL’;
}
}
}
else {
print $res->status_line, “\n” if ($verbose);
return ‘CRITICAL’;
}

return ‘OK’;
}

sub print_help() {
print_revision($PROGNAME,’$Revision: 1196 $ ‘);
print “Copyright (c) 2009 Stefan Strigler\n”;
print “adopted from check_rpc by Karl DeBisschop/Truongchinh Nguyen/Subhendu Ghosh\n”;
print “\n”;
print “Check if bosh service is up and running\n”;
print “\n”;
print_usage();
print “\n”;
print ” URL of bosh service\n”;
print ” XMPP host name\n”;
print ” [-v] Verbose \n”;
print ” [-v -v] Verbose - will print supported programs and numbers \n”;
print “\n”;
support();
}

sub print_usage () {
print “Usage: \n”;
print ” $PROGNAME -U url -H host [-v]\n”;
print ” $PROGNAME [-h | --help]\n”;
print ” $PROGNAME [-V | --version]\n”;
}

As part of some changes to the VPS that Dizzy and I have shared for a number of years, I’ve just migrated the XMPP server that I run at stpeter.im from ejabberd 1.x to Prosody. So far I like Prosody quite a bit: it’s a lean little server that’s very easy to install and configure, especially on Debian. It’s even got an ejabberd2prosody migration script, but I was too lazy to run that so I’ve wiped my roster clean (this is a backup account that I use mainly when jabber.org goes down or when I can’t handle the conversational load of the ~2300 people in my jabber.org roster).

Congrats to the Prosody team on a solid piece of software!

The added value feature of clustering changes in jabberd2 is a possibility to multiplicate not only Session Manager component, but also components connected with XEP-0114 component protocol. Namely: Transports to other networks.

Component protocol connections differ from jabberd2 components that the hash used for selecting component instance to route to is computed using a ‘from’ address, not ‘to’ address. This means that the server (and s2s) userbase is distributed amongst the components (evenly).

I am currently running several clustered transports with no changes to the transport code. Pretty neat. :-)

In the last few weeks I have been struggling to separate the MUC-Services we provide on our XMPP-Cluster into smaller sub-services. Let’s take the following scenario:

You have a website that users can play games on, and simply chat. So you will have a lot of MUC-Rooms, since (hopefully) you will have a lot of games. But it does not make any sense for the (say) poker players to see all the rooms of the chess-players in a discovery request to the MUC-Service. You you could filter this on the client side but I guess it’s good practice to send as little uninteresting data as possible. Sou you want to separate the MUC-Rooms like this:

• conference.website.de
• chess.example.com
• poker.example.com

People will log in to the server using a domain-name of @website.de (let’s say this is for historical reasons). You will have to set up your ejabberd something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87

%% serve all the hosts you need chatservices for
{hosts, ["website.de", "poker.example.com", "chess.example.com"]}.
 
%% configure all COMMON modules, you can make changes to these
%% later, so define everything you need here.
{modules, [
        {mod_adhoc,    []},
        {mod_announce, [{access, announce}]}, 
        {mod_caps,     []},
        {mod_configure,[]},
        {mod_disco,    []},
        {mod_offline,  []},
        {mod_muc_log, [
                       {access_log, muc},
                       {outdir, "/tmp/jabberspool/muc_logs"},
                       {dirtype, subdirs},
                       {timezone, universal}
        ]},
        {mod_stats,    []},
        {mod_time,     []},
        {mod_version,  []}
]}.
 
%% define your MAIN host, this will be the host users log into.
%% you will maybe have some more modules here than in your
%% chat-only hosts
{host_config, "website.de", [{
        %% this loads the common modules, DO NOT FORGET THIS!
        {add, modules},
        [
                %% this will let users see the extra-services in service discovery
                {mod_disco,    [
                                {extra_domains, ["poker.example.com",
                                                        "chess.example.com"]}
                ]},
                {mod_ctlextra, []},
                {mod_http_bind,[]},
                {mod_pubsub,   [
                                {host, "pubsub.@HOST@"},
                                {access_createnode, pubsub_createnode},
                                {plugins, ["flat", "pep"]}
                ]},
                {mod_muc,      [
                                %% THIS HAS TO BE A SUBDOMAIN, THINGS WON'T WORK
                                %% IF YOU JUST USE @HOST@ HERE.
                                {host, "conference.@HOST@"},
                                {access, muc},
                                {access_create, muc},
                                {access_persistent, muc},
                                {access_admin, muc_admin}
                ]},
                {mod_roster,   []},
                {mod_shared_roster,[]},
                {mod_vcard,  []}
        ]
}]}.
 
%% now configure the chat-only hosts
{host_config, "poker.example.com", [{
        {add, modules},
        [
                {mod_muc,      [
                                %% THIS HAS TO BE A SUBDOMAIN, THINGS WON'T WORK
                                %% IF YOU JUST USE @HOST@ HERE.
                                {host, "muc.@HOST@"},
                                {access, muc},
                                {access_create, muc},
                                {access_persistent, muc},
                                {access_admin, muc_admin}
                ]}
        ]
}]}.
 
{host_config, "chess.example.com", [{
        {add, modules},
        [
                {mod_muc,      [
                                %% THIS HAS TO BE A SUBDOMAIN, THINGS WON'T WORK
                                %% IF YOU JUST USE @HOST@ HERE.
                                {host, "muc.@HOST@"},
                                {access, muc},
                                {access_create, muc},
                                {access_persistent, muc},
                                {access_admin, muc_admin}
                ]}
        ]
}]}.

This way everything should work as expected and you should have three MUC-Services running on the same machine. Here is what took me so long to realise how it works:

• The name of the MUC-Service HAS TO BE different from the name of the host serving it. Otherwise you will run into this.
• You will need to have the {add, modules} line in every host-config, otherwise the “global” options won’t count
• Having mod_disco in the common modules is a very good idea. Reconfigure it in your main host to show the other muc-services. Define the HOSTNAME here, not the SERVICENAME.
• Same goes for the {hosts} directive, HOSTNAME not SERVICENAME

The only drawback I see is that you will have a “subdomain” for each host, and then another subdomain of that for the muc service. But if you name those cleverly it may be not so bad.

Good luck!

The largest problem for large jabberd2 deployments is a single-process, single-threaded Session Manager.
Being single-threaded means that it can process only one XMPP packet at a time. More importantly, when it waits for some data (from the storage mostly) it does not process any other packets.

One possible solution would be to rewrite the SM to be multi-threaded. And I do mean rewrite – the way it is designed now, would mean rewrite (or reassembly) from scratch.

Other possible solution is to make the SM multi-processed. This means a possibility to launch many Session Manager instances to service one XMPP domain. This would balance processing load between many processes (possibly machines) and spread storage load on many separate DB connections.

Most server implementation approach this with a separate ‘clustering’ component, which tracks which session component is handling which user.
In case of jabberd2 I think it is not necessary. We already have the all decision making overseer – the router component.

I just committed to SVN trunk a router version, that handles many SM and/or external components (i.e. transports) instances for one XMPP domain. Router select the destination instance using a hash of user JID (receiver in case of SM instances, sender in case of external components). This is an approach that most clustering HTTP deployments use, which select the web server using session-id hash.

I’m running clustered jabberd2 on Chrome.pl with no problems.

[Reporters: Nicolas Vérité and Peter Saint-Andre]

For those who love round numbers, welcome to the tenth edition of the XMPP Roundup, our semi-regular review of news and events in the XMPP community. Since our last roundup on May 11, here’s what’s been happening…

Software

Google Wave

Google has announced the existence of Google Wave, an early-stage technology for real-time interaction that will incorporate aspects of IM, email, wikis, and other collaboration techniques. The technology for federating different Wave instances is pure XMPP, so the developer community is watching this project quite closely.

Minbif

Minbif is an IRC gateway to IM networks which lets the user connect to Jabber (and many proprietary legacy IM systems), through IRC command lines. It is release under the GPLv2 license, in version 1.0alpha as the time of this writing.

Utterance

Nolan Darilek has announced the development of Utterance, an XMPP microblog transport, written in Ruby under a BSD license.

modular_muc

Eric Cestari wrote a Multi-User Chat (MUC) module in ejabberd that is as modular as the PubSub module.

exmpp

ProcessOne has launched its exmpp Erlang XMPP library under the Erlang Public License (EPL), in its newly opened Labs.

Hemlock

Hemlock is an opensource soon-to-be-released Flash XMPP framework.

Services

Ask me

http://askme.im/ has a set of XMPP bots that cover quotes, jokes, movie reviews, stock prices and daliy horoscopes, and much more…

Jabbim’s URL shorting service

Jabbim community has released a new URL shortening service at xmpp:shorty@jabbim.com: just add it to your roster and start chatting.

Community

Members of the XMPP community have started work on two new initiatives: an open framework for XMPP protocol testing and a website for information about the XMPP network. To participate, join the interop@xmpp.org discussion list or the operators@xmpp.org discussion list, respectively. Expect to see more work on these projects in the coming months!

Protocols

The big news here is that the core Jingle specifications have finally advanced from Experimental to Draft status (version 1.0)! We’ll post more about this in the near future. The specs that have moved forward are:

• XEP-0166: Jingle
• XEP-0167: Jingle RTP Sessions
• XEP-0176: Jingle ICE-UDP Transport Method
• XEP-0177: Jingle Raw UDP Transport Method

Another important specification to advance from Experimental to Draft is Roster Versioning. This technology will make it much more efficient for IM clients to log into XMPP servers, since the server can inform the client if the user’s contact list has not changed since the client last logged in (or send a small diff if it has changed). This is especially important in mobile applications such as cell phones and PDAs. The roster versioning technology is one of the mobile optimizations we are working on, and will be incorporated into the revisions to RFC 3921 within the IETF’s newly re-formed XMPP Working Group.

In addition, the XMPP Council has been working to advance more XEPs from Draft to Final. The latest specification to be so honored are:

• XEP-0138: Stream Compression
• XEP-0199: XMPP Ping

Finally, the Council has also been performing some “spring cleaning” by officially obsoleting XEP-0003: Proxy Accept Socket Service, XEP-0011: Jabber Browsing, XEP-0022: Message Events, XEP-0023: Message Expiration, XEP-0025: Jabber HTTP Polling, XEP-0090: Legacy Entity Time, and XEP-0091: Legacy Delayed Delivery.

Conclusion

Again, 2009 seems to be a very active year for XMPP technologies.

One of jabberd2 annoyances was its requirement to run one SM instance for every serviced domain. I already did some work for dynamic jabberd2 configuration introducing default host option in c2s.xml to remove requirement of restarting C2S every time one adds new serviced domain. But it still required launching separate SM process instance.

Today I committed virtual domains support changes to SVN branch http://codex.xiaoka.com/svn/jabberd2/branches/vhosts which allows you to configure more than one serviced domain in only one SM instance.
I successfully configured 3 domains in one SM instance and was able to talk between these 3 and one another in second SM process. All 4 connected to one Router process instance.

Unexpectedly it was pretty simple – it took me only ~6h. to get it working. :-)

Of course all domains serviced in one SM process have identical configuration. If you need domains with different configuration, you need to create another sm.xml and launch separate SM process for them.

The SVN vhosts branch changes need some polishing and testing. I would appreciate if you tried these and tell me what you think.

Zlib compression can reach very high compression rates with XMPP (up to 90%), however starting a session with a mobile client can be still expensive. The best solution would be less greediness of carriers, but I'm not dealing with science fiction, so I prefer to reduce data ;) These are the rough figures about a login with my roster (~250 contacts, with ~60-70 online) and with compression enabled using Lampiro:

• ~ 1.5KB for the authentication and  stream setup
• ~ 12-13KB for the presence broadcast and other related stanzas 
• ~ 4KB for roster retrieval

Can we do better? There is a new proposal, Stanza Interception and Filtering Technology (SIFT ), which may help a little by filtering incoming stanzas and allowing session "hush". Which is the importance of hush, besides not receiving unecessary presence updates? As Dave Cridland pointed out in a private chat it can improve Zlib compression by stuffing more data during hush and delaying flushes. The question was: how relevant is this action? So I did few experiments with a log of a login of mines and these are the results:

• A complete login session:
  
• Initial data size: 136175
• Compression of the whole block 16036
• Compression flushing each packet 22739
• Only presence:
• Initial data size:55729
• Compression of the whole block 4881
• Compression flushing each packet 7323

 Almost 1/3 of bandwidth gained, SIFT is definitely good!

Our labs are producing wonderfull pieces of code and we want to make sure that code is more largely accessible. We develop ejabberd, Tsung and various Erlang projects in general and those projects stay at the core of our business.

However, we want to make sure that the benefit of some of our pieces of code are open to anyone. We of course commercially support the project that goes out from our labs, so you can contact us if you are interested by commercial support.

ProcessOne Labs can be reach on http://www.process-one.net/en/labs/.

The first project released is a robust, fast and scalable Erlang XMPP library called exmpp.

Enjoy !

You can find more details on ProcessOne Labs or on our exmpp Wiki page.

This is beta release and your feedback is welcome before we reach version 1.0.

OneTeam for iPhone 2.1.0 will add the following features after Apple approval:

• File transfer support. Client can receive and display PDF, MS Word documents and Excel spreadsheets, Images. User can take pictures and send them to contact with file transfer.
• Support for HTTP links and ability to open them in OneTeam web view.
• Roster caching: A OneTeam entry as been added in the preference panel to decide contact list update policy. When not updated during the session, the user pane can be use to manually force download.
• Access possible to password protected chat rooms.
• Ability to send crash report to the development team and report anonymous usage statistics (OS version, network connection type), to help us prepare to iPhone firmware 3.0.
• Improved user interface.
• Bug fixes.

Personnaly, I love the new file transfer feature and think the development team has done a great job. It is pleasure receiving a PDF file or an image and being able to review it in a usefull way. Sending images is usefull in many context as well: Sending paperboard copy, flat visit picture, etc.

Next version will likely be OneTeam 2.2.0 that will add the long awaited push features.

Here is a few screenshots:

Groupchat session in Jabber development chat room.
Note the ability to access links

File received can be opened from the chat tab and history

Dialog to send an image to your contact

This is pre-final release of the new version for the Tigase XMPP Server. It has been extensively tested under a different load and in different environments. Also it went through all our automated tests system. It seems to be stable. Please feel free to check it out and report all the problems you may find.

For installation instructions please refer to quick start documentation. The pre-final release is published only as JAR file - the installer. The final version will be also released on all other formats: exe, tar.gz and zip.

This is a major release with huge number of changes. Here is a brief list of what's new and what has been changed in quite random order:

1. Virtual hosts support - now virtual hosts are stored in database. Tigase can now effectively run for thousands of virtual domain. Virtual domains can be added/removed at runtime without any disruption to the working system. You can also limit number of users per domain, temporarily block registration or user login for the domain and this is all available through ad-hoc commands.
2. Scripting support - you can load scripts to the Tigase server at runtime also via ad-hoc commands. At the moment scripts are supported for administration tasks only. All JSR-223 languages compatible are supported which includes but is not limited to: Groovy, JRuby, Jython, Scala and possible a few others. 
3. Hugely improved performance on multi-cpu/multi-core systems. You can now control number of threads for almost every component of the system. Again big thank you to Sun Microsystems for offering they hardware for tests.
4. User repository connection pool added.
5. New, much more effective clustering code added.
6. Monitoring and self-monitoring functionality added. You can watch the system running and track the server statistics via SNMP, JMX, HTTP and XMPP. But you don't have to watch them. The server monitors itself and can send you notifications when something is going wrong. The monitoring framework is highly extensible and you can easily plug your own monitors too. At the moment the Tigase looks at Memory, CPU, Disk usage as well as it watches logs.
7. New installer has been improved too. It does create administrator accounts for you during installation and often requested feature - the command line mode for the installer is now supported too.
8. Alternatively - Debian/Ubuntu packages are available too. Directly from SVN so you can install and play with the most hot version of the server.
9. PubSub further improved. Now, you can optionally use PubSub with dedicated database schema instead of Tigase built-in UserRepository. The new schema greatly improved performance and was successfully tested with up to 1mln nodes and 1 thousand subscription each.

The full and detailed description of what's new will be compiled for the final release. I am awaiting your comments and please report all the problems you may find.

read more

This is pre-final release of the new version for the Tigase XMPP Server. It has been extensively tested under a different load and in different environments. Also it went through all our automated tests system. It seems to be stable. Please feel free to check it out and report all the problems you may find.

For installation instructions please refer to quick start documentation. The pre-final release is published only as JAR file - the installer. The final version will be also released on all other formats: exe, tar.gz and zip.

This is a major release with huge number of changes. Here is a brief list of what's new and what has been changed in quite random order:

1. Virtual hosts support - now virtual hosts are stored in database. Tigase can now effectively run for thousands of virtual domain. Virtual domains can be added/removed at runtime without any disruption to the working system. You can also limit number of users per domain, temporarily block registration or user login for the domain and this is all available through ad-hoc commands.
2. Scripting support - you can load scripts to the Tigase server at runtime also via ad-hoc commands. At the moment scripts are supported for administration tasks only. All JSR-223 languages compatible are supported which includes but is not limited to: Groovy, JRuby, Jython, Scala and possible a few others. 
3. Hugely improved performance on multi-cpu/multi-core systems. You can now control number of threads for almost every component of the system. Again big thank you to Sun Microsystems for offering they hardware for tests.
4. User repository connection pool added.
5. New, much more effective clustering code added.
6. Monitoring and self-monitoring functionality added. You can watch the system running and track the server statistics via SNMP, JMX, HTTP and XMPP. But you don't have to watch them. The server monitors itself and can send you notifications when something is going wrong. The monitoring framework is highly extensible and you can easily plug your own monitors too. At the moment the Tigase looks at Memory, CPU, Disk usage as well as it watches logs.
7. New installer has been improved too. It does create administrator accounts for you during installation and often requested feature - the command line mode for the installer is now supported too.
8. Alternatively - Debian/Ubuntu packages are available too. Directly from SVN so you can install and play with the most hot version of the server.
9. PubSub further improved. Now, you can optionally use PubSub with dedicated database schema instead of Tigase built-in UserRepository. The new schema greatly improved performance and was successfully tested with up to 1mln nodes and 1 thousand subscription each.

The full and detailed description of what's new will be compiled for the final release. I am awaiting your comments and please report all the problems you may find.

read more

PubSubHubbub is a protocol and reference implementation for doing publish-subscribe using web hooks, polling in feeds triggered by a ping from the publisher, and POSTing Atom entries to notify subscribers. The notification part is similar to what I've been working on for the publish-subscribe stuff at Mediamatic Lab, where we spiced up Idavoll with an HTTP interface to bridge the gap between XMPP Publish-Subscribe and HTTP speaking entities.

Although I spend a lot of time working on XMPP based publish-subscribe, I understand the reasons for going for a full HTTP-based approach. XMPP can be intimidating for developers of web applications. While the differences between XMPP and HTTP are important (stateful connections, asynchronous processing, etc), the fact that it is different is reason often enough. Hosting facilities don't always offer ways to do XMPP, and there is not nearly enough running code out there to make it easier for people to play with these technologies to spice up their web application with non-IM XMPP functionality. Having platforms like Google App Engine provide sending and handling raw XMPP stanzas as part of the API would surely help.

That said, PubSubHubbub has two separate sides to it, the publishing part and the notification part. There's nothing that prevents a hub to do the publishing part using regular XMPP publish-subscribe. Instead of fetching the Atom Feed over HTTP every time, it could use autodiscovery to find out the publish-subscribe node and upgrade by subscribing to it instead. Similarly, the notification part could send out XMPP notifications. Combined with existing HTTP aggregator, that combination is very similar to how the aggregator for Mimír works.

I'm still not convinced that PubSubHubbub is the answer to the efficient exchange of updates on social objects, but I do think it is a good way to make smaller entities be part of a federation of social networking sites. Likely, we'll see a hybrid approach, to begin with.

Last month I was fortunate enough to attend Social Web FooCamp at O'Reilly HQ in Sebastopol, CA, a follow up to Social Graph FooCamp in 2008. I can't express how inspiring such events are, being able to have a continuous, in-depth conversation with so many bright minds about so many topics that keep you busy on regular days, and more. I'll give a quick overview of the whole trip, and then go into depth in a series of posts.

My trip started with a visit to friend and former Jaiku colleague Andy Smith, who was kind enough to take me in at Houseku. As soon as I landed on SFO, I got an SMS from him to make a detour to his office. Besides meeting a bunch of Andy's fellow googlers, I got to spend some time with Brett Slatkin talking about PubSubHubbub.

The next day I got a ride to Sebastopol from Edwin Aoki. After a trip full of interesting conversation, we arrived at the O'Reilly offices. Sebastopol was a lot warmer than San Francisco, perfect for camping. Lots of familiar faces, but also a lot of new ones. During the Friday evening, apart from the general introduction, I didn't get to any sessions, but instead spent talking to a bunch of people on XMPP, Publish-Subscribe and the work I am doing on federating social networks under that name Open-CI at Mediamatic Lab.

The next two days were filled with sessions and hallway talk on OpenID, OAuth, different approaches to Publish-Subscribe and inter-site communication, resource and service discovery and service scalability. While most of the topics were similar to last year, I was glad to share what we've done at Mediamatic Lab over the past year, while learning how others have fared. We used these technologies to make a true federation of social networking sites where you can make cross-site relations between people and their social objects. Some of our discoveries there we're shared among the participants, while others had interesting other approaches.

Especially interesting to me was a session on OAuth and OpenID where I could explain how we tried to improve upon the user experience. Both technologies have a bad reputation in this area. With some smart defaults and trust between sites, we could eliminate some of the screens. There was talk about using pop-ups in some situations, either as lightboxes or as new (small) windows. In our experience the former can't be used if you want to do SSL (since you can't validate the address and certificate). The latter was deemed confusing in our user tests. Research is still ongoing, I suppose. The other issue had to do with presenting OpenID providers. We currently use a drop down, but that doesn't scale up very nicely. Logos might work, but in the end has the same issue.

I also got to show Blaine Cook the code I wrote recently to make it easier to write XMPP publish-subscribe enabled services (code-as-a-node), that has been included in the recent Wokkel release. In turn, Blaine shared his thoughts on simple addressing on the web and we got to hash it out with a bunch of people like Brad Fitzpatrick, who also organized the pubsub shootout session. Finally, Eran Hammer-Lahav showed his work on XRD.

I'm pretty sure I forgot to mention a lot of things, but when it comes back to me, I'll write about it some other time.

Adam Goucher and Tim Riley (Director of QA at Mozilla) announced a few months ago that they are putting together a Beautiful Testing book for O’Reilly. I took the opportunity to write a chapter about testing in the context of XMPP (more specifically, about testing protocol implementations in Swift), and just submitted the final draft for technical review. The book is expected to be released this August.

Although there are many types of testing being done in the XMPP world, the chapter focuses on the beauty of testing the functionality of XMPP protocol implementations. After a brief introduction on XMPP, it starts out with a description of unit testing simple IQ request/response protocols, and then gradually moves on to higher-level testing of more complex, multi-stage protocols such as session initialization. As you might expect from a developer like me, the chapter is quite heavy on the (C++) code, but I’m told it compensates for the rest of the book

As with all other books in the O’Reilly “Beautiful” series (which started with Beautiful Code, but has since been followed up by Beautiful Architecture, Beautiful Teams, Beautiful Security, and Beautiful Data), all proceeds of the book go to charity, in this case “Nothing But Nets” (which provides mosquito netting to malaria infested areas of Africa). This means that I can plug this book as much as I want, and still have the feeling I’m actually doing a noble, unselfish thing. (contrary to when I casually mention that you can buy our book XMPP: The Definitive Guide at very sharp prices these days). Some time after the book’s release this summer, I will even make a free version of the chapter available here, so check back soon!

As many observers (Like Jeremiah Owyang from Forrester) have pointed out, the realtime collaborative platform itself is not impressive: This is feature you can find in many competitors offer.

What is interesting however is that Google tries to leverage a development community platform by building the Wave platform on a distributed federated model, based on the open messaging standard XMPP (eXtensible Messaging and Presence Protocol). It means that you can use other XMPP / Wave enabled server to provide content for Wave platform consumers. With this move, the actual platform is really not that important. What matter is the API. The real question is whether Google will manage to attract enough developers to support this new protocol. I feel this is another step in the war with Facebook (and its Facebook API), more than an attack toward traditional collaborative platform providers like IBM and Microsoft. However, to appeal to a broader audience, Google market its initiative as "email as it should be". The way of demonstrating the platform hide the essential point: Google's aim is to build a development platform and applications are just their way to promote this initiative. Google Wave is really an initiative that complement Google App Engine and Open Social.

XMPP is already at the heart of several cloud computing platform and is becoming increasingly popular as a glueing tool for various types of services around the world. It is gaining mindshare ans supporters as a near real time protocol to power the next generation of web services and applications.

As Peter Saint Andre explained on his blog, this move demonstrates once more that

XMPP is fast. XMPP is secure. XMPP is extensible.

Will Google succeed ? The fight will be hard to gather a big enough development community. Only time will tell if they can provide the missing piece that everyone was waiting for. XMPP and the extension produced by the XMPP Standard Foundation already cover a large piece of what Google is currently promoting. However, what is sure is that this move strengthen XMPP. XMPP is set to play a major role in the Web infrastructure of tomorrow. Be sure that ProcessOne is preparing as well to be a major player in this upcoming shift of paradigm.

At the Google I/O developer conference earlier today, Google made a big splash by announcing Wave, a radical rethinking of communication over the Internet. As Tim O’Reilly noted, the Wave vision is to combine the best of different conversational media and collaboration tools, including email, IM, phone, microblogging, and file sharing.

Little noted among hoopla is the fact that “The Google Wave Federation Protocol is an open extension to XMPP core [RFC3920] protocol to allow near real-time communication between two wave servers.”

Yes, underneath all these hot new collaborative technologies we find none other than XMPP, the same XML streaming protocol that has been powering the Jabber instant messaging network since 1999. Those in the know have long realized that XMPP can be used for much more than basic IM. The power of XMPP comes from the fact that in essence it is really just a way to quickly and securely deliver small pieces of structured information from one entity to another over the Internet.

Let’s parse that out. XMPP is fast. XMPP is secure. XMPP is extensible.

That kind of extensibility enables folks like Google to define their own XMPP extensions, but it also means that the XMPP developer community has had 10 years to define most of the core services you might need. As Kevin Smith, Remko Tronçon, and I explore in XMPP: The Definitive Guide (just out from O’Reilly), XMPP includes encryption, strong authentication, interdomain federation, fully internationalized messaging, ubiquitous presence, contact lists, service discovery, multi-user chat, publish-subscribe, lightweight forms and workflows, multimedia negotiation, serverless messaging for ad-hoc networks, and much more. If all that isn’t enough, you can define your own custom features or work with the XMPP Standards Foundation on new protocol extensions.

Despite 10 years of hard work, in many ways we’re still just starting to understand what we can build with XMPP. Google itself paved the way for some of the recent innovations by launching its XMPP-based Google Talk service in the summer of 2005, including voice and video features (XMPP is used here as a control channel, not for exchanging the media traffic). Several years ago we started to see XMPP-based gaming services like Chesspark, the brains behind WuChess. Even more recently the folks at drop.io have built a real-time collaboration platform entirely on XMPP. All the while, lots of companies and open-source projects have been using XMPP to make their most cutting-edge products and services. And now Google Wave has upped the ante even further. (I’ll have some further comments about Wave in future posts.)

This is a great time to get involved with XMPP. There are dozens of open-source projects that will help you get started, including a wide variety of servers, clients for every operating system and device, and libraries for just about every programming language on the planet. Our new book provides a friendly introduction that shows you how to “think in XMPP”. The XMPP developer community is active on multiple fronts, from application development to interoperability testing to frequent meetups around the world (and don’t miss the XMPP Summit at OSCON this summer). So catch the wave and join the conversation!