Planet Jabber

September 15, 2017

ProcessOne

Let’s encrypt ejabberd

Back in May we announced that 2 ejabberd projects will participate in this year’s Google Summer of Code (GSoC) through the BEAM Community. The summer has ended and now it’s time to see the results!

Today we will look at the first ejabberd project, aimed at implementing ejabberd support for “Let’s Encrypt” ACME protocol. It is developed by Konstantinos Kallas, with ProcessOne’s Evgeny Khramtsov as the mentor. In the days when encryption should be widespread, it certainly would be convenient to be able to create certificates for ejabberd quickly, easily and for free.

The outcome of this project was quite successful. The pull request in question is available on GitHub. “We will merge it for sure (modified or not), because we’re interested in having this feature” Evgeny says. For Konstantinos, it was his first GSoC. “I would sure like to participate again next summer” he says. “The ability to discuss with an experienced developer and ask them questions about issues is the most interesting and beneficial part of it. It is also very pleasing to see that your work is integrated in a real life open source project.”

Konstantinos was also keen on continuing his participation in open source projects and being active in the community. This sentiment is exactly what GSoC is all about.

“The whole GSoC experience was fascinating. I never had the chance to work on such a big and interesting project collaborating with experienced people. The development went pretty smoothly after the initial period that I had to absorb a lot of new information. My mentor, Evgeny Khramtsov, helped me a lot especially with all the technical issues that appeared throughout the process.”

When I asked Evgeny, he was happy with the experience as well, saying “This is my first GSoC project. The student asked the questions, I answered them :)”

Here at ProcessOne we believe open source software and open development communities benefit everyone. We will continue to support BEAM Community and Google Summer of Code, and keep ejabberd community strong.

by Marek Foss at September 15, 2017 07:33

September 11, 2017

ProcessOne

Real-time Enterprise Issue #1

Here are the articles concerning business aspects of real-time enterprise we found interesting in Issue #1. To receive this newsletter straight in your inbox on the day it’s published, subscribe here.

Welcome to the era of conversation!

The Social Client offers a white paper on Messaging aimed to provide the keys to a connected customer experience. More than 50% of French consumers are now mobile first. Every day, there are 26 million people connecting to the internet via their smartphone.

The good, the bad, and the ugly of chatbots

It’s amazing how rapidly the world of technology can turn commerce and customer service on its ear. Transformational tech is completely revolutionizing the online sales experience.

Four ways startups can harness innovation and disruption

We live in a time when doctors no longer have to rely on costly and unwieldy medical imaging devices to diagnose illnesses. A simple “visual stethoscope” would help them see deep into the human body more easily than ever, thereby accelerating both diagnosis and therapy.

So you want to go digital? How to avoid the next legacy IT platform debacle

Most companies realise their IT systems are overly complex, having been added to multiple times over the years without much effective pruning. The most common response is to simplify systems using new technologies like the public cloud and SaaS platforms, which help to gain speed and lower costs. But those are temporary fixes. Complexity inexorably creeps back over time, due to the fundamental nature of people and organisations.

Instant messaging apps invade the workplace

With the internet, we have more ways to communicate than ever. Email, instant messaging, and smartphones completely changed how we interact with family and friends. Increasingly, they’re also changing how we connect with colleagues at work.

Instant messaging in on-site and online classes in higher education

In the past, instant messaging (IM) was considered “a teen thing” rather than a serious tool for education. As teenagers who rely on IM as a communication tool arrive on college campuses, however, IM usage will become more prevalent in higher education.

The business case for customer service chatbots

By embracing messaging channels, companies will immediately benefit from an “eternal conversation thread” with their customers.

by Marek Foss at September 11, 2017 11:20

Real-time Stack Issue #1

At the end of June, ProcessOne introduced a new series of its newsletters: Real-time Stack, focused on all real-time technologies and Real-time Enterprise focused on business aspects of using real-time infrastructure. You can subscribe to these newsletters here. Here are the technology articles we found interesting in Issue #1:

MQTT: The Nerve System of IoT

There are billions of smart devices in our world today, but what if these devices were interconnected? What if these devices can interact with each other just like how their owners do and form a kind of global nervous system? This essentially describes what people call the Internet of Things or IoT.

Practical IoT Cryptography on the Espressif ESP8266

We’re often critical of the lack of security in the IoT sector, and frequently cover botnets and other attacks, but will we hold our projects to the same standards we demand? Will we stop at identifying the problem, or can we be part of the solution?

Kivy and XMPP

Kivy is an open source Python library for rapid development of applications that make use of innovative user interfaces, such as multi-touch apps. In this presentation, we learn how to create an example XMPP client using Kivy environment.

Device-friendly XMPP Client

Kaidan is a simple, user-friendly Jabber/XMPP client providing a modern user-interface using Kirigami and QtQuick. The back-end of Kaidan is completely written in C++ using the Swiften Library of the Swift Instant Messenger and Qt5.

Naturalizing IOT Through Standardization

Internet of Things is one of fast growing technology in the recent years. It’s expected that the growth of connected things will be around 50 billion by 2020.

Delivering Billions of Messages Exactly Once

Read about a new de-duplication system to get as close as possible to exactly-once delivery, in the face of a wide variety of failure modes. This system is able to track 100x the number of messages of the old system, with increased reliability, at a fraction of the cost.

Want to know more about ProcessOne Real-time Newsletters? Read the announcement.

by Marek Foss at September 11, 2017 11:17

September 06, 2017

Fanout Blog

Realtime processing and edge computing: the end of the cloud?

At Fanout we’re always interested in trends involving moving and processing data in realtime. A major shift is coming, driven by the rise of connected devices and the vast amount of data they are going to collect. According to a Gartner report, 8.4 billion connected “things” will be in use in 2017, representing a 31% increase from 2016 – and every one of these IoT devices is going to need to collect, process, and transmit data in order to be effective.

...

by daniel at September 06, 2017 17:30

August 29, 2017

Tigase Blog

Tigase XMPP Server v7.1.1 Released!

Tigase XMPP Server v7.1.1

has been released! Please review the change notes below to see what has changed since our last release.

by Daniel at August 29, 2017 22:24

Tarun Gupta (GSoC 2017)

Wrap-Up

Hello all,

This was indeed an exciting summer. During this summer, I initially implemented the elements along with their parsers and serializers, required to implement the MIX features. Then I implemented a MIX Registry to keep track of channels joined by a client, and sync the same information with their rosters. This Registry is responsible for returning an object of the joined channel, which can be used for various features like setting nick for the channel, sending messages, etc. Finally. I implemented basic functionalities for MIX in Swiften, along with a MIX Example client and MIX Mock Server to respond to clients. Let me explain the features I have added as part of GSoC'17 along with examples:

1.) Client querying MIX service for hosted channels: This allows my client to query server for hosted channels.

2.) Client querying for publish-subscribe nodes supported by MIX channel: This allows my client to query server for supported standard nodes for a MIX channel.

3. Client joining a channel hosted on MIX domain: Client then published its already joined channels and sends a request to channel service to join.
./Swiften/Examples/MIXListAndJoin/MIXListAndJoin another@example.com/some mixtest2 example.com
Connecting...Connected
Request list of channels.
List of rooms at example.com
1. coven - coven@example.com

Request supported nodes for MIX channel: coven@example.com
Nodes supported by channel coven@example.com
1. urn:xmpp:mix:nodes:participants
2. urn:xmpp:mix:nodes:messages
3. urn:xmpp:mix:nodes:presence
4. urn:xmpp:mix:nodes:jidmap

Already Joined channels:
No channels already joined.

Successfully joined channel coven@example.com
[warning] Swiften/MIX/MIXRegistry.cpp:40 joinChannel: Channel already joined: coven@example.com
In the example above, user with JID another@example.com/some initially connects to server and request list of channels, followed by supported nodes for MIX channel.
  • urn:xmpp:mix:nodes:participants: holds information about all participants of the channel along with their nick.
  • urn:xmpp:mix:nodes:messages: holds all messages sent to the channel.
  • urn:xmpp:mix:nodes:presence: holds presence status for all clients of the channel.
  • urn:xmpp:mix:nodes:jidmap: holds mapping information from client's real JID to proxy JID.
The warning in above example indicates a use case where user tries to join an already joined channel.

4.) Join Sync: This is an interesting and much required feature, which allows different clients of the same user to sync already joined channels and other information.
./Swiften/Examples/MIXListAndJoin/MIXListAndJoin another@example.com/someelse mixtest2 example.com
Connecting...Connected
Request list of channels.
List of rooms at example.com
1. coven - coven@example.com

Request supported nodes for MIX channel: coven@example.com
Nodes supported by channel coven@example.com
1. urn:xmpp:mix:nodes:participants
2. urn:xmpp:mix:nodes:messages
3. urn:xmpp:mix:nodes:presence
4. urn:xmpp:mix:nodes:jidmap

Already Joined channels:
1. coven - coven@example.com

[warning] Swiften/MIX/MIXRegistry.cpp:40 joinChannel: Channel already joined: coven@example.com
In the example above, same user another@example.com tries to connect to server with JID another@example.com/someelse initially connects to server and client successfully indicates already joined channels and yet another warning when user tries to join an already joined channel.

5.) Retrieve Participants of Joined channel: This allows a client to retrieve the list of participants of the joined channel. This however, only returns proxy JIDs of the participants along with their nicks, if set by user.

6.) Real JID lookup of participants: This allows a client to lookup participants proxy JID in the channel and retrieve its real JID. Right now, the present client only offers join as a simple join with subscriptions, however for join with preferences, if user have jid-hidden preference for its JID Visibility, then this will still return participants proxy JID.

7.) Sending/Receiving Messages: The client can now send messages to the channel, which will be forwarded to all online clients and a replicated message will be sent to sender with same submission ID as the ID of message sent.

Client I
./Swiften/Examples/MIXListAndJoin/MIXListAndJoin another@example.com/some mixtest2 example.com
Connecting...Connected
Request list of channels.
List of rooms at example.com
1. coven - coven@example.com

Request supported nodes for MIX channel: coven@example.com
Nodes supported by channel coven@example.com
1. urn:xmpp:mix:nodes:participants
2. urn:xmpp:mix:nodes:messages
3. urn:xmpp:mix:nodes:presence
4. urn:xmpp:mix:nodes:jidmap

Already Joined channels:
No channels already joined.

Successfully joined channel coven@example.com
[warning] Swiften/MIX/MIXRegistry.cpp:40 joinChannel: Channel already joined: coven@example.com

Participants of channel coven@example.com
1. 99ba30b1-0d0a-4739-95ad-fc1eb3c12a49#coven@example.com

Lookup of participant in channel coven@example.com
1. 99ba30b1-0d0a-4739-95ad-fc1eb3c12a49#coven@example.com - another@example.com

[ coven@example.com ] : Hello, I am here! some
Client II
./Swiften/Examples/MIXListAndJoin/MIXListAndJoin some@example.com/some mixtest example.com
Connecting...Connected
Request list of channels.
List of rooms at example.com
1. coven - coven@example.com

Request supported nodes for MIX channel: coven@example.com
Nodes supported by channel coven@example.com
1. urn:xmpp:mix:nodes:participants
2. urn:xmpp:mix:nodes:messages
3. urn:xmpp:mix:nodes:presence
4. urn:xmpp:mix:nodes:jidmap

Already Joined channels:
No channels already joined.

Successfully joined channel coven@example.com
[warning] Swiften/MIX/MIXRegistry.cpp:40 joinChannel: Channel already joined: coven@example.com

Participants of channel coven@example.com
1. c0a202ff-adce-4a07-8d90-577b4e986321#coven@example.com
2. 99ba30b1-0d0a-4739-95ad-fc1eb3c12a49#coven@example.com

Lookup of participant in channel coven@example.com
1. 99ba30b1-0d0a-4739-95ad-fc1eb3c12a49#coven@example.com - another@example.com
2. c0a202ff-adce-4a07-8d90-577b4e986321#coven@example.com - some@example.com
Above two examples shows client I joining, retrieving participant list (only himself for now), and lookup of real JIDs of participants. Then client II connects, repeats the same procedure as client I, and sends a message Hello, I am here! some to channel, which is forwarded to client I. Client II simply ignores the replicated message, however it can still be useful to correlate the message with the submitted message.

8.) Setting Nick for joined channel: The client can set its nick for joined channels. If there is a nick conflict, an error payload will be returned with ErrorPayload:Condition:Conflict.

9.) Presence updates: With this client can now receive presence of all participants of the channel, as well as updated its  own presence. Client can go offline, and indicate its unavailability to other clients and similarly come online again. When a client joins a new channel or comes online after being online, channel presence of all participants is pushed to the client. 

Client I
./Swiften/Examples/MIXListAndJoin/MIXListAndJoin some@example.com/someelse mixtest example.com
Connecting...Connected
Request list of channels.
List of rooms at example.com
1. coven - coven@example.com

Request supported nodes for MIX channel: coven@example.com
Nodes supported by channel coven@example.com
1. urn:xmpp:mix:nodes:participants
2. urn:xmpp:mix:nodes:messages
3. urn:xmpp:mix:nodes:presence
4. urn:xmpp:mix:nodes:jidmap

Already Joined channels:
No channels already joined.


Successfully joined channel coven@example.com
[warning] Swiften/MIX/MIXRegistry.cpp:38 joinChannel: Channel already joined: coven@example.com

Nick Assigned: some

Participants of channel coven@example.com
1. 0b8f6eb8-f42e-4ad8-b059-7052224790dc#coven@example.com Nick: [some]

Lookup of participant in channel coven@example.com
1. 0b8f6eb8-f42e-4ad8-b059-7052224790dc#coven@example.com - some@example.com

Client is now online

[ coven@example.com ] : fff754f9-a050-48ad-87d8-3178fab98774#coven@example.com is available
[ coven@example.com ] : Hello, I am here! yetanother
[ coven@example.com ] : 8cb2cef7-37d6-47f4-9af4-42796043bb33#coven@example.com is now unavailable
[ coven@example.com ] : 8cb2cef7-37d6-47f4-9af4-42796043bb33#coven@example.com is available
[ coven@example.com ] : Hello, I am here! another

Client II
./Swiften/Examples/MIXListAndJoin/MIXListAndJoin yetanother@example.com/someelse mixtest example.com
Connecting...Connected
Request list of channels.
List of rooms at example.com
1. coven - coven@example.com

Request supported nodes for MIX channel: coven@example.com
Nodes supported by channel coven@example.com
1. urn:xmpp:mix:nodes:participants
2. urn:xmpp:mix:nodes:messages
3. urn:xmpp:mix:nodes:presence
4. urn:xmpp:mix:nodes:jidmap

Already Joined channels:
No channels already joined.


Successfully joined channel coven@example.com
[warning] Swiften/MIX/MIXRegistry.cpp:38 joinChannel: Channel already joined: coven@example.com

[ coven@example.com ] : 0b8f6eb8-f42e-4ad8-b059-7052224790dc#coven@example.com is available
Nick Assigned: yetanother

Participants of channel coven@example.com
1. fff754f9-a050-48ad-87d8-3178fab98774#coven@example.com Nick: [yetanother]
2. 0b8f6eb8-f42e-4ad8-b059-7052224790dc#coven@example.com Nick: [some]

Lookup of participant in channel coven@example.com
1. 0b8f6eb8-f42e-4ad8-b059-7052224790dc#coven@example.com - some@example.com
2. fff754f9-a050-48ad-87d8-3178fab98774#coven@example.com - yetanother@example.com

Client is now online

[ coven@example.com ] : 8cb2cef7-37d6-47f4-9af4-42796043bb33#coven@example.com is now unavailable
[ coven@example.com ] : 8cb2cef7-37d6-47f4-9af4-42796043bb33#coven@example.com is available
[ coven@example.com ] : Hello, I am here! another
Client III
./Swiften/Examples/MIXListAndJoin/MIXListAndJoin another@example.com/someelse mixtest example.com
Connecting...Connected
Request list of channels.
List of rooms at example.com
1. coven - coven@example.com

Request supported nodes for MIX channel: coven@example.com
Nodes supported by channel coven@example.com
1. urn:xmpp:mix:nodes:participants
2. urn:xmpp:mix:nodes:messages
3. urn:xmpp:mix:nodes:presence
4. urn:xmpp:mix:nodes:jidmap

Already Joined channels:
No channels already joined.


Successfully joined channel coven@example.com
[warning] Swiften/MIX/MIXRegistry.cpp:38 joinChannel: Channel already joined: coven@example.com

[ coven@example.com ] : fff754f9-a050-48ad-87d8-3178fab98774#coven@example.com is available
[ coven@example.com ] : 0b8f6eb8-f42e-4ad8-b059-7052224790dc#coven@example.com is available
Nick Assigned: another

Participants of channel coven@example.com
1. 8cb2cef7-37d6-47f4-9af4-42796043bb33#coven@example.com Nick: [another]
2. fff754f9-a050-48ad-87d8-3178fab98774#coven@example.com Nick: [yetanother]
3. 0b8f6eb8-f42e-4ad8-b059-7052224790dc#coven@example.com Nick: [some]

Lookup of participant in channel coven@example.com
1. 0b8f6eb8-f42e-4ad8-b059-7052224790dc#coven@example.com - some@example.com
2. fff754f9-a050-48ad-87d8-3178fab98774#coven@example.com - yetanother@example.com
3. 8cb2cef7-37d6-47f4-9af4-42796043bb33#coven@example.com - another@example.com

Client is going offline

Client is now online

[ coven@example.com ] : fff754f9-a050-48ad-87d8-3178fab98774#coven@example.com is available
[ coven@example.com ] : 0b8f6eb8-f42e-4ad8-b059-7052224790dc#coven@example.com is available
In above examples, client I joins first, sets its nick and indicates itself to be online. Then it just wait for message/presence updates from other clients. Then client II joins, receive presence status of client I on successful join, and sends a message which is received by client I (only other online client). Then client III joins, sets its nick, receive presence status of client I and client II on successful join, and then goes offline. As it goes offline, client I and II now receive presence updates indicating that client III is now unavailable. Then client III comes online again after 3 seconds and receive full presence updates from channel. Finally, it sends a message to channel, which will be received by client I and II. When clients go offline, neither they will receive any message nor any presence updates.

Challenges/RoadBlocks:

I would also like to include what challenges and progress struggles I faced while working on this project:

  • Firstly, the major challenge I faced was unavailability of a working MIX server. If a working MIX server was available, checking my client implementation could have been a lot easier. This forced me to write a mock server using limber framework, which could mock the responses of an actual server. This wasn't originally planned as per the proposal, but was an immediate requirement.
  • Secondly, having worked on implementing MIX features till last week of July, we had a major design change for our MIX implementation in first week of August. Initially the MIXImpl was handling all channel requests, however as per advice from mentors, I created a MIX Registry class which took care of joining / leaving channels and provide instances of MIXImpl for the joined channel. So in essence, the MIXImpl class will now only handle methods for one joined channel. This is indeed a very efficient design, and also allows syncing of roster across different clients of same  user.
  • This roadblock was rather a personal one, and required me to work on my masters thesis to be submitted in first week of August and its presentation in upcoming weeks. This costed me around 1.5 weeks of my time. However, after chatting with mentor, I worked double during my last week and submitted pull requests for all the use cases. I know that working during last week might not have been sufficient to compensate.
In the end, I was at able to submit my pull requests demonstrating the testing of use cases via mock server and client. I could've achieved more by working full time during the lost weeks and maybe PR's would have been merged by now.

I would like to thank my mentors Edwin, Tobias and Kevin for successfully guiding me through this project. I would like to continue working on this project after GSoC as a lot of new features could be added and also integration of MIX in Swift.

That's all for now :)

by Tarun Gupta (noreply@blogger.com) at August 29, 2017 11:48

August 25, 2017

Paweł Alameyo Ścibiorski (GSoC 2017)

#15 The End of the GSoC

Hey,

 

This week I changed a bit way of using blacklist KeyStore. Earlier when certificate was revoked I was moving it from original KeyStore to the blacklist KeyStore. Later in the certificate table content of the both KeyStores was displayed. Now I only add copy of the revoked certificate to the blacklist and later when I want to display content of each other KeyStores I compare it's certificates with blacklist. If certificate match one of the blacklisted I add to it’s GUI model status: „revoked”. What I achieve by that? It simplifies a bit managing certificates. I had to move certificates beetween TrustStore, ExceptionsStore and BlacklistStore, now it is done only between first two KeyStores. What else? Spark now make use of JRE default cacerts KeyStore. That should give users some widely trusted certificates at the beginning. Access to this KeyStore is made in way that content of this KeyStore isn’t ever deleted. When user will remove such certificate via my GUI it will be added to the list of distusted „non displayed and not used” certificates.

 

It’s the last week of the Google Summer of Code so it is also time to sum up my work. From technical point of view I have done it pretty well here so there is no reason to repeat myself. Though it is last week of the Google Summer of Code I have still some work to do fixing some smaller bugs. But generally I think the project ended up well.

 

From non technical point of view it was fun. Sometimes a bit tiring and frustrating but that’s happen when something doesn’t work as you wish and you are hopelessly trying to fix that. The thing I like is feeling after succeeding when you just want to run few times around your chair from happiness that something finally work. Both situations were occurring during this summer. Definitely GSoC was great opportunity to improve my coding abilities and extend my knowledge about XMPP and Public Key Infrastructure. I have meet many other contributors which is also great experience. To anyone hesitating to join to the Google Summer of Code I can say it is worthy.

 

Also thanks for my mentor Guus because it was really teamwork and I could learn a lot from him. And thanks for other community members who were supportive like wroot, speedy, dwd, akrherz, Flow and Kevish.

 

Farewell*

Paweł

 

*Or not really farewell as I still plan to contribute.

by Paweł Alameyo Ścibiorski (GSoC 2017) (igniterealtime@jiveon.com) at August 25, 2017 20:19

Paul Schaub

Final GSoC Blog Post – Results

This is my final GSoC update post. My name is Paul Schaub and I participated in the Google Summer of Code for the XMPP Standards Foundation. My project was about implementing encrypted Jingle File Transfer for the client library Smack.

Google Summer of Code was a great experience! This is a sentence you probably read in almost every single students GSoC resumé. Let’s take a look at how I experienced GSoC.

GSoC for me was…

  • tiring. Writing tests is something almost everybody hates. I do especially. In the beginning I imposed on myself to do test-driven development, but I quickly began to focus on real coding instead. Nevertheless, often I sat down for hours and wrote tests for classes and methods that were designed to allow easy tests, so I have an okayish test coverage, but it is far from perfect.
  • nerve-wracking. Dealing with bugs of sub-protocols I never worked with before drove me crazy sometimes. Especially that SOCKS5 Transport bug has a special place on my hit list. All in all I think the Jingle protocol has some flaws which make it harder to implement than it should be and those flaws (more precise – the decisions I derived from the Jingle XEP design) often really bugged me.
  • highly demotivational. Can you imagine how devastating it can be, when you spend days and days getting your implementation working with itself, only to see it miserably fail when you test it the first time with another implementation? It didn’t help that I tested my coded not only against one, but two other applications.
  • desocializing. I had to dedicate a huge part of my day to coding. As a result I often had no time left for friends and family.
  • devastating. In the end I wrote far more code than I should have. Most of it was discarded along the way and got replaced. It really annoyed me having to start from zero over and over again. Also giving up on goals like using NIO for asynchronous event handling pricked my pride.
  • to the highest degree depressing. Especially near the end I sometimes sat down and starred at my screen for some time without really knowing what to do. Those days I ended up making small meaningless changes like fixing typos in my documentation or refactoring. Going to bed later on those days made me feel bad and kinda guilty.

But on the other hand, GSoC also…

  • taught me, that coding is not only fun. Annoying parts like tests are (sometimes) important and in the end it is very satisfying to see that I managed to tame my code to a degree where all test cases run successful.
  • gave me deep insights into areas and protocols that were completely new to me. Also I think you can only fully gain understanding of how things work if you tinker with it for more than just one day. Finding and fixing bugs are a good exercise for this purpose.
  • was super motivational. Contrary (or let’s say additional) to what I wrote above, it is highly motivational to see your code finally work hand in hand with other implementations. That’s the magic of decentralized protocols – knowing that on the other end is a completely different device, running a completely different operating system with a language you might have never seen before, but still in some magical way, both implementations harmonize with each other like two musicians from different countries do when playing together.
  • was very social by itself. Meeting online with members of the community was a real pleasure and I hope to be able to participate in conversations and meetings in the future too. Also there were one or two (or three…) evenings that I spent gaming with my friends, but *shht* don’t tell anyone ;P.
  • taught me important lessons. While I usually aim too high with my goals, I learned that sometimes you have to take a step back to set more reasonable goals. Nevertheless ambitious goals can’t hurt and I heard that you grow together with your challenges.
  • was satisfying as f***. Sure, sometimes I was feeling bad because I could have done more on that day, but in the end I was a little surprised seeing the whole picture and how much I really accomplished.

The Result

An overview about what I achieved during the Google Summer of Code can be found on my project page.

This week I did some more changes to my code and wrote more tests *gah*. I’m really proud that my JET proposal found it’s way into the XSFs inbox. I’m really excited, what the future may bring for my specification :)

I want to take the chance to thank everyone in the XMPP community for welcoming me and allowing me to become a part of it. I also want to thank Florian Schmaus for mentoring me and giving me the idea to participate in GSoC in the first place.

As a last impulse -  why do we need Google (thank you too btw :D ) to initiate programs like GSoC? Shouldn’t there be more public, government financiered programs? I certainly think so, even though Google did a very good job.

So in the end GSoC was really a great experience. Sometimes it was hard and challenging, but I’m sure it would have been boring without a certain degree of difficulty. I’d absolutely do it again (either as a student or maybe a mentor?) and I can only encourage students interested in free software to apply next year :)

Happy Hacking!

by vanitasvitae at August 25, 2017 11:21

August 18, 2017

Paweł Alameyo Ścibiorski (GSoC 2017)

#14 Wrapping up and bug fixing

Hello,

 

This week apart from finishing last week things with mutual authentication I didn't done much new things. Instead I am focusing on fixing bugs in code made up to this point. One thing is creating KeyStores at Spark launch. While managing it's content work's Spark currently doesn't create KeyStores at the start up so user have to manually put them in security folder. Now it will not be a problem anymore but Spark lack a default certificates so I am still working on use of Java's cacerts content in Spark. That add a bit more complexity to existing Spark's KeyStore Management system as I want to use cacerts file only to read it's certificates, not to modify it's content (other applications can use it as well). I already use 4 KeyStores which are treated with different rules. Example problem is adding certificate to the exceptions list which usually mean adding certificate to the exceptions KeyStore and removing it from previous KeyStore. Not removing it would mean that there would be now 2 copies of the certificate in certificate table. There are some ideas how to work on that but as you can see things starting here to be complicate which might result in not so good looking code.

 

See you next week,

Paweł

by Paweł Alameyo Ścibiorski (GSoC 2017) (igniterealtime@jiveon.com) at August 18, 2017 22:42

Fanout Blog

Realtime data and continuous delivery

Software development teams are beginning to realize the benefits of continuous, test-driven delivery of new releases.

Instead of a single, milestone release (waterfall development) or multiple, internal test releases before major external ones (agile development), continuous delivery focuses on constant releasing of features to market throughout the development process.

The goal of continuous delivery is that code is always ready to deploy and features are constantly rolled out independent of ‘releases’ – and doing so properly requires realtime data.

...

by daniel at August 18, 2017 16:37

August 16, 2017

Paul Schaub

GSoC Week 11.5: Success!

Newsflash: My Jingle File Transfer is compatible with Gajim!

The Gajim developers recently fixed a bug in their Jingle Socks5 transport implementation and now I can send and receive files without any problems between Gajim and my test client. Funny side note: The bug they fixed was very familiar to me *cough* :P

Happy Hacking!

by vanitasvitae at August 16, 2017 14:49

Peter Saint-Andre

Providing an Alternative

Although I pay no attention to the news, I'm vaguely aware that perhaps the world is becoming more negative and irrational. Rather than getting lost in despair and confusion, I try to model a positive, reasonable alternative by living the best life I can and by writing philosophical books that are clear, concise, and constructive. I've written six such books, with a seventh half done and an eighth outlined. Even though each of these books takes 500-1000 hours to research and write, I make them all available for free on my website (however, I do charge for the paperback and ebook versions). To date I've also made no effort to market them, so I depend on word of mouth. If you have found value in any of these books, I'd greatly appreciate it if you could tell a friend, write a review at Amazon, or give a copy as a gift. Thanks. :-)...

August 16, 2017 00:00

August 14, 2017

Ignite Realtime Blog

Smack 4.2.1 released

I've tagged and released Smack 4.2.1 to Maven Central. We decided to omit the smack-omemo* in this release packages as the API is not yet stable. Paul is currently working on encrypted Jingle Transports as part of his GSOC project which also involves OMEMO, and we probably will make some API changes to the existing OMEMO API because of that. As always daily snapshot builds are available from igniterealtime.org/repo.

by Ignite Realtime Blog (igniterealtime@jiveon.com) at August 14, 2017 19:08

Fanout Blog

Building a realtime chat app with Django and Fanout Cloud

Chat is one of the most popular uses of realtime data. In this article we’ll explain how to build a web chat app in Django, using Django EventStream and Fanout Cloud. The Django EventStream module makes it easy to push JSON events through Fanout Cloud to connected clients.

djangochat

...

by justin at August 14, 2017 17:16

ProcessOne

ejabberd 17.08

Happy summer with ejabberd 17.08 !
This release includes great improvements and new features. It also includes fixes and closes the biggest milestone about refactor we’ve made last couple of months.
If you have issues with 17.04 or troubles using PEP, upgrade to 17.08 will fix most known issues.

New features

Introduce ‘hosts’ option

The option can be used as a replacement of ‘host’ option when several (sub)domains are needed to be registered for the module.
Note that you cannot combine both ‘host’ and ‘hosts’ in the config because ‘host’ option is of a higher priority. Example:

mod_pubsub:
   ...
   hosts:
     - "pubsub1.@HOST@"
     - "pubsub2.@HOST@"

XEP-0357: Push Notifications

This module tries to keep pending stream management sessions of push clients alive (as long as the disconnected clients are reachable via push notifications).

Modular cluster

For configuring the cluster new global option ‘cluster_backend’ is introduced. The default and only available value at the moment is ‘mnesia’.

Use c2s ‘certfile’ option

Use the ‘certfile’ listener option rather than a ‘domain_certfile’ for ejabberd_c2s listeners that have “tls: true” configured. A ‘domain_certfile’ should only be preferred for STARTTLS connections.

New mod_muc hooks

There are four new events: create_room, destroy_room, join_room and leave_room.
Note: destroy_room was previously used already by mod_mam and was named remove_room. remove_room is not renamed back to destroy_room for consistency.

Changes

Core

  • Erlang/OTP 17.5 or higher is required, and 20 is now supported
  • Make ejabberd_cluster modular
  • Replace gen_fsm with p1_fsm to avoid warnings in OTP20+
  • Fix clustering table reg_users_counter
  • ext_mod: Update spec from custom and allow modules dependencies
  • extauth.py: Fix to support : in passwords
  • Set high water mark in lager for all backends
  • Fix old route record in mnesia’s route table haven’t been remove when restarting in some cases
  • ejabberd_cluster*.erl: Add copyright and fix description
  • Add support of rfc6120 section 4.9.3.16 on node shutdown

Configuration

  • ejabberd_c2s: Fix priority of ‘certfile’ option
  • Introduce ‘hosts’ modules option
  • Fix ERLANG_OPTS, INET_DIST_INTERFACE and FIREWALL_WINDOW option
  • Remove unused ‘managers’ option, related to the deferred XEP-0321

Commands

  • Fix errors when running ejabberdctl as root
  • Fix set_presence command to work in recent ejabberd
  • Rename stop_all_connections to stop_s2s_connections for consistency
  • Change policy of user_resources command, from user to admin
  • Remove old command calling interface
  • Describe more command arguments and results

Modules

  • mod_http_api: Use hide_sensitive_log_data option when registering users
  • mod_http_fileserver: Request basic auth dialog from browser
  • mod_muc: Fix nick bug with MUC on riak
  • mod_muc: new hooks
  • mod_push: Support XEP-0357: Push Notifications
  • mod_push_keepalive: New module

PubSub/PEP

  • Keep disco#info on PEP compatible with XEP-0060
  • Preliminary export PubSub data from Mnesia tables to SQL file
  • Fix PubSub send last published items
  • Fix PEP node removal
  • Fix PEP node identity
  • Fix disco#items on PEP service
  • Fix getting cached last item
  • Add import of PEP from prosody

Documentation

  • Improved API documentation
    see https://docs.ejabberd.im/developer/ejabberd-api/admin-api/

Feedback

As usual, the release is tagged in the Git source code repository on Github.

The source package and binary installers are available at ProcessOne.

If you suspect that you’ve found a bug, please search or fill a bug report on Github.

by Christophe Romain at August 14, 2017 15:23

Paul Schaub

GSoC Week 11: Practical Use

You know what makes code of a software library even better? Client code that makes it do stuff! I present to you xmpp_sync!

Xmpp_sync is a small command line tool which allows you to sync file from one devices to one or more other devices via XMPP. It works a little bit like you might now it from eg. owncloud or nextcloud. Just drop the files into one folder and they automagically appear on your other devices. At the moment it works only unidirectional, so files get synchronized in one direction, but not in the other.

The program has to modes; master mode and slave mode. In general, a client started in master mode will send files to all clients started in slave mode. So lets say we want to mirror contents from one directory to another. We start the client on our master machine and give it a path to the directory we want to monitor. On the other machines we start the client in slave mode, and then add them to the master client. Whenever we now drop a file into the directory, it will automatically be sent to all registered slaves via Jingle File Transfer. Files do also get send when they get modified by the user. I registered a FileWatcher in order to get notified of such events. For this purpose I got in touch with java NIO again.

Currently the transmission is made unencrypted (as described in XEP-0234), but I plan to also utilize my Jingle Encrypted Transports (JET) code/spec to send the files OMEMO encrypted in the future. My plan for the long run is to further improve JET, so that it might get implemented by other clients.

Besides that I found the configuration error in my ejabberd configuration which prevented my Socks5 proxy from working. The server was listening at 127.0.0.1 by default, so the port was not reachable from the outside world. Now I can finally test on my own server :D

I also tested my code against Gajims implementation and found some more mistakes I made, which are now fixed. The Jingle InBandBytestream Transport is sort of working, but there are some more smaller things I need to change.

Thats all for the week.

Happy Hacking :)

by vanitasvitae at August 14, 2017 14:43

August 13, 2017

The XMPP Standards Foundation

Easy XMPP: The Challenges

Over the last years, the XMPP community has had a hard time competing with other Instant Messaging implementations, especially in the mobile / smartphone ecosystems. By focusing a small part of our resources on user experience (UX), we can gain significant improvements. This article is the first in a series of "Easy XMPP" posts: easy ways for application developers to make XMPP easy to use.

Complexity of Federation

As opposed to most other Instant Messaging solutions, XMPP is a federated protocol. That allows everyone to run their own servers, at the cost of additional complexity for users:

  • the user identifier always consists of a user and a domain part,
  • there is no central registry that will consume your phone book and tell you who else is using XMPP,
  • some servers might be running an older stack not supporting modern features, etc.

This inherent complexity, together with many developers' lack of attention to good UX, have left us in a situation where on-boarding of new users and finding contacts is painfully hard, especially when compared to proprietary/centralized IM solutions.

It is not possible to remove the inherent complexity of federation without replacing XMPP with a completely different protocol. However, there is another federated system that is well established and used by people all over the world: email. XMPP and email share the same basic principles - they are federated, user identifiers are user@domain and it is possible (albeit less common) to run your own server (or to get your own domain hosted with a commercial provider).

By leveraging users' knowledge about how email works, learning from over forty years of email evolution, and applying ideas from modern UX design on top, it is possible to make XMPP easier to use today.

Our Challenges

There are several areas where our community needs to improve. This post provides an overview of the challenges we are currently facing in different areas. Subsequent posts will dive deeper into the individual topics and work out possible solutions.

Terminology

The XMPP terminology is driven by technical requirements and exposes complexities of the protocol. Normal IM users don't want to know about PubSub Publish Options, asymmetric presence subscription in their roster, or MUC-PM Carbons. All they care about is to see whether their friends are online and that they can exchange cat pictures.

It is our task as the developer community to create understandable abstractions of the technical complexities, and to set up a common glossary for the user-facing elements, including translations into common languages. We need to define what the difference between "Jabber" and "XMPP" is, whether the user identifier is a "JID", a "Jabber ID", an "XMPP address", or a "user identifier", and fix all the other terms that are exposed to users and make clients inconsistent today. And then we need to apply this glossary to our implementations.

First-time User Experience

The first-time user experience for IM users needs to be streamlined. The users who need the most assistance are newcomers to the ecosystem who got invited by a friend, and start out with nothing but their friend's Jabber ID. They need help to accomplish these tasks:

  • install a modern client
  • create an account
  • add their friend
  • automatically find all their other friends
  • find other users and chat rooms

Projects like Easy XMPP Invitations, Pre-Authenticated Roster Subscription, and invite URLs are a first step in that direction, and getting them into your client today will make new users' lives easier. But there is also potential to further streamline the whole process.

Easy Group Chats

There are two typical IM use cases for group chats: chat with friends or family (private groups), or participate in a public chat room (typically for support purposes, pioneered by Internet Relay Chat).

The latter is well-covered by MUC, and the upcoming MIX should provide a solid technical basis for both use cases. The next logical step is to allow the easy creation and sharing of ad-hoc rooms between different clients, with a coherent user experience.

Fighting Spam

If the amount of spam is a measure of XMPP's popularity, the network is doing exceptionally well. Almost all of today's XMPP spam can be blocked with some easy pattern matching rules, and the inevitable arms race will move to the next round.

The community needs to prepare for that, by improving reporting between server operators, adding anti-spam features into clients and providing better whitelisting mechanisms to users.

Call for Action

You can help making XMPP easier to use. As a user, you can contribute ideas, add user stories, check your favorite client for confusing UI elements and non-helping wizards, and report those to the developers. As a developer, you can streamline the on-boarding process in your client and contribute to the common glossary.

Get yourself a wiki account and start hacking on Easy XMPP today!

by ge0rg at August 13, 2017 22:00

August 11, 2017

Paweł Alameyo Ścibiorski (GSoC 2017)

#13 Mutual authentication - continutation

Hi there,

It took me a bit longer than I thought to do mutual authentication but in the end, today it worked. How to present own certificates to remote party? Quite easily, 3 lines of code:

//Create KeyManagerFactory
KeyManagerFactory kmf = getInstance(String algorithm, String provider);

//Initialize it with content of the Keystore - it have to contain private key as well as certificate which will be send to remote server
public void init(KeyStore ks, char[] password);

//and during initializing SSLContext add:
context.init(kmf.getKeyManagers(), tmf.getTrustManagers(), new SecureRandom());

So what took me so long? It appears that somehow I was losing my private key during importing it to the Keystore. Also I had to create GUI for that. Still I have to polish it a bit and finish functionalities to create Certificate Sign Request and Self Signed certificates.

Here screenshot how it looks now:

sparkMutAuth.png

Generally compared to Certificates tab, here table showing Keystore content is smaller as I assume user rarely will need more than one certificate. However it might happen, especially  when user often switch domains using the same client and different servers have different CA in it's Truststores. I hope to finish it over weekend and then I will move to bugs  from earlier part of project. There is some of them but one is particularly bad as during moving certificates from and to Exceptions it happens that content of one of the Keystores is deleted. I still didn't figured out what is reason but soon, after finishing mutual authentication panel I should be able to look more into that.

by Paweł Alameyo Ścibiorski (GSoC 2017) (igniterealtime@jiveon.com) at August 11, 2017 22:10

Tigase Blog

XEP-0142 Workgroup Queues

Workgroups were developed as an extension to enable an XMPP server to behave like queue line for users awaiting assistance. The idea is that users are placed on a first-come first-served waiting queue to enter a chat room where they may communicate. Although an experimental extension, it is just one of the many ways in which XMPP can be shaped into a custom solution.

by Daniel at August 11, 2017 04:14

August 10, 2017

Peter Saint-Andre

None of My Business

Apropos of my latest journal entry "Politics is a Disease", here is a quote from Beyond Good and Evil (Section 251) by Friedrich Nietzsche:...

August 10, 2017 00:00

August 08, 2017

Tarun Gupta (GSoC 2017)

Week 9 & 10

Hello all,

We have a new design for MIX ! As proposed by mentors, we have a new implementation scheme for MIX involving MIX Registry, which will have access to all MIX objects for successfully joined channels. 

The new design allows requesting a MIX Registry from Client and detecting whether local server is supported.  It also adds XMPPRoster to be a part of MIXRegistry. A standard roster encoding is as follows:
<item jid='romeo@example.net'/>
However, if a roster item is a MIX channel, the new encoding will be :
<item jid='balcony@example.net'>
<channel xmlns=‘urn:xmpp:mix:roster:0'/>
</item>
Based on this, we now define a channel to be successfully joined or left based on the presence of channel JID in the user's roster. If a channel is successfully joined, a roster encoding with a channel element is added to user's roster. Similarly, when channel is left, roster encoding is removed from user's roster. 

In the upcoming weeks, I will update limber to have minor support for MIX and test my implementation against it.

by Tarun Gupta (noreply@blogger.com) at August 08, 2017 14:03

August 07, 2017

Paul Schaub

GSoC Week 10: Finding that damn little bug

Finally!! I found a bug which I was after for the last week. Now I finally got that little bas****.

The bug happened in my code for the Jingle SOCKS5 Bytestream Transport (XEP-0260). SOCKS5 proxies are used whenever the two endpoints can’t reach one another directly due to firewalls etc. In such a case, another entity (eg. the XMPP server) can jump in to act as a proxy between both endpoints. For that reason, the initiator (Alice) first collects available proxies, and sends them over to the responder (Bob). The responder does the same and sends its candidates back to the initiator. Both then try to connect to the candidates (in this case proxies) they got sent from their peer. In order for the proxy to know, who wants to talk to whom, both include a destination address, which is calculated as SHA1(sid, providerJid, targetJid), where the provider is the party which sent the candidates to the target.

The alert reader will know, that there are two different destination addresses in the game by now. The first one being SHA1(sid, Alice, Bob) and the second one SHA1(sid, Bob, Alice). The issue is that somewhere in the logs I ended up with 3 different destination addresses. How the hell did that happen. For the answer lets look at an example stanza:

<iq from=’romeo@montague.lit/orchard’
id=’xn28s7gk’
to=’juliet@capulet.lit/balcony’
type=’set’>
<jingle xmlns=’urn:xmpp:jingle:1′
action=’session-initiate’
initiator=’romeo@montague.lit/orchard’
sid=’a73sjjvkla37jfea’>
<content creator=’initiator’ name=’ex’>
<description xmlns=’urn:xmpp:example’/>
<transport xmlns=’urn:xmpp:jingle:transports:s5b:1′
dstaddr=’972b7bf47291ca609517f67f86b5081086052dad’
mode=’tcp’
sid=’vj3hs98y’>
<candidate cid=’hft54dqy’
host=’192.168.4.1′
jid=’romeo@montague.lit/orchard’
port=’5086′
priority=’8257636′
type=’direct’/>
</transport>
</content>
</jingle>
</iq>

Here we have a session initiation with a Jingle SOCKS5 Bytestream transport. The transport exists of one candidate. Now where was my error?

You might have noted, that there are two attributes with the name ‘sid’ in the stanza. The first one is the so called session id, the id of the session. This should not be of interest for our case. The second one however is the stream id. Thats the sid that gets crunched in the SHA1 algorithm to produce the destination address.

Well, yeah… In one tiny method used to update my transport with the candidates of the responder, I used session.getSid() instead of transport.getSid()… That was the bug, that cost me a week.

Now it wasn’t too bad. While I searched for the error, I read through the XEPs again and again, discovering some more issues which I discussed with other developers. Also I began testing my implementation against Gajim and I’m happy to tell you that the InBand Bytestream code is already working sort of. Sometimes a few bytes get lost, but we live in times of Big Data, so thats not too bad, am I right :P ?

In the last 3 weeks I plan to stabilize the API some more. Currently you can only receive data into files, but I plan to add another method which gives back a bytestream instead.

Also I need more tests. Things like that nasty sid bug can be prevented and found using junit tests, so I’ll definitely stock up on that front.

Thats all for now :) Happy Hacking!

by vanitasvitae at August 07, 2017 22:31

August 05, 2017

Remko Tronçon

Ken Burns Effect Slideshows with FFMPeg

One of the first things that impressed me about Mac OS X when I first saw it was its screensaver. Instead of just showing a simple slideshow of your pictures, it actually used a ‘Ken Burns’ panning and zooming effect with a fancy fading transition to make the otherwise static pictures really come to life. It always sounded like a fun project to create a standalone tool to create slideshow movies that used this effect, with full control over where and how much pictures should be zoomed. It turns out you don’t really need a new tool: you can get the same result using just FFMpeg. In this post, I’ll walk through the steps of creating slideshows using the Ken Burns effect.

Oh, and cats. There will be cats!

Continue reading post

by Remko Tronçon at August 05, 2017 22:00

August 04, 2017

Paweł Alameyo Ścibiorski (GSoC 2017)

#12 Client Side Authentication

Hello,

What's going on this week? I started working on next and the last big feature which is client side authentication. Support for it will allow for response to server certificate request from client. If we add it to server side authentication then we can say that it is Mutual Authentication. It isn't so wide used as usually server presenting it's certificates to the client is enough. Nonetheless if server is meant to connect only to trusted clients then it is nice feature.

Adding client side authentication require to provide to client's certificate chain and having private key for decryption. Overly this adding client side authentication seems simpler than validating chain of certificates received from server. As I understand it for now I have to only create instance KeyManagerFactory and initialize it with credentials which will be provided by SSLContext to the server. As part of implementation of mutual authentication I intend to add utilities for creating certificate signing request and self signed certificates. So far this with proper KeyStore managing and connect it well with GUI can give me some headache but I hope to overcome any problems soon. There is still some things I am yet to figure how to do but I know general direction for implementing mutual authentication.

 

See you next week,

Paweł

by Paweł Alameyo Ścibiorski (GSoC 2017) (igniterealtime@jiveon.com) at August 04, 2017 21:33

August 03, 2017

Fanout Blog

Reliable Server-Sent Events for Django

Today we’re pleased to introduce Django EventStream, a module that provides Server-Sent Events capability to your Django server application. It relies on Pushpin or Fanout Cloud to manage the client connections. Events can optionally be persisted to your database, for highly reliable delivery.

...

by justin at August 03, 2017 01:42