Planet Jabber

June 19, 2019

João Duarte

2nd GSoC week summary

What do you mean, its on fire?

Second week has been a little uneventful. Preparation for those last evaluations and working on GSoC is proving to be a hard task. The first GSoC evaluation is already around the corner, so pressure is definitely rising!

Events:

Research:

  • prosodyctl: 
    • Trying to reverse engineer mod_admin_telnet.lua and modulemanager.lua to deal with plugins. This path was discarded earlier, by my mentor's suggestion
    • Studying prosodyctl's util.startup.lua and how it installs plugins after reading the configuration file
  • Luarocks:
    • Studying luarocks_admin.add command and experimenting with it 

Difficulties:

  • Time management between university and GSoC's courses/projects evaluations

Future goals:

  • Import luarocks.admin's add, list and remove commands into prosody
  • Allowing the added functionality to recognize prosody's plugins and not just Rocks

Considerations: 

Fully understanding Prosody and Luarocks in order to make my own code is taking more time than it really should. I feel like time is going by and things are stalling too much. There is a lot of code around both projects that I don't really need but are quite mixed in the scripts, making it difficult for me to tell if my time is really being well spent when I'm trying to understand a utility script or some other piece of code.
After talking with my mentor, we've decided to simply add luarocks.admin's functionalities into prosody, and after that strip away unnecessary code. This should allow me to make some improvement and give the required background to build the solution we desire for the installer.
I feel like the most crucial thing right now is achieving a good level of focus, given that there are a lot of evaluations coming up, each one demanding attention. This seems to continuously cut time each day, and taking the required concentration to perform deep work, as Cal Newport would say. Despite my mentor's clear directions, it is challenging to take in all the code and concepts floating around and do something with it, without a clear mind. But challenges are what we are for, so I'm optimistic about this next week!



by João Duarte (noreply@blogger.com) at June 19, 2019 11:27

June 18, 2019

Isode

Isode at NITEC19

NITEC is the NCI Agency’s flagship annual industry conference, focusing on advancing technological solutions and business practices to strengthen NATO operations. This year’s NITEC was held in the Norwegian capital of Oslo between 20 and 22 May.

A regular visitor to NITEC, Isode spent three enjoyable days at this year’s show demonstrating our software for chat and email messaging to an audience of systems integrators, technical influencers and representatives of end-user organisations (the militaries of NATO nations).

Isode’s Louise Hill (Pre-Sales Engineer) and Jeff Tillotson (Business Development Manager: Defence) setting up the Isode stand.

Of particular interest to visitors was Isode’s ability to connect different services, employing different chat and messaging standards, so that they become seamless end-to-end systems. Our demonstration of connecting XMPP based chat systems to those employing the legacy IRC system (using Isode’s M-Link IRC Gateway) was particularly popular.

Both chat and messaging demonstrations also featured security label translation, mapping labelled messages from one security domain to another (for the show we used Norwegian and NATO labels on the different “sides” of the messaging/chat systems).

Evaluation software for both Email Messaging and XMPP Chat software is available from the Isode website.

by admin at June 18, 2019 15:15

ProcessOne

Real-time Stack Issue #23

ProcessOne curates two monthly newsletters – tech-focused Real-time Stack and business-focused Real-time Enterprise. Here are the articles concerning tech aspects of real-time development we found interesting in Issue #23. To receive this newsletter straight in your inbox on the day it’s published, subscribe here.

Mozilla relaunches open source IoT platform Project Things as WebThings

For the better part of two years, the folks at Mozilla have been diligently chipping away at Project Things, an open implementation of the World Wide Web Consortium’s Web of Things standard for monitoring and controlling connected devices, and today, they reached a major milestone.

Jaguar Land Rover to let drivers earn cryptocurrency by sharing data

Owners of cars made by Jaguar Land Rover will be able to earn cryptocurrency as they drive by enabling their vehicles to report “useful” data.

Easy start into the world of IoT with MQTT

The Internet of Things is getting everywhere – so more important it is to understand the principals behind. In this example we show how easy everybody can build its own ideas for the digital world based on open standards like Arduino, MQTT, Node-RED… in a simple and easy “visual” coding.

Mainflux LoRaWAN?tutorial

LoRa (Long Range) is a patented digital wireless data communication technology developed by Cycleo of Grenoble, France, and acquired by Semtech in 2012. LoRa uses license-free sub-gigahertz radio frequency bands like 169 MHz, 433 MHz, 868 MHz (Europe) and 915 MHz (North America).

Device originating communications, or how and why MQTT rocks

The story goes that a valve manufacturer wanted to have their networked smart valves certified for use in a nuclear reactor plant. The smart valve could report all sorts of critical data points to a database system and also be controlled by that SCADA system in the plant.

Public Jabber chat rooms

A listing of public XMPP/Jabber chat rooms/groups curated by Christopher Muclumbus.

by Marek Foss at June 18, 2019 07:10

hrxi

Third week: Interoperability testing

I managed to get the basic in-band bytestream file transfers working. It still doesn’t have any user interface, that’s depending on a project by my mentor. For now, it’s simply triggered by a hardcoded target JID that gets a file transfer when it sends a presence.

After finishing the proof of concept, I added some error handling and did some code cleanup. With that done, I did some interoperability testing. With Gajim, the file transfers now worked well, so I moved on to test with Conversations. Here, something funny happened, my Dino code sent (abbreviated for brevity):

<iq type='set'>
  <jingle action='session-initiate'>
    <content>
      <description xmlns='urn:xmpp:jingle:apps:file-transfer:5'></description>
      <transport xmlns='urn:xmpp:jingle:transports:ibb:1' block-size='4096' />
    </content>
  </jingle>
</iq>

And Conversations responded:

<iq type='set'>
  <jingle action='session-accept'>
    <content>
      <description xmlns='urn:xmpp:jingle:apps:file-transfer:5'></description>
      <transport xmlns='urn:xmpp:jingle:transports:s5b:1'>
        <candidate host='146.255.57.229' type='proxy' jid='proxy.jabber.ccc.de' port='5224' />
      </transport>
    </content>
  </jingle>
</iq>

In essence, my Dino code asked to open a Jingle session for a file transfer, using in-band-bytestreams as the transport. Then, Conversations responded that it accepts my offer of Socks 5-based Jingle transport for the file transfer, accepting an offer I didn’t make. I think this is a violation of the Jingle XEP (XEP-0166): If the other party wants to negotiate a different transport, it should probably use the transport-replace Jingle action. I created an issue (Conversations#) which was quickly resolved. I have yet to test the fix, I don’t have a development setup for Conversations yet.

The in-band bytestream file transfers I tested were quite slow (<2KB/s), probably some rate-limiting of the servers, I didn’t investigate furtherly.

by hrxi at June 18, 2019 00:00

June 17, 2019

The XMPP Standards Foundation

XSF's Google Summer of Code 2019 Projects

The XMPP Standards Foundation is happy to act as umbrella organization for Google Summer of Code (GSoC) 2019. This year we are glad to mentor and support the following three GSoC projects.

Prosody plugin installer

Student: João Duarte GSoC Blog: https://gsoc-prosody-2019.blogspot.com Project: Prosody - An XMPP server written in Lua Mentors: MattJ, Zash

João lives in Portugal and usually studies Aerospace Engineering. This summer he will be developing an integration between Prosody and the LuaRocks package manager and ecosystem,allowing installation and management of third-party Prosody modules via simple commands.

Poezio infinite scrolling using MAM

Student: Madhur Garg GSoC Blog: https://madhur96.github.io/gsoc19/ Project: Poezio - A free console XMPP client in Python Mentors: pep, Link Mauve

Madhur is a student of the International Institute of Information Technology in Hyderabad, India. He likes Python and C++. Outside of programming, he likes guitar and trekking. This summer he will be working on adding “Infinite scroll” support to poezio, the console client, using XEP-0313: Message Archive Management, as well as some other general improvements.

Jingle File Transfer for Dino

Student: hrxi GSoC Blog: https://hrxi.github.io/gsoc/ Project: Dino - A modern open-source chat client for the desktop in Vala Mentors: fixah, la|r|ma

hrxi will implement encrypted peer-to-peer Jingle file transfers in Dino. The encrypted connection will be set up by either one of the supported encryption protocols of Dino, OMEMO and OpenPGP. In order to achieve compatibility with web clients, it is planned to support WebRTC as a transport protocol. The implementation will be done in a way that allows building support for audio/video chat on top of it. hrxi is based in Germany, studies mathematics and computer science, and likes to write code in Rust.

by Flow at June 17, 2019 20:00

Monal IM

iOS 3.8 Released

Release 3.8 should make jumping on to XMPP easier for non technical users. The new registration screen will bring us closer to a chat platform that anyone can download and use.

by Anu at June 17, 2019 15:19

June 13, 2019

Jérôme Poisson

SàT progress note 2019-W16

Hello everybody,

this is the time for a second progress report.

After the implementation of the button for translations, I've updated the French version of Libervia, and of the new website I'm currently working on (which is not yet online). For now this is done locally using tools like Gtranslator or Poedit, I plan to install at some point a web app like Weblate or Pootle and if possible to integrate is with SàT/XMPP (at least for authentication) to make contributions easier.

Beside that I've mainly been working on photos album, I want to be able to use it with 0.7.
For a bit of background, photo album is a specialized view of file sharing. I've evaluated two XEP (XMPP Extension Protocol) for that:

  • File Repository and Sharing (XEP-0214) which is based on Pubsub and Collections Nodes (XEP-0248)
  • File Information Sharing (XEP-0329) which is a simple way to share a repository

File sharing is usable either with direct sharing of a repository from a device (e.g. photos from a mobile phone), or with a server component which host files.

I've chosed the second one (File Information Sharing) for now because the Pubsub one is based on Collections which is, in my opinion, currently not usable: permissions from collection nodes are overwritting ones of leaf nodes, and as a result a private node can be accidentaly opened. This needs to be addressed, it's not the first time I discard pubsub collections because of that, I'll try to propose changes to standard after the 0.7 release.

The other reason I've chosen "File Information Sharing" (FIS) is that the Pubsub one handles mirrors and versions, which I feeled overcomplicated at the time. With FIS, I could make an implementation quickly, and I have a working UI now, which is already quite usable (see this blog post to see how it looks like).

But when I'm using the component, I have no way to change access (everything is managed, there is just no interface to change it), so when I put a file on the file sharing component, it stays private for now, not ideal when you want to share a photo album.

So I've worked on a quick way to do it, using ad-hoc commands: one to change file/directory permissions, and one to delete files. It's nearly finished and will be the last thing before starting beta phase.

That said, Pubsub has already everything needed to manage access and subscriptions (to know when new files/photos are available), so I plan to re-evaluate XEP-0214 at a later point, and if I still find it ill-adapted, maybe propose an other option.

I've also noticed a couple of CSS issues on the blog engine (mainly some padding around paragraph would make it easier to read), and I've been noticed that link to Atom feed is missing on the blog. I was planning to fix that this week but could not find the time (I'm working on SàT on my free time). So I hope to do this in the next few days.

That's it for this week. I'm looking forward to start debugging phase, and then finally release.

by goffi at June 13, 2019 07:30

SàT progress note 2019-W17

Hello,

this week I've had little time to work on SàT : thanks to the extended easter week-end, I've had a break.

I did manage to work a bit on the file sharing service, still focusing on the photo album. As I've explained last week, file sharing can be used directly from device to device (e.g. retrieving pictures taken on a mobile phone from desktop computer), or with a server-side service.

XMPP allows to create "components" which can be seen as generic plugins to create services on the server (generic in the way that it's not tied to a specific server implementation, but can be used with anyone implementing XEP-0114). Starting with the incoming v0.7, SàT can be used as a component, and file sharing is the first one.

Until now, it was only used to store personal data, so the files could be accessed with the jid of the component. For instance, if your server is example.net your file sharing component may have the jid files.example.net. To access photo albums, we need the path of the album (e.g. /photos/week-end) which is set in the node attribute in XEP-0329, so you can request files.example.net to retrieve your files (i.e. photo album) from /photos/week-end.
Problem: if Louise is connected to files.example.net and wants to access /photos/week-end, she will get her own album. How to do if she wants to access the one of Pierre?

That's were the local part is used, i.e. the part before the @ in a jid. The files.example.net component will receive all traffic sent to any jid finishing with @files.example.net. So when Louise wants to access Pierre files, she can use pierre@files.example.net instead of just the domain name. That's what I've implemented this week. By using just pierre, the component will associate this with its own server (i.e. example.net), and will look for files from pierre@example.net (and check which ones Louise can access).

But what for people with an account outside of example.net, for instance what if file sharing service allows nestor@example.org to store files? Louise can't request nestor@files.example.net because it would get the files of nestor@example.net (note the .net insteaf of .org , it's a different user). To allow to retrieve the files of an external account, we should be able to use a full jid in the request, but it's forbidden to use @ in local part of the jid.

That's where XEP-0106 is used, it allows to escape a jid to do exactly that. I've implemented that so Louise can request nestor\40example.org@files.example.net to retrieve files from nestor@example.org. Don't worry for the complicated address, the end-user should not have to type it herself.

Last but not least, I have also fixed the margin issue with paragraphs and added missing Atom feeds links, it should hopefully be more confortable to read this blog.

To summarize, during this short week I've implemented a way to retrieve files from an other user in file sharing component, I've implemented XEP-0106, I've fixed a small CSS issue on the blog and I've added missing links for Atom feed.

by goffi at June 13, 2019 07:29

SàT progress note 2019-W18

Hello,

this week I've had to fix the way jid are used for file transfer.

To explain you the problem, I need to explain you how communication is done in XMPP (*). To be brief, when an entity (with jid louise@example.net) want to access a component (with jid files.example.net), it sends a "stanza" which is an XML element. There are 3 kinds of stanza (presence, message, and iq).
In our case we use iq stanza, so Louise sends something like:

<iqfrom="louise@example.net"to="files.example.net"type="set">
  […]
</iq>

and the component answers with something like this:

<iqfrom="files.example.net"to="louise@example.net"type="result">
  […]
</iq>

Last week I've explained how I've used the local part in components to access files from somebody else. So far, when creating a stanza, I was using the jid we are connecting with in the from attribute (i.e. where we set the sender of the message).

This is working well with clients, or components if you don't use local part, but when somebody was accessing files from pierre@files.example.net, the jid used by the component is files.example.net, a different one, so this was not working anymore.

To fix that, I've simply had to change the code to use the jid used to contact the components (i.e. the one in the to attribute of the original request) instead.

I've also fixed a MAM (Message Archive Management, XEP-0313) bug where the same messages were requested again on next start up under certain conditions.

Finally, I've completed a generic invitation mechanism to notify somebody when an event or photo album is available. When the invitation is received, the data needed to retrieve the thing is saved in a private "list of interest", which is a pubsub node. With that I can now diplay in Libervia the available photo albums as you can see in the screen capture below.

Libervia Photo Album Vignette

We are nearly there, I still have to complete the guest page (page for people without XMPP account), so they can see events or photo albums, and I can launch the beta. It's a matter of days now.

Thanks for reading, as always feedbacks are welcome.

(*) to learn more about XMPP, you can check my series of articles "Let's talk XMPP". It's originally written in French, and only 4 articles on 10 are translated, help would be welcomed to translate the other ones.

by goffi at June 13, 2019 07:28

SàT progress note 2019-W24

As mentioned in my note 2 weeks ago, I was in holidays and skipped last week note.
It has been a really good thing, a relief, to be out of the screen for a couple of days, I recommend!

So I'm now back to work in my paid job, and working again on SàT on my free time. We are already in the middle of June, and the release is still not done; it's time for the home straight. As you can see if you follow progress notes, I'm working regularly, and moving forward each week, but there is so much to do. Anyway things are getting better now, the new website is online with documentation, the tickets system is working even if still buggy (it's getting better with time), and I start to have feedbacks, meaning that people are testing.

During my holidays I've got a report on ActivityPub about an issue with <link> elements, followed by a bug report. With those informations, I could spot the issue and fix it (it was a problem with caching, the data used in template were not deeply copied, and the list containing links was reused and growing each time the page was accessed). Thanks to Rydroid for reporting the issue.

When you test and see something wrong, the right thing to do is to contact me, and if possible report a bug on https://bugs.goffi.org. If it's easier for you, you can contact me by other means, but the ticket creation is the better way to be sure it's not forgotten.

I've finished and deployed the pubsub experimental option I have talked about in a previous note. I've called it consistent publisher and the goal is to keep original publisher if a node owner or an admin modify an item. The main reason is to maintain permission to edit the ticket for the original publisher, and thanks to that I can now change status of a ticket without replacing publisher.

Overwriting an item is not yet ideal in pubsub: either you follow XEP-0060 and any publisher can overwrite any other publisher item (not great in case of tickets or blog comments), or you restrict overwriting to your own published items but you need to do a workaround like this. I have ideas on how to improve the situation with node options and I'll probably propose a XEP at some point after SàT release.

As you know, I'm focusing on debugging, and I've spotted a couple of annoying bugs that I'll fix in the incoming week. My target is a release before the end of June. I have started work on packaging Cagou for desktop with Flatpak, so it should be easy to install and test on most GNU/Linux distributions, I'll probably talk about that in next week note.

by goffi at June 13, 2019 07:24

June 11, 2019

Ignite Realtime Blog

XMPP Sprint in The Hague, NL, June 2019

@guus wrote:

Last weekend, I participated in an XSF-community organized sprint, in The Hague (conveniently located in my country of residence). Graciously hosted by IPerity, a group of people from various XMPP related projects got together.

Apart from it always being nice to meet up with fellow enthusiasts, we got quite some work done. In the sprint preparation, we had already established that one of the topics that we’d like to address, was the lack of top-notch (mobile) client support, in XMPP world.

Much of the work that we did revolved around adding better support for client notifications. This lead pep. and Laszlo to build a prototype for browser-based push notifications, which they got working in Converse (the web client that Openfire ships in the “inVerse” plugin) and the Prosody XMPP server. A goal for this effort is to evolve into a new XEP for web push.

Yours truly is happy to report that a new plugin was created for Openfire, that adds XEP-0357-defined push notifications to Openfire! The code is still somewhat rough around the edges, but I’d love for people to give this a test run, and how the prototype holds up!

Unrelated to the various notification-related efforts, Erik got quite a bit of work done on a refactoring project in the Gajim client, which over the past weekend is growing to be the basis for a refactoring of Gajim’s chat window UI.

I’m thrilled by the prospect of hosting a major, hopefully Openfire-based, hackathon in the near future, that is geared towards a particular industry in the Netherlands. That might be an exciting precursor for another increase in XMPP utilization! More on that later though!

I’ve had a blast! I loved meeting up with everyone, and enjoyed the discussions we had. Let’s do this again, soon!

For other Ignite Realtime release announcements and news follow us on Twitter

Posts: 3

Participants: 2

Read full topic

by @guus Guus der Kinderen at June 11, 2019 12:59

Monal IM

iOS 3.8 Registration and Change List

iOS 3.8 is ready to be tested and available on Testflight. The change list is:

  1. In app registration on blabber.im
  2. In app password changes
  3. Addressed compatibility issue specifically with prosody that caused higher batter use.

All of these changes are important but we really need to test the registration flow. Please go to settings->quick setup and try registering a new account on blabber.im. This is the welcome screen new users see so this has to work perfectly. If you are having issues with Monal in general, having a blabber.im account is a good idea because I have tested with this server and can confirm that everything works as expected. This will let us determine if any bugs you experience are related to your specific server, Monal or your device.

by Anu at June 11, 2019 01:06

June 10, 2019

hrxi

Second week: IBB Proof of Concept

As the first task, we planned to get a working, proof-of-concept Jingle file transfer over an “in-band bytestream” (IBB, XEP-0047, XEP-0261). To get to this point, I planned to write a module for basic Jingle interactions, supporting the necessities for file transfers, and a non-final API for transport methods on the other side.

What do in-band bytestreams look like? “In-band” means that the file data will go through the same connection as the rest of the XMPP stream that also contains your chat messages and metadata like read receipts, etc. It has to fit into the XML stream, an example with an 28-byte file below:

<iq xmlns="jabber:client" to="tish@example.com/gajim" from="blagovest@example.com/dino" type="set" id="e8656828-2da8-48a5-828d-804542945851">
	<data xmlns="http://jabber.org/protocol/ibb" seq="0" sid="514ef10b-22e7-4e58-bf46-cca01629e826">
		SGVsbG8sIGluLWJhbmQgYnl0ZXN0cmVhbXMhCg==
	</data>
</iq>

As you can see, this is quite inefficient. The <iq>/<data> tags amount to roughly 256 bytes. The actual data has to be base64-encoded because XML cannot contain arbitrary binary data, leading to an extra 33% overhead. With the default blocksize of 4096 bytes as specified in the IBB XEP, we get a total overhead of 40%. Nevertheless, this transport is simple to implement as it does not need additional network connections beyond the already existing.


While trying different approaches for the Jingle transport API, fiaxh (my mentor) helped me decide: I’m now using abstract classes for the different transports, and signals for the users of a Single session, a pretty Vala-esque approach. After this planning, I went ahead and tried to realize this in actual code. This resulted in basic session opening support (only initiating), support for in-band bytestreams and file transfers.

What’s still missing for merging this? I was pretty happy that the code almost worked after fixing syntax error and obvious oversights. It still has a pretty serious bug: Currently, when testing the file transfer from Dino to Gajim, Gajim drops the last chunk we send. Additionally, the code needs proper error handling, currently it just writes a message to the standard error output.


During the weekly organization chat, flow sent me an interesting document written by someone who implemented the Jingle protocol in multiple clients and wrote about the “enlightenment” he achieved in doing so. Thanks for that, I’ll check it out the next week.

by hrxi at June 10, 2019 00:00

June 08, 2019

João Duarte

1st GSoC week summary

README


These summary posts aim at being direct and concise, unlike those aimed at brainstormings, which are more verbose. I'll only post here the main points, without much elaboration in order to keep it short.

Events:

Prosodyctl:

  • Made a simple command function that prints out a welcoming message
  • Made a command function that prints out a list of currently enabled modules
  • Made a command function that prints out locally available modules

Research:

  • Mercurial practice:
    • evolve extension
    • ssh connections
    • branching
    • resolving conflicts
  • Prosody: Re-reading most of the documentation
  • Luarocks: 
    • Studying Luarocks-admin commands
    • Looking for packages.path and packages.cpath related code

 Difficulties:

  • Community Bounding Period mindset - I feel like I'm still biased to do research like if I was in the CBP in order to review and understand a growing number of concepts
  • Few commits - A feeling that I'm still not totally aware of what is going on has led to few commits, in my opinion, this week
  • Final weeks of the semester - Well, everyone should be having time issues related to this by now, but nevertheless it still has a considerable toll on project development

Possible/Planned future goals:

  • Improve current prosodyctl commands to list all locally available modules and tell which ones are enabled and disabled
  • Implement a command that enables or disables local modules (Probably similar with what happens with the telnet console commands)
  • Decide how should the program behave with modules that require configuration
  • Understand how Luarocks deals with package management and dependency resolution

Considerations:

The first GSoC has been stressful. One of my main personal problems is a continuous struggle between researching and trying to figure stuff out before actually starting to something, even after been given perfect instructions by my mentors. At some point this week it was clear to me that this was happening, since I was continuously delaying my blog post and code writing. Therefore I forced myself to put out some simple commits, has a starting point. Hopefully I will be able to build more on top of them, while grabbing all the info that I need.

by João Duarte (noreply@blogger.com) at June 08, 2019 18:02

June 07, 2019

Tigase Blog

BeagleIM and SiskinIM just got OMEMO support

New versions of our client applications targeted for Apple ecosystem just got released. The biggest feature included in both BeagleIM (for macOS) and SiskinIM (for iOS) is support for OMEMO end-to-end encryption protocol.

by wojtek at June 07, 2019 16:51

ProcessOne

ejabberd & Nintendo Switch NPNS

Last week, Taiyo Watanabe from Nintendo shared some insights about Nintendo Switch Push Notification infrastructure (NPNS) implementation. Here are some details from his presentation at ElixirFest conference.

The Nintendo Switch NPNS is a large-scale realtime messaging service based on ejabberd. The game consoles are constantly connected to the infrastructure, listening for push notifications. So far, Nintendo shipped more than 34 million units, thus the infrastructure needs to handle millions of concurrent connections.

The notifications include user-based (directed at active players) and system-based (automatic updates, parental controls). They also include the PubSub topic-based ones like the update notifications and important news, and are available when connected and after a successful login.

In terms of technology, XMPP was chosen due to the large feature set of that protocol. ejabberd was chosen because of its scalability and robustness. It is built in Erlang and inherits many of Erlang strengths: scalability, robustness, native clustering support, and more.

To ease maintenance, Nintendo ejabberd cluster is split into two areas: the inner ejabberd (cpu-bound) and outer ejabberd (memory-bound). All the nodes are connected by the distributed Erlang communication.

The job of the inner ejabberd is to retrieve messages from Amazon Simple Queue Service (SQS), while the outer ejabberd handles the distribution to the Switch game consoles. Business requirements include 1 million concurrent connections, and the recovery for bursting logins in max 30 minutes.

The first performance results of the load test with a simple stock ejabberd service was not sufficient for the production requirements, with capacity at 300k connections and 150 connections per second. Therefore, performance tuning and system redesign was conducted.

The first improvement to reduce memory footprint was to remove the XML parser except from the session memory when it was not active. The second change was to tune OpenSSL parameters to reduce the buffer size. Consequently, the maximum capacity increased to around 750k concurrent connections.

Another issue was the congestion on sockets during burst logins and slow item distribution for some topics, due to the process bottleneck. To solve this, a dedicated distribution worker process was created for each socket. As a result, login throughput increased from 150 per second to 300.

However, the mass delivery of contents created another congestion between the databases and Erlang cluster. To mitigate this, the action was to move content distribution processes from the inner to the outer cluster. Additionally, selective relaying from the inner to outer cluster was set up using a process dictionary to speed up access.

Another interesting problem appeared from the fact that XMPP accepts multiple logins of the same ID (for example, multiple devices). This resulted in too many session processes being created and too many controls being performed to apply standard XMPP business rules. As those rules were not needed in Nintendo use case, the quick solution was to just do nothing when a session is disconnected, and to make the session process die in 1 minute instead of 30.

The above issue was found using Erlang tracing. However, log processing time became a pain and was increasing exponentially, because of reductions (erlang:bumpreductions/1). To be able to handle that, a “dam” process was created to perform rate limiting. Thanks to it, 500k log lines could be processed in 30 seconds instead of 2 hours!

After all the tuning, the infrastructure met business requirements and was ready for production deployment. But that wasn’t the end of tweaking.

Production service was up, but monitoring signalled Redis load was five times bigger than what was expected. A message loop was discovered under certain disconnection conditions. One possible cause might be unprocessed messages triggering an exchange loop. However, testing any hypothesis on the whole production system was too risky. Therefore, a manual hot code deployment was applied directly on only one of the nodes, handling just a small part of the traffic.

Luckily, the patch on that node was behaving properly, and it was extended to all other nodes. Another higher load was detected in relation to small messages, caused by “busy waiting” BEAM scheduler. The solution: switch +sbwt from medium to very_short resulting in CPU use dropping from 30% to 21%.

The key lesson from this project is to always perform load-testing and profiling. Searching for bottlenecks should be continuous, keeping a lookout for C extensions (NIFs). A good approach is to keep increasing parallelism, reduce per-process load and set rate-limiting. Scaling a service is hard and directly depends on your use case. As you scale, you may discover new bottlenecks and parameters to tune and adjust. Scaling a service is an ongoing effort.

Today, the Nintendo Switch NPNS handles 10 000 000 (10 million!) simultaneous connections, 2 000 000 000 (2 billion!!) messages per day with 100-200k connections per node and 600 messages per second!

The greatest advantage of ejabberd is its incredible scalability and throughput. Erlang VM was also very useful, allowing troubleshooting a live system using Erlang remote shell. In the end, this project was implemented in just 6 months, with 1 month of prototyping. It’s been up and running ever since without major issues.

Here is the link to the slides of the presentation (in Japanese).

by Marek Foss at June 07, 2019 07:30

June 06, 2019

Monal IM

iOS 13: Embracing Darkness

Monal will support dark mode the day iOS 13 launches. There are a few things to fix here and there regarding images and contrast, but we are definitely ready for iOS 13. This may seem like an odd diversion but is very important. The first is this is someone’s first impression of XMPP and not supporting the latest UI is a great way to make them think XMPP is old. The second is the day iOS 13 comes out, people will seek out apps that support the latest features. These are people who will likely install Monal and experience XMPP for the first time. We need these new people.

by Anu at June 06, 2019 00:42

June 05, 2019

Monal IM

How wwdc 2019 will change Monal

There are a lot of new developments coming out to wwdc right now that will impact Monal and XMPP. I am still going over the technical documentation, but the most relevant items are :

  1. I will switch to catalyst to bring iOS ans Mac apps to parity. Maintaining one ui will also speed up development
  2. cryptokit will change the way we do omemo (for the better). I expect to be able to leverage this to release in France.

by Anu at June 05, 2019 16:41

June 03, 2019

Ignite Realtime Blog

User Status Openfire plugin 1.2.1 released

@wroot wrote:

The Ignite Realtime community is happy to announce the immediate release of version 1.2.1 of the User Status plugin for Openfire!

User Status plugin automatically saves the last status (presence, IP address, logon and logoff time) per user and resource to userStatus table in the Openfire database.

This update adds support for longer IPv6 addresses and fixes a few issues with plugin’s build.

Your instance of Openfire should automatically display the availability of the update in the next few hours. Alternatively, you can download the new release of the plugin at the User Status plugin archive page.

For other release announcements and news follow us on Twitter

Posts: 1

Participants: 1

Read full topic

by @wroot wroot at June 03, 2019 20:11

June 02, 2019

Monal IM

Registration

The goal here is to get more users who aren’t power users. For these users there needs to be less exposure to the technical workings of XMPP and fewer options. The usual account screens are still there in settings but a new user will encounter something like this. Someone should be able to download Monal, hit register and talk to another XMPP user without any hand holding. Part of plan to get to that goal is the new registration screen. This is a work in progress and currently hard coded to blabber.im but could be any server if needed.

by Anu at June 02, 2019 02:43

May 31, 2019

Jérôme Poisson

SàT progress note 2019-W22

Hello everybody,

during this week I've put online the new official website at https://salut-a-toi.org.

This is an important step for several reasons, let me explain.

The former site (which is currently still available at https://old.salut-a-toi.org, not sure yet how long it will stay here, but it won't be maintained anymore) was made with Django + bootstrap. It was working and not so bad, but outdated (with a really old screenshot on the front page), and it was a pain to maintain it in parallel of the work on SàT
Django is great, powerful, with a big community, but using it only for our presentation website means that we have to follow its evolution, and we have to solve issues from time to time, e.g. when we update the version. Django template engine is similar to Jinja2 that we use, but it's different enough to have to check documentation to maintain it, not a big deal, but some work that we could avoid.

By moving the website to our own Libervia framework, we stay on the exact same stack as SàT itself, so we don't have to follow other frameworks/libraries, it's easier to maintain. Also it's a good use case to test the framework, and make it evolve according to real life use case.
Furthermore, it simplifies our life for some features, like the news available at https://salut-a-toi.org/news, which is an XMPP blog. With Django we were using a iframe displaying a Libervia instance, while now we can directly show the blog.

With the new website I have added a "documentation" section. So far the documentation was mostly on the wiki, but it was not well maintained and a bit messy. Now it's inside the project itself, and written using reStructuredText which is the markup used most of time with Python project. Also we use Sphinx to render it nicely. As a result, we have a better rendered documentation, which is closer to the project (we can even use source code itself and its docstrings) so it should be up-to-date, and we can use the same source to produce things like man pages or a PDF manual.
Having to render the documentation with Sphinx, was the main reason to create our task manager in Libervia Web Framework.

The documentation is far from being complete, but I'm working on it. You can already see how to use jp for blogging or sending end-to-end encrypted message.

Now the TODO list for the release is done. What is left is debugging, writing documentation and packaging… And some rest! I'm currently in holidays so there will probably not be any progress note next week.

by goffi at May 31, 2019 15:08

May 30, 2019

Monal IM

WWDC 2019 and the Future

Lots of things in the pipeline and many changes are afoot. iOS 3.7 is available. 3.8 will come next week and there is a Mac refresh coming with the same fixes. The WWDC keynote is on Monday and we will see a lot of changes coming to Mac and iOS.

iOS is expected to have a dark mode and I plan on having Monal support on day of the iOS 13 release (likely Sept). I will likely start adding support for it and have it in betas as soon as I can. The process of adding dark mode in macOS was pretty straightforward and I have many of the art assets ready for it already.

Mac is going to see a huge change. At the moment I maintain two apps a Mac app and an iOS app. We expect to see marzipan based iOS apps running on Mac OS and depending on what the capabilities are, I hope to have it ready on day 1 as well. This will present an interesting scenario where I will have two Mac apps. As a long time Mac user, I know the thought of iOS apps seems dreadful at first glance, but the reality is this is likely the shot in the arm the OS needs for app support. The current one will continue to exist because there are still many Macs out there that will not run iOS apps. However the experience for the two apps will be very different, I am going to have to decide what I will name the two (the current one will likely be Monal Classic) .

The plan for both apps (but iOS first) is to have an app that requires no explanation. We should be able to tell people to grab the Monal app, register or login and then chat. There are a lot of messaging apps and either it works perfectly on the first try or it gets uninstalled.

The general roadmap before I take a chunk of the summer off:

iOS:

Registration, Password change and QR codes.
Dark Mode
Muc fixes /Mix
New Contact List
encrypted attachments
simple video/audio support

Mac:

Marzipan
Classic App iOS Parity

by Anu at May 30, 2019 23:29

iOS 3.7 update

Late in testing there was a critical bug discovered that prevented openfire servers from working with Monal. This has been resolved in the new beta out today and I hope to have 3.7 released this week.

Testing on a local open fire server I was reminded of a few things, Openfire is super easy to use, its super popular and the fact that it doesn’t support push is maddening. Plugin development seems to be reasonably easy any Java developers out there interested in helping me implement push?

by Anu at May 30, 2019 14:00

Arnaud Joset

Using sat-pubsub, a great pubsub component

In the continuity with the previous post about jp, the following article present sat_pubsub, a XMPP Publish-Subscribe (Pubsub) Service Component, build for the need of the « Salut à Toi » project.

Salut à toi (SàT) is a unique XMPP client. As its official description says, it's a "multipurpose, multi front-end, free (libre) and decentralized communication tool". It has been actively developed by Jérôme Poisson (Goffi) and Adrien Cossa (Souliane) since 2008.

sat_pubsub allows us to use our own up-to-date persistent pubsub service.

This article is composed of several sections

Why sat_pubsub ?

There are three main reasons to use sat_pubsub.

First, the XMPP servers come with variable pubsub support. Using an external component allows users to benefits constant features independent of the XMPP server or provider.

Secondly, sat_pubub is the only free (libre) implementation that support Pubsub MAM (Message Archive Management). It allows one to research a pubsub node. This feature is used to research blog post in the database.

Thirdly, sat_pubsub is used to test new functionalities like the ability to restrict blog post to specific group of contacts (think about circles on Google+). 1.

Let's install it !

This section describes the installation process of sat_pubsub in a python virtualenv. Unfortunately, I have encountered difficulties to install twisted, a dependency of SàT, in a virtualenv with pip because my production machine has no compiler. I managed to avoid the problem by installing python2-twisted (the distribution package) then the virtualenv was created with the option --system-site-packages to give it access to the system packages. Thereafter, it is possible to install sat_tmp, a small python module that monkey patch wokkel, a dependency of sat_pubsub.

$ sudo apt-get install python2-twisted
[...]
$ hg clone http://repos.goffi.org/sat_tmp
$ cd sat_tmp/
$ virtualenv env -ppython2.7 --system-site-packages
$ source env/bin/activate
(env)$  python2 setup.py install
(env)$ cd ../
(env)$ hg clone https://repos.goffi.org/sat_pubsub
(env)$ cd sat_pubsub
(env)$ python2 setup.py install

Setup the database

According to the official documentation, we need to create the database and install the SQL schema.

$ sudo -u postgres createuser -d -P `whoami`
$ createdb pubsub
$ psql pubsub < sat_pubsub/db/pubsub.sql

Configure Prosody

In order to use sat_pubsub, we need to declare it in our prosody config file. Once again, the information is available in the documentation. Two files need to be modified:

  • /etc/prosody/prosody.cfg.lua
[...]
modules_enabled = {
    [...]
    "delegation";
    "privilege";
}
[...]
  • /etc/prosody/conf.avail/your_domain.cfg.lua

This file defines the domain configuration of your XMPP server.

This is a copy of the configuration for the agayon server. The pubsub component is available at the address pubsub.agayon.be. The following configuration:

  • enables pubsub MAM,
  • uses the component for microblogging activity,
  • announces the pubsub nodes and items in disco requests,
  • lets the component access the roster and presence informations (used for PEP).
VirtualHost "agayon.be"
    enabled = true
    [...]
    privileged_entities = {

            ["pubsub.agayon.be"] = {
            roster = "get";
            message = "outgoing";
            presence = "roster";
            },
       }
    delegations = {
                ["urn:xmpp:mam:1"] = {
                        filtering = {"node"};
                        jid = "pubsub.agayon.be";
                },
                 ["http://jabber.org/protocol/pubsub"] = {
                        jid = "pubsub.agayon.be";
                 },
                 ["http://jabber.org/protocol/pubsub#owner"] = {
                        jid = "pubsub.agayon.be";
                 },
                 ["https://salut-a-toi/protocol/schema:0"] = {
                        jid = "pubsub.agayon.be";
                 },
                 ["http://jabber.org/protocol/disco#items:*"] = {
                        jid = "pubsub.agayon.be";
                 },
            }


Component "pubsub.agayon.be"
        component_secret = "shared_secret"
        modules_enabled = {"delegation", "privilege"}

Launch

(env)$ twistd  sat_pubsub  --jid=pubsub.agayon.be --secret=shared_secret --rhost=127.0.0.1 \
--rport=5347 --backend=pgsql --dbuser=user --dbpass=pass_psql --dbname=pubsub --dbhost=localhost --dbport=5432

Let's use it !

Once prosody is restarted, the component is accessible through any XMPP client. See the following example with jp.

$ jp info disco pubsub.agayon.be -c
Features

http://jabber.org/protocol/disco#items
http://jabber.org/protocol/pubsub#auto-create
http://jabber.org/protocol/pubsub#config-node
http://jabber.org/protocol/pubsub#create-nodes
http://jabber.org/protocol/pubsub#delete-any
http://jabber.org/protocol/pubsub#delete-nodes
http://jabber.org/protocol/pubsub#groupblog
http://jabber.org/protocol/pubsub#instant-nodes
http://jabber.org/protocol/pubsub#item-ids
http://jabber.org/protocol/pubsub#meta-data
http://jabber.org/protocol/pubsub#outcast-affiliation
http://jabber.org/protocol/pubsub#persistent-items
http://jabber.org/protocol/pubsub#publish
http://jabber.org/protocol/pubsub#publish-options
http://jabber.org/protocol/pubsub#publisher-affiliation
http://jabber.org/protocol/pubsub#purge-nodes
http://jabber.org/protocol/pubsub#retract-items
http://jabber.org/protocol/pubsub#retrieve-affiliations
http://jabber.org/protocol/pubsub#retrieve-default
http://jabber.org/protocol/pubsub#retrieve-items
http://jabber.org/protocol/pubsub#retrieve-subscriptions
http://jabber.org/protocol/pubsub#subscribe
https://salut-a-toi.org/spec/pubsub_admin:0
https://salut-a-toi/protocol/schema:0
jabber:iq:version
urn:xmpp:mam:2
urn:xmpp:order-by:0

Identities

┌───────┬─────────┬───────────────────────────┐
│catego │ type    │ name                      │
├┄┄┄┄┄┄┄┼┄┄┄┄┄┄┄┄┄┼┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┤
│pubsub │ service │ Salut à Toi pubsub service│
└───────┴─────────┴───────────────────────────┘

Items

┌─────────────────┬───────────────────┬─┐
│entity           │ node              │ │
├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┼┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┼┄┤
│pubsub.agayon.be │ be.agayon.errol:0 │ │
│pubsub.agayon.be │ tests             │ │
│pubsub.agayon.be │ Agayon            │ │
└─────────────────┴───────────────────┴─┘

The following commands are used to create a node, subscribe to it and edit an item (see the previous article).

$ jp pubsub node create -s pubsub.agayon.be -n node_name
$ jp pubsub subscribe -s pubsub.agayon.be -n node_name
$ jp pubsub edit -s pubsub.agayon.be -n node_name

The default text editor is then opened. It is possible to directly edit a XML file. This command is useful for debugging purpose.

About Agayon XMPP service

All the tests described in this blog are realized on the Agayon.be XMPP service. It is not opened for registration but having this field of experimentation is great to learn, practice and question the potential uses of XMPP. I use it with several accounts, depending on my use:

Most moderns XEPs are enabled in order to provide an up-to-date experience: PEP, Carbon, Stream Management, persistent pubsub, MAM, SRV records over TLS (useful to pass blocking WiFi accesses), etc.

Future of SàT

In a near future, the SàT project will make great announcements about new uses of pubsub. I hope this article will make you want to install sat_pubsub and experiment with nodes, data and notifications.

Stay tuned !

Links

  1. See the following blog post for more information.

by Arnaud at May 30, 2019 14:00

JP, a powerful command line interface for Salut-à-Toi

Salut à toi is a unique XMPP client. As its official description says, it's a "multipurpose, multi front-end, free (libre) and decentralized communication tool". It has been actively developed by Jérôme Poisson (Goffi) and Adrien Cossa (Souliane) since 2008. Today, I will focus on the use of "JP", a command-line interface. It can be used to send or receive files directly from a shell, pipe commands to or from XMPP, use XMPP easily in a script and of course play with pubsub nodes.

The following article describes uses of JP. Several of them are availabled in the trunk version of Salut-à-Toi.

Introduction

JP can be used to launch complexes commands in script, for debugging purpose or to explore XMPP services. JP is the command line interface and it connect to the daemon Salut-à-Toi (SàT). You can share the session between the front-ends:

  • JP of course.
  • Primitivus as a console front-end.
  • Livervia, a web based front-end.
  • Sententia an Emacs front-end (WIP).
  • Cagou an original mobile XMPP client based on SàT (WIP).

If you want to send a file easily, discover which services are available on a server, send messages in your scripts, manage your xmpp account, control video player, edit your blog post with your favorite editor, pipe streams, manage your pubsub nodes etc, JP is for you!

Usage

JP can be used as a command line tools or in a small shell environment.

First, we need to configure and launch the daemon sat.

$ sat

Your default profile will be connected. If you have no profile, JP can be used to create one.

$ jp profile create -j my_jid@example.org  -x mypassword profile_name
$ jp profile connect -p profile_name

Your password is saved in the sat settings. You can connect automatically with the option -c. It should be noted that SàT defines a default profile. It can be bypassed with the option -p. 1

$ echo "test message" | jp message send recipient@example.org -p profile_name -c

You can obtain help about a command with the option -h.

$ jp pubsub -h
usage: jp pubsub [-h]
                 {get,delete,edit,subscribe,unsubscribe,subscriptions,node,affiliations,search,hook,uri}
                 ...

positional arguments:
  {get,delete,edit,subscribe,unsubscribe,subscriptions,node,affiliations,search,hook,uri}
    get                 get pubsub item(s)
    delete              delete an item
    edit                edit an existing or new pubsub item
    subscribe           subscribe to a node
    unsubscribe         unsubscribe from a node
    subscriptions       retrieve all subscriptions on a service
    node                node handling
    affiliations        retrieve all affiliations on a service
    search              search items corresponding to filters
    hook                trigger action on Pubsub notifications
    uri                 build URI

optional arguments:
  -h, --help            show this help message and exit

JP is a Swiss army knife. Let's discover its possibilities through a few examples.

Examples

$ jp -h
usage: jp [-h] [--version]
          {file,input,uri,message,event,info,account,param,debug,ad-hoc,ticket,invitation,profile,shell,avatar,pipe,pubsub,bookmarks,roster,identity,blog}
          ...

This software is a command line tool for XMPP.
Get the latest version at http://salut-a-toi.org

optional arguments:
  -h, --help            show this help message and exit
  --version             show programʼs version number and exit

Available commands:
  {file,input,uri,message,event,info,account,param,debug,ad-hoc,ticket,invitation,profile,shell,avatar,pipe,pubsub,bookmarks,roster,identity,blog}
    file                File sending/receiving
    input               launch command with external input
    uri                 XMPP URI parsing/generation
    message             messages handling
    event               event management
    info                Get various pieces of information on entities
    account             XMPP account management
    param               Save/load parameters template
    debug               debugging tools
    ad-hoc              Ad-hoc commands
    ticket              tickets handling
    invitation          invitation of user(s) without XMPP account
    profile             profile commands
    shell               launch jp in shell (REPL) mode
    avatar              avatar uploading/retrieving
    pipe                stream piping through XMPP
    pubsub              PubSub nodes/items management
    bookmarks           manage bookmarks
    roster              Manage an entityʼs roster
    identity            identity management
    blog                blog/microblog management

Send a message

$ echo "Hello World" > filetest
$ jp message send recipient@example.org < filetest

or

$ echo "test  jp" | jp message send recipient@example.org

Send files

The following command allows to send the file file.txt to recipient@example.org.

$ jp file send file.txt recipient@example.org

Receive files

The following command allows to receive a file in the /tmp directory.

$ jp file receive -p agayon --path /tmp

Get information about a server

$ jp info version agayon.be
Client name: Prosody
Client version: 0.10.0
Operating System: Linux

Get disco information

Query a Server

$ jp info disco agayon.be
Features

http://jabber.org/protocol/commands
http://jabber.org/protocol/disco#info
http://jabber.org/protocol/disco#items
http://jabber.org/protocol/pubsub#auto-create
http://jabber.org/protocol/pubsub#config-node
http://jabber.org/protocol/pubsub#create-nodes
http://jabber.org/protocol/pubsub#delete-any
http://jabber.org/protocol/pubsub#delete-nodes
http://jabber.org/protocol/pubsub#groupblog
http://jabber.org/protocol/pubsub#instant-nodes
http://jabber.org/protocol/pubsub#item-ids
http://jabber.org/protocol/pubsub#meta-data
http://jabber.org/protocol/pubsub#outcast-affiliation
http://jabber.org/protocol/pubsub#persistent-items
http://jabber.org/protocol/pubsub#publish
http://jabber.org/protocol/pubsub#publish-options
http://jabber.org/protocol/pubsub#publisher-affiliation
http://jabber.org/protocol/pubsub#purge-nodes
http://jabber.org/protocol/pubsub#retract-items
http://jabber.org/protocol/pubsub#retrieve-affiliations
http://jabber.org/protocol/pubsub#retrieve-default
http://jabber.org/protocol/pubsub#retrieve-items
http://jabber.org/protocol/pubsub#retrieve-subscriptions
http://jabber.org/protocol/pubsub#subscribe
https://salut-a-toi.org/spec/pubsub_admin:0
https://salut-a-toi/protocol/schema:0
jabber:iq:auth
jabber:iq:last
jabber:iq:private
jabber:iq:roster
jabber:iq:time
jabber:iq:version
msgoffline
urn:xmpp:blocking
urn:xmpp:carbons:1
urn:xmpp:carbons:2
urn:xmpp:http:upload
urn:xmpp:http:upload:0
urn:xmpp:mam:0
urn:xmpp:order-by:0
urn:xmpp:ping
urn:xmpp:time
vcard-temp

Identities

┌───────┬──────┬─────────────────┐
│catego │ type │ name            │
├┄┄┄┄┄┄┄┼┄┄┄┄┄┄┼┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┤
│store  │ file │ HTTP File Upload│
│pubsub │ pep  │ Prosody         │
│server │ im   │ Prosody         │
└───────┴──────┴─────────────────┘

Extensions

urn:xmpp:http:upload
        type: text-single
        var: max-file-size
        value: 1048576
urn:xmpp:http:upload:0
        type: text-single
        var: max-file-size
        value: 1048576

Items

┌──────────────────────┬──┬────────────────────┐
│entity                │  │ name               │
├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┼┄┄┼┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┤
│test-pubsub.agayon.be │  │                    │
│upload.agayon.be      │  │                    │
│auth.agayon.be        │  │                    │
│proxy.agayon.be       │  │                    │
│pubsub.agayon.be      │  │                    │
│chat.agayon.be        │  │ Agayon.be chatrooms│
└──────────────────────┴──┴────────────────────┘

Query an account

$ jp info disco info@agayon.be
Items

┌───────────────┬───────────────────────────────────────────────────┬─┐
│entity         │ node                                              │ │
├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┼┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┼┄┤
│info@agayon.be │ eu.siacs.conversations.axolotl.bundles:1948395845 │ │
│info@agayon.be │ http://jabber.org/protocol/activity               │ │
│info@agayon.be │ http://jabber.org/protocol/mood                   │ │
│info@agayon.be │ eu.siacs.conversations.axolotl.devicelist         │ │
│info@agayon.be │ storage:bookmarks                                 │ │
└───────────────┴───────────────────────────────────────────────────┴─┘

Manage pubsub nodes

The following examples show how to manage pubsub nodes on the service pubsub.agayon.be.

Create a node

$ jp pubsub node create -s pubsub.agayon.be -n node_name

Subscribe to a node

$ jp pubsub subscribe -s pubsub.agayon.be -n node_name

Edit

The edit command allows to edit an item under a node.

$ jp pubsub edit -s pubsub.agayon.be -n node_name

The default text editor is opened. It is possible to directly edit a XML file. This command is useful for debugging purpose.

If you want to edit a post without having to edit xml directly, use jp blog edit.

As an example, you can try to edit the following xml file.

<entry xmlns="http://www.w3.org/2005/Atom">
  <author>
    <name>author@example.com</name>
  </author>
  <generator>JP (SàT)</generator>
  <title>I am a pubsub post !</title>
  <content>This is the content of this great post.</content>
</entry>

Once the file is saved, a notification appears in Gajim. For now, the name of the post is a hash.

pubsub post on Gajim

pubsub post on Gajim

Manage your XMPP Blog

It is possible to manage a post with XMPP. It is based on PEP, a simplified version of pubsub. If your server support PEP, JP can help you to manage this blog easily.

Publish a post

First, you need to define your preferred syntax. In this example, I select markdown. This option can also be set in any other frontend (e.g. Primitivus). Whereafter you can edit a new post with the syntax jp blog edit.

$ jp param set Composition Syntax markdown -p agayon
$ jp blog edit

Your favorite editor open and you can edit your blog post with markdown syntax. When you save and close it, another file open. You can edit your settings:

{
    "allow_comments": true,
}

You can verify the accessibility of your post with the following command:

$ jp blog get 
content: 
Great test !
Make testing great again !
Yuuuuuuge publication.

It is also possible to modify your last blog post simply with jp blog edit --last-item.

Your blog post are also visible on other clients like Movim (see below) and interfaces like Livervia.

Post on movim

Use jp shell:

In order to ease debugging of services, JP comes with a shell interface. You only need to launch jp shell. You can obtain help by typing ?.

$ jp shell
cmd pubsub
pubsub> ?
Shell commands:
Documented commands (type help <topic>):
========================================
cmd    do    help  shell  use_clear  version
debug  exit  quit  use    verbose    whoami 

Action commands:
positional arguments:
  {get,delete,edit,subscribe,unsubscribe,subscriptions,node,affiliations,search,hook,uri}
    get                 get pubsub item(s)
    delete              delete an item
    edit                edit an existing or new pubsub item
    subscribe           subscribe to a node
    unsubscribe         unsubscribe from a node
    subscriptions       retrieve all subscriptions on a service
    node                node handling
    affiliations        retrieve all affiliations on a service
    search              search items corresponding to filters
    hook                trigger action on Pubsub notifications
    uri                 build URI

Select a command

> cmd pubsub
pubsub> use node urn:xmpp:microblog:0

Navigate into commands

> cmd bookmarks/list
bookmarks/list> -c
... CMD result
bookmarks/list> cmd ..
bookmarks> cmd
>

Example: List bookmarks

$  jp shell
> cmd bookmarks
bookmarks> cmd list
bookmarks/list> ?
Shell commands:
Documented commands (type help <topic>):
========================================
cmd    do    help  shell  use_clear  version
debug  exit  quit  use    verbose    whoami 

Action commands:
optional arguments:
  -h, --help            show this help message and exit
  -p PROFILE, --profile PROFILE
                        Use PROFILE profile key (default: @DEFAULT@)
  --pwd PASSWORD        Password used to connect profile, if necessary
  -c, --connect         Connect the profile before doing anything else
  -l {all,local,private,pubsub}, --location {all,local,private,pubsub}
                        storage location (default: all)
  -t {muc,url}, --type {muc,url}
                        bookmarks type (default: muc)
bookmarks/list> -t muc
private:
    Movim [movim@conference.movim.eu] (*)
    Archlinux - blah blah [archlinux-fr@chat.jabberfr.org]
    bot [bot@chat.agayon.be] (*)

Example: get disco information

> cmd info
info> use jid pubsub.agayon.be 
info> disco
Features

http://jabber.org/protocol/disco#info
http://jabber.org/protocol/disco#items
http://jabber.org/protocol/pubsub
http://jabber.org/protocol/pubsub#create-nodes
http://jabber.org/protocol/pubsub#delete-items
http://jabber.org/protocol/pubsub#delete-nodes
http://jabber.org/protocol/pubsub#instant-nodes
http://jabber.org/protocol/pubsub#item-ids
http://jabber.org/protocol/pubsub#publish
http://jabber.org/protocol/pubsub#publisher-affiliation
http://jabber.org/protocol/pubsub#purge-nodes
http://jabber.org/protocol/pubsub#retract-items
http://jabber.org/protocol/pubsub#retrieve-items
http://jabber.org/protocol/pubsub#retrieve-subscriptions
http://jabber.org/protocol/pubsub#subscribe

Identities

┌───────┬─────────┬───────────────────────┐
│catego │ type    │ name                  │
├┄┄┄┄┄┄┄┼┄┄┄┄┄┄┄┄┄┼┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┤
│pubsub │ service │ Prosody PubSub Service│
└───────┴─────────┴───────────────────────┘

More

Some alternatives

SleekXMPP

SleekXMPP is a xmpp python library. It provide two examples of pubsub clients. The first, pubsub_client.py can manage nodes and the second one, pubsub_events.py, can manage events.

./pubsub_client.py -j jid --password=passwd pubsub.agayon.be create node_name

Gajim

Gajim is a great desktop client. It can be used to post on pubsub nodes too.

pubsub post on Gajim

Movim

Movim is a social network platform based on XMPP. It massively use pubsub to allow users to interact.

Conclusions

Salut-à-Toi is a great XMPP client. JP, its command-line interface, is a powerful tool. It can be useful to debug services, pubsub nodes and play with 'social' capacities of XMPP. It is actively developed and new features are coming. We will be soon able to make todolist or fill a bug ticket on a XMPP based system.

Links

  1. It is possible to use jp profile modify to set the default profile.

by Arnaud at May 30, 2019 14:00