At first glance, it may sound absurd. Here we have technical debt, a purely engineering problem, as technical as it can get, and another area, HR, dealing with psychology and emotions, put into one sentence. Is it possible that they are closely related? Let’s take it apart and see.

Exploring technical debt

What is technical debt, anyway? A tongue-in-cheek definition is that it is code written by someone else. But there is more to it – it is code written years ago, possibly by someone who has left the company. Also, every major piece of software is written incrementally over many years. Even if it started with a very detailed, well-thought-through design, there inevitably came plenty of modifications and additions which the original design could not easily accommodate.

Your predecessors sometimes had to cut corners and bend over backwards to achieve desired results in an acceptable amount of time. Then they moved on, someone else took over and so on.

What you now have is a tangled mess, mixing coding styles, techniques and design patterns, with an extra addition of ad-hoc solutions and hacks. You see a docstring like “temporary solution, TODO: clean up”, you run git blame and it is seven years old. It happens to everybody.

The business behind technical debt

Technical debt is a business issue. You can read about it in more detail in our previous post.

Source: Medium

The daily tasks of most developers are fixing bugs and implementing new features in existing code. The more messy and convoluted the code is, the more time it takes every time one has to read it and reason about it. And it is real money: according to McKinsey Report, this burden amounts to 20%-40% of an average business’ technology stack. Engineers are estimated to spend up to 50% of their time struggling with it.

So what can businesses do to get their code in check? Here are some suggestions:

• Taking a step back 
• Reassessing the architecture and techniques 
• Making more informed choices 
• Rewriting parts of the code to make it consistent and understandable, removing unused code and duplications

Unfortunately, this is very rarely done, since it does not bring any visible improvements to the product – clients are not interested in code quality, they want software that does its job. Improving the code costs real money, while the increase in developer productivity is impossible to quantify.

Technical debt also has another property – it is annoying. And this brings us nicely to the second topic.

Happy HR, Happier devs

What is HR about? In part, it is about the well-being of employees. Every employer wants good people to stay in the company. The most valuable employee is someone who likes their job and feels good about the place. HR departments go to great lengths to achieve this.

But, you can buy new chairs and phones, decorate the office, buy pizza, organise board games evenings – all this is mostly wasted if the following morning your devs show up in their workplace only to say “Oh no, not this old cruft again”, embellishing that statement with a substantial amount of profanities.

Now I tell you this: Nothing makes developers happier than allowing them to address their pain points. Ask them what they hate the most about the codebase and let them improve it, the way they choose to, at their own pace. They will be delighted.

You may ask how I know. Firstly, I’m a dev myself. Secondly, I’m fortunate enough to be currently working for a company that took the steps and did exactly that:

Step 1: Set up a small “tech debt” team

Step 2: Collected improvement proposals from all developers

Step 3: Documented them

Step 4: Defined priorities

Currently, the technical debt team or the proposers themselves are gradually putting these proposals into action, one by one. The code is getting better. We are becoming more productive. And if we’re happy, isn’t HR?

Calling upon the compassionate and proactive HR professionals out there: talk to your CTOs, tell them you all are after the same thing – you want these frustrated, burned-out coders happy, enthusiastic and more productive, and that you have an idea of how to achieve this.

Chances are they will be interested.

The post Technical debt and HR – what do they have in common? appeared first on Erlang Solutions.

The ejabberd Docs website did just get a major rework: new content management system, reorganized navigation, improved markdown, and several improvements!

Brief documentation timeline

ejabberd started in November 2002 (see a timeline in the ejabberd turns 20 blog post). And the first documentation was published in January 2003, using LaTeX, see Ejabberd Installation and Operation Guide. That was one single file, hosted in the ejabberd CVS source code repository, and was available as a single HTML file and a PDF.

As the project grew and got more content, in 2015 the documentation was converted from LaTeX to Markdown, moved from ejabberd repository to a dedicated docs.ejabberd.im git repository, and published using a Go HTTP server in docs.ejabberd.im, see an archived ejabberd Docs site.

New ejabberd Docs site

Now the ejabberd documentation has moved to MkDocs+Material, and this brings several changes and improvements:

Site and Web Server:

• Replaced Go site with MkDocs
• Material theme for great features and visual appeal, including light/dark color schemes
• Still written in Markdown, but now using several MkDocs, Material and Python-Markdown extensions
• The online site is built by GitHub Actions and hosted in Pages, with smaller
  automatic deployment time
• Offline reading: the ejabberd Docs site can be downloaded as a PDF or zipped HTML, see the links in home page

Navigation

• Major navigation reorganization, keeping URLs intact so old links still work (only Install got some relevant URL changes)
• Install section is split into several sections: Containers, Binaries, Compile, …
• Reorganized the Archive section, and now it includes the corresponding Upgrade notes
• Several markdown files from the ejabberd and docker-ejabberd repositories are now incorporated here

Content

• Many markdown visual improvements, specially in code snippets
• Options and commands that were modified in the last release will show a mark, see for example API Reference
• Version annotations are shown after the corresponding title, see for example sql_flags
• Modules can have version annotations, see for example mod_matrix_gw
• Links to modules, options and API now use the real name with _ character instead of - (compare old #auth-opts with #auth_opts). The old links are still supported, no broken links.
• Listen Modules section is now better organized
• New experimental ejabberd Developer Livebook

So, please check the revamped ejabberd Docs site, and head to docs.ejabberd.im git repository to report problems and propose improvements.

The age plugin for Apple’s Secure Enclave is a small, platform-independent Swift CLI app, built using the Swift Package Manager. Because it does not use Xcode for building, you can’t use the Xcode IDE for collecting and browsing test coverage of your source files. I therefore wrote a small (self-contained, dependencyless, cross-platform) Swift script to transform SwiftPM’s raw coverage data into annotated source code, together with an SVG badge to put on your project page.

Welcome to part three of our ‘Making Sense of Blockchain’ blog post series. Here we’ll explore how our attitudes to ownership are changing and how this relates to the value we attach to digital assets in the blockchain space. You can check out ‘Innovating with Erlang and Elixir’ here if you missed part two of the series.

Digital Assets: Ownership in the era of Blockchain

While physical goods contain an abstract element: the design, the capacity to model, package and make it appealing to the owners or consumers. Digital assets have a far stronger element of abstraction which defines their value. In contrast, their physical element is often negligible and replaceable (e.g. software can be stored on disk, transferred or printed). These types of assets typically stimulate our intellect and imagination.

Digital goods have a unique quality in that they can be duplicated effortlessly and inexpensively. They can exist in multiple forms across various platforms, thanks to the simple way we store them using binary code. They can be recreated endlessly from identical copies. This is a feature that dramatically influences how we value digital assets.  Because replicas are so easy to make, their copies or representations don’t hold value, but the original digital creation itself. This principle is a cornerstone of blockchain technology, with its hash lock feature safeguarding the integrity of digital assets.

If digital items are used correctly, the capacity to clone a digital item can increase confidence that it will exist indefinitely, which keeps its value steady. However, the immutable and perpetual existence of digital goods isn’t guaranteed forever.

They are dependent on a physical medium (e.g. hard disk storage), that could be potentially altered, degraded or become obsolete over time. 

A blockchain, like the one used in the Bitcoin network, is a way to replicate and reinforce digital information via Distributed Ledger Technology (DLT). 

An example of the DLT network

Distributed Ledger Technology lets users record, share, and synchronise data and transactions across a distributed network comprising of many participants. 

It includes mechanisms to repair issues, should data become corrupted due to hard disk failure or a malicious attack.

However, as genetic evolution suggests, clones with the same characteristics can all die out by the introduction of an actor that makes the environment unfit for survival. So it might be sensible to introduce different types of ledgers to keep data safe on various physical platforms, increasing the likelihood of survival of information.

The evolution of services and their automation

Now let’s consider how we have started to attach value to services and how we are becoming increasingly demanding about their performance and quality.

Services are a type of abstract value often traded on the market. They involve actions defined by contracts that lead to some kind of change. This change can apply to physical goods, digital assets, and other services themselves or people. What we trade is the potential to exercise a transformation, which in some instances might have been applied already. For example, a refined product like oil that has already been changed from its original raw state.

As transformations become more automated and the human element decreases,  services are gradually taking the shape of automated algorithms, which are yet another form of digital assets. Take smart contracts, for example, a rapid-growth industry projected to grow from USD 1.9 Billion in 2023 to USD 9.2 Billion by 2032, according to Market Research Future.

Smart Contracts Market Projection Overview

But it’s important to state that an algorithm alone isn’t enough to apply digital transformation, we also require an executor, like a physical or virtual machine.

Sustainability and access to resources

Stimulation of the intellect and/or imagination isn’t the only motivator that explains the increasing interest in digital goods and ultimately their rising market value. Physical goods are known to be quite expensive to handle. To create, trade, own and preserve them, there is a significant expenditure required for storage, transport, insurance, maintenance, extraction of raw materials etc.

There’s a competitive and environmental cost involved, making obtaining physical resources inherently difficult to scale and sometimes costly- especially in densely populated urban areas. As a result, people are motivated to possess and exchange digital goods and services.

The high power consumption required by the Bitcoin network’s method of consensus would potentially negate these environmental benefits. Although power consumption is a concern, it should be remembered that blockchain technology can act as a force for good, being used for environmentally beneficial projects. 

A great example is the work being done by dClimate, a decentralised climate information ecosystem making it easier for businesses to find and utilise essential environmental information that could impact their sector. These important decisions in turn provide information on: 

• Where businesses can build infrastructure
• Where they can manage water resources
• How businesses can protect vulnerable communities

However, some of these activities (such as those requiring non-physical effort, like stock market trading, and legal or accounting services) are best suited for significant cost reduction through algorithmic automation  (assuming that the high carbon footprint required to drive the ‘Proof of Work’ consensus mechanism used in many DLT ecosystems can be avoided).

Barriers to acceptance of digital assets

While it is sensible to forecast a significant expansion of the digital assets market in the coming years, it is also true that, at present, there are still many psychological barriers to overcome to get broader traction in the market.

The main challenge relates to trust. A buyer wants some assurance that traded assets are genuine and that the seller owns them or acts on behalf of the owner. DLT provides a solid way to work out the history of a registered item without, interrogating a centralised trusted entity. Provenance and ownership are inferable and verifiable from several replicated ledgers while block sequences can help ensure there is no double spending or double sale taking place within a certain time frame.

Another challenge is linked to the meaning of ownership outside of the context of a specific market. Take the closure of Microsoft’s ebook store. Microsoft’s decision to pull out of the ebook market, presumably motivated by a lack of profit, could have an impact on all ebook purchases that were made on that platform. The perception of the customer was obviously that owning an ebook was the same as owning a physical book. 

What Microsoft might have contractually agreed through its End-User License Agreement (EULA), however, is that this is true only within the contextual existence of its platform.

There is a push, in this sense, towards forms of ownership that can break out from the restrictions of a specific market and be maintained in a broader context. Blockchain’s DLT in conjunction with smart contracts, that exist potentially indefinitely, can be used to serve this purpose allowing people to effectively retain their digital items’ use across multiple applications.

The transition to these new notions of ownership is particularly demanding when it comes to digital non-fungible assets. Meanwhile, embracing fungible assets, such as cryptocurrency, has been somewhat easier for customers who are already used to relating to financial instruments. 

This is probably because fungible assets serve the unique function of paying for something, while in the case of non-fungible assets, there is a range of functions that define their meaning in the digital or physical space.

What this will mean for blockchain adopters

In discussing the major emerging innovation that blockchain technology has influenced dramatically over the last few years, the ownership of digital assets, it is clear that what we are witnessing is a new era that is likely to revolutionise the perception of ownership and reliance on trusted and trustless forms of automation. This is driven by the need to increase interoperability, cost compression, sustainability, performance and customisation. For any business size in any industry, we’re ready to investigate, build and deploy your blockchain-based project on time and to budget. Let us know about your blockchain project here.

The post Blockchain Tech Deep Dive| Meaning of Ownership appeared first on Erlang Solutions.

Radically Open Security (ROS) kindly performed a security audit of some parts of Monal.
Specifically they audited the usage of our XML query language and the implementations of SASL2, SCRAM and SSDP.

The results in a nutshell: no security issues found, read the full report here: Monal IM penetration test report 2024 1.0 .

We initially shared this news on our social media page, thinking this was a temporary issue. But we’ve had no response from Google for several days, and want to explain the situation in more detail.

Update 16th April: Over a week after this began, Google have reinstated the Snikket app on the Play Store and everything works again. Thanks to everyone who gave us encouragement and support during this time! Feel free to read on for details of what happened.

Summary

We merged some changes from our upstream project, Conversations, and we submitted the new version to Google for review. Before responding, they removed the existing published version from the store. We have submitted a new version (on 10th April) that we believe should satisfy Google, but they have not yet published it or provided any feedback.

This means that it’s not currently possible for Android users to install the app using Google Play. We recommend that you install it via F-Droid instead.

Workaround for Android users

If you receive an invitation to Snikket, the Play Store link in the invitation will not work. The best course of action is to install the app using an open-source marketplace instead: F-Droid.

1. Follow the instructions on f-droid.org to download and install F-Droid.
2. Install Snikket using F-Droid.
3. After the Snikket app is installed, open your Snikket invitation link again.
4. Tap the ‘Open the app’ button.
5. Follow the Snikket app’s instructions to set up your new Snikket account.

The full story

I’m Matthew, founder of Snikket and lead developer. This is the story of how we arrived at this situation with Google.

It all began when…

A few months ago, Snikket, along with a number of other XMPP apps, found our updates rejected by Google’s review team, claiming that because we upload the address book entries of users to our servers, we need a “prominent disclosure” of this within the app. The problem is… we don’t upload the user’s address book anywhere!

The app requests permission to read the address book. Granting this permission is optional, and the reason is explained before the permission is requested. If you grant the permission, the app has a local-only (no upload!) feature that allows you to “link” your XMPP contacts with your phone address book contacts, allowing you to unify things like contact photos. Contact information from your address book is never uploaded.

Many messaging apps, such as WhatsApp, Signal, and others, request access to your address book so they can upload them to their servers and determine who else you know that is using their service. Google have decided that’s what we’re doing, and they won’t accept any evidence that we’re not.

We don’t have telemetry in our app, but we assumed that this feature is probably not used by most people, so we decided to remove it from the Play Store version of the app rather than continue fighting with Google.

Amusingly, Google also rejected the update that removed the ‘READ_CONTACTS’ permission. Multiple times. It took an appeal before they revealed that they were rejecting the new version it because one of the beta tracks still had an older version with the READ_CONTACTS permission. Weird.

I fixed that, and submitted again. They rejected it again. This time they said that they required a test login for the app. Funny, because we already provided one long ago. I assumed the old test account was no longer working, so I made them a new one and resubmitted the app. They rejected it again with the same reason - saying we had not provided valid test account credentials.

“You didn’t provide an active demo/guest account or a valid username and password which we need to access your app.” – Google reviewers

The weird thing was, when I logged in to that account to test it, I saw that they had logged in and even sent some messages. So they were lying?!

We submitted an appeal with all the evidence that the account was working, and their reviewers had even logged in and used it successfully. After some time, they eventually responded that they wanted a second test account. Why couldn’t they just say that in the first place?!

After adding credentials for a second account, and using the Snikket circles features to ensure they could find each other easily, we resubmitted.

Rejected again.

This time the rejection reason was really the best one so far: they claimed the app was unable to send or receive messages. Rather funny for a messaging app that thousands of people use to send and receive messages daily.

Wait, a messaging app that can’t send messages?

Once again, I logged into the test account we had provided to Google, and once again saw that they had successfully exchanged messages between their two test accounts. We submitted another appeal, with evidence.

Eventually they responded, clarifying that their complaint was specifically about the app when used with Android Auto, their smart car integration. I do not have such a car, and couldn’t find any contributor who had, but I found that Google provide an emulator that can run on a PC, so I set that up on my laptop and proceeded to test.

You won’t be surprised to learn at this point that the messaging functionality worked fine. We responded to the appeal, including a screencast I made of the messaging functionality working with Android Auto. They informed us that they were “unable to assist with the implementation” of their policies. Then at the end of their response, suggested that if we think the app is compliant, that we should resubmit it for review.

So we resubmitted the app, which by this point had already been rejected 7 times. We resubmitted it with no modification at all. We resubmitted the version they rejected. They emailed us later that day to say it was live.

How would I rate the developer experience of publishing an app with Google Play? An unsurprising 1 star out of 5. If I could give zero, I would.

The removal

But this was all a couple of months ago. Everything was fine. Until I merged some of the nice things Daniel has been working on recently in Conversations, the app upon which Snikket Android is based. We put the new version out for beta testing and everything was going fine - the app passed review, and a few weeks later with no major issues reported, we pushed the button to promote the new version from beta to live on the store.

On the 8th April we received an email from Google with the subject line:

“Action Required: Your app is not compliant with Google Play Policies (Snikket)”

I was ill this day, and barely working. For reasons that, if you have read this far, you will hopefully understand, I decided to take up this fight when I was feeling better. Confusingly, a couple of days later we received another email with the same subject. At this point I realised with horror that the first email was not about the new update - they had reviewed the current published version and decided to remove it entirely from the store.

With Snikket unavailable, anyone trying to add a new Android user to their Snikket instance (whether hosted or self-hosted) is going to have a hard time. This is not good.

Their complaint was that the privacy policy was not prominent enough within the app. They had previously hit Conversations with the same thing. Daniel had already put a link to the privacy policy in the main menu of that app and this was already in the update waiting for their review. They didn’t reject the update until a couple of days later, and for a different reason.

Unknown to me, Daniel had tried to re-add the ‘READ_CONTACTS’ permission to Conversations, hoping that with the new privacy policy link and other disclaimers in place, that would be enough. They had already rejected that, and he had removed the permission again. But he did this after I had already started testing the new beta release of Snikket. The order of events went something like this:

• Daniel experimentally re-adds READ_CONTACTS permission to Conversations
• I merge Conversations changes into Snikket, and begin beta testing
• Conversations update gets rejected due to the permission, and Daniel reverts the READ_CONTACTS change
• Without knowing of the Conversations rejection, I promote the Snikket beta to the store.
• Google rejects the Snikket update

What’s interesting is that Google rejected only on the permission change. The contacts integration itself was still disabled in Snikket. This is strong evidence that Google just assumes that if you have the permission (and presumably network permission too) then of course you must be uploading the user’s contacts somewhere.

As soon as I realised the problem, I merged the new changes from Conversations and rushed a new upload to Google Play. However at the time of writing this, several days later, Snikket remains unavailable in the store and no feedback has been received from Google.

This is an unsustainable situation

During this period we have had multiple people sign up for hosted Snikket instances, and then cancel shortly after. This is almost certainly because a vital step of the onboarding process (installing the app) is currently broken. This is providing a bad experience for our users and customers, negatively affecting the project’s reputation and income.

We are grateful that alternatives such as F-Droid exist, and allow people access to open-source apps via a transparent process and without the tyranny of Google and their faceless unaccountable review team. We need to ensure these projects are supported, and continue to improve their functionality, usability and user awareness.

Finally, we also welcome the efforts that the EU has been working on with things like the Digital Markets Act, to help break up the control that Google’s (demonstrably) arbitrary review process has over the the success and failure of projects, and the livelihoods of app developers.

Google, are you there?

We’re back with the latest in our Blockchain series, where we explore in-depth In our first post, we explored the Six Key Principles of Blockchain. 

In our latest post, we’re making the case for using Erlang,Elixir and the BEAMVM to power your blockchain project.

Blockchain and business needs

Building a robust and scalable blockchain presents many challenges that a research and development team typically needs to address. The often ambitious goals to drive decentralised consensus and governance require unconventional approaches to achieve extra performance and reliability.

Improved Transactions per Second (TPS) is the most common challenge that blockchain-related use cases expose. TPS as the name suggests, is a metric that indicates how quickly a network can execute transactions per second. It is inherently difficult to produce a distributed peer-to-peer (P2P) network that can register transactions into a single data structure.

Guaranteeing consensus while delivering high TPS throughput among a vast number of nodes active on the network is even more challenging. Also, the fact that most public blockchains need to operate in a non-trusted mode requires adequate mechanisms for validation, which implies that contextual data needs to be available on demand. A blockchain should also be able to respond to anomalies such as network connectivity loss, node failure and malicious actors.

All of the above is further complicated by the continuous growth of the blockchain data structure itself, which becomes problematic in terms of storage.

It is clear that, unless businesses are prepared to invest vast amounts of resources, they would benefit from a high-level language and technology that allows them to quickly prototype and amend their code.

The ideal technology should also:

• Offer a strong network and concurrent capabilities
• Have technology built with distribution in mind 
• Offer a friendly paradigm for asynchronous computation
• Not collapse under heavy load
• Deliver when traffic exceeds capacity

The Erlang Beam VM (available also in the Elixir syntax) undoubtedly scores high on the above list of desirable features.

Erlang & Elixir’s strengths for blockchain

Fast development

The challenge: Blockchain technology is present in extremely competitive markets. According to Grandview Marketing Analysis report, The global blockchain technology market size was valued at USD 17.46 billion in 2023 and is expected to grow at a compound annual growth rate (CAGR) of 87.7% from 2023 to 2030. 

Grandview Marketing Analysis report

It is critical for organisations operating in them to be able to release new features in time to attract new customers and retain existing ones.

The answer: Both Erlang and Elixir are functional languages, operating at a very high level of abstraction which is ideal for fast prototyping and development of new ideas. By using these languages on top of the Beam VM, developers dramatically increase the speed to market when compared to other lower-level or object-oriented technologies.

Solutions developed in Erlang or Elixir also lead to a significantly smaller code base, which is easier to test and adapt to changes of direction. This is helpful when you proceed to fast prototyping new solutions and when you discover that amendments and upgrades are necessary, which is very typical in blockchain R&D activity. Both languages offer support for unit testing in their standard library. This enables developers to adopt Test Driven approaches ensuring the quality is preserved when modules and libraries get refactored. The common test framework also provides support for distributed tests and can be integrated with Continuous Integration frameworks like Jenkins. Both Erlang and Elixir shells let the programmer flesh out ideas fast and access running nodes for inspection.

Introspection

The challenge: To keep a competitive advantage in a fast-changing market, it is critical for organisations to promptly identify issues and opportunities by extracting relevant information about their running systems so that actions can be taken to upgrade them where appropriate.

The answer: Erlang and Elixir allow connection to an already running system and a status check. This is an extremely useful debugging tool, both in the development and production environment. Statuses of processes can be checked, and deadlocks in the live system can be analysed. Some various metrics and tools can show overload, bottlenecks and other key performance indicators. Enhanced introspection tools such as Erlang Solutions’ Wombat OAM are also helping with the identification of scalability issues when you run performance tests.

Networking

The challenge: Delivering a highly scalable and distributed P2P network is critical for blockchain enterprises. It is important to rely on stable battle-proven network libraries as reliable building blocks for exploiting use case-specific innovative approaches.

The answer: Erlang and Elixir come with strong and easy-to-manage network capabilities. There is a proven record of successful enterprises that rely on the BEAM VM’s networking strengths; including Adroll, WhatsApp, Bleacher Report, Klarna, Bet365 and Ericsson. Most of their use cases have strong analogies with the P2P networking that is required to deliver a distributed blockchain.

Combined with massive concurrency, the networking makes Erlang and Elixir ideal for server applications and means it can handle many clients. The binary and bitstring syntax makes parsing binary protocols particularly easy.

Massively concurrent

The challenge: There is a weakness afflicting Bitcoin and Ethereum where the computation of a block is competitive rather than collaborative. There is the opportunity to drive a collaborative concurrent approach: i.e. via sharding so that each actor can compute a portion of a block.

The answer: The BEAM VM powering Erlang and Elixir provides lightweight processes for applications. These are lightweight so that hundreds of thousands of them can run simultaneously. These processes do not share memory, communication is done over asynchronous messages (unlike goroutines) so there’s no need to synchronise them. The BEAM VM implementation also makes use of all of the available CPUs and cores. This makes Erlang and Elixir ideal for workloads that involve a huge amount of concurrency and consist of mostly independent workflows. This feature is especially useful in addressing the coordinated distribution of portions of work to compute a Merkle Tree of transactions.

High availability and resilience

The challenge: These are the requirements for every type of application and even more so for competitive and highly distributed blockchain networks. The communication and preservation of a state need to be as available as possible to avoid inconsistent states, network forks and disruptions experienced by the users.

The answer: The fault tolerance properties mentioned in the previous paragraph combined with built-in distribution leads to high availability even in cases of hardware malfunction. Erlang and Elixir have the built-in mnesia database system with the ability to replicate data over a cluster of nodes so if one node goes down, the state of the system is not lost.

Erlang and Elixir provide the supervisor pattern to handle errors.

An example of a Supervision Tree, used to build a hierarchical process structure 

Computing is done in multiple processes and if an error occurs and a process crashes, the supervisor is there to handle the situation, restart the computing or take some other measures. This pattern lets the actual code be cleaned as error handling can be implemented elsewhere. As processes are isolated, they do not share memory meaning errors are localised.

Built-in distribution

The challenge: This is highly relevant for trusted or hybrid networks where a central network takes authoritative decisions on top of a broader P2P network. Using the “out of the box” Erlang distribution and proven consistency approaches such as RAFT can be a quick win towards a fast prototype of a blockchain solution.

The answer: Erlang and Elixir provide out-of-the-box tools to run a distributed system. The message-handling functionalities of the system are transparent, so sending a message to a remote process is just as simple as sending it to a local process. There’s no need for convoluted IDLs, naming services or brokers.

Safety and Security

The challenge: Among the security features that both trusted and untrusted blockchain solutions strongly require, is the critical protection of access to the state memory, therefore reducing the exposure to a range of vulnerabilities.

The answer: Erlang and Elixir, just like many high-level languages, do not manipulate memory directly so applications written in Erlang and Elixir are effectively immune to many vulnerabilities like buffer overflows and return-oriented programming. Exploiting these vulnerabilities would require direct access to the memory, but the BEAM VM hides the memory from the application code.

While many business leaders are still trying to figure out how to put the technology to work for maximum ROI, most agree on two things:

1. Blockchain unlocks vast value potential from optimised business operations.
2. It’s here to stay.

Unlocking the potential of technology

Talking about blockchain implementation is no longer merely food for thought. Organisations should keep an eye on developments in blockchain tech and start planning how to best use this transformative technology, to unleash trapped value in key operational processes.

It’s clear – blockchain should be on every company’s agenda, regardless of industry.

If you want to start a conversation about engaging us for your project. Just drop the team a line.

The post Blockchain Tech Deep Dive | Innovating with Erlang and Elixir appeared first on Erlang Solutions.

XMPP Newsletter Banner

The 50th release of the XMPP Newsletter!

This is the 50th release of the XMPP Newsletter since it started in February 2019. We think it is worth to celebrate this achievement and say thanks to all the contributors as well as all our readers! Back at the Summit in Brussels, JC Brand, Nicolas Vérité (Nyco) and Severino Ferrer (S0ul) proposed and initiated the XMPP Newsletter. Since then almost every month there has been a release full of news from the XMPP universe. Hence, we are looking forward to the next releases and invite you to participate in this community effort! We would love to see more contributors as well as more translations of the XMPP Newsletter. Read more about how to help below.

That being said, welcome to the 50th edition of the XMPP Newsletter, great to have you here again! This issue covers the month of March 2024.

XSF Announcements

Welcome to our reapplicants and new members in Q1 2024! If you are interested to join the XMPP Standards Foundation as member, please apply now.

XMPP and Google Summer of Code 2024

The XSF has been accepted as a hosting organisation at GSoC in 2024 again! Currently XMPP project mentors are reviewing the proposals. GSoC project ideas from XMPP-related projects are:

• Monal
  • Modern Onboarding (90 hours, easy)
  • Media Gallery (90 hours, easy)
  • MDM support (175 hours, easy)
  • SiriKit support (350 hours, medium)
• Dino
  • Inline link preview (175 hours, easy)
  • Rich message support (175 hours, easy)
• Prav.app
  • Standards compliant SMS OTP based authentication (350 hours, medium)

XSF and Google Summer of Code 2024

XSF Fiscal Hosting Projects

The XSF offers fiscal hosting for XMPP projects. Please apply via Open Collective. For more information, see the announcement blog post. Current projects you can support:

• Mellium Co-op
• Prav iOS

XMPP Events

• XMPP Italian happy hour [IT]: monthly Italian XMPP web meeting, every third Monday of the month at 7:00 PM local time (online event, with web meeting mode and live streaming).

Talks

• Last November the Linux Inlaws Podcast hosted Edward and Matt from the XSF Board 2024. The episode has been published recently and can be accessed via archive.org.

Articles

Axel Reimer published an XMPP app decision table. It compares typical instant messaging features of popular XMPP apps and might be a guideline for new users who want to try an XMPP app. The table is available in English and German.

You can now ask all your questions about XMPP Providers in the new support chat. That chat is hosted on an own XMPP server which is set up with the new XMPP Providers Server project. It is a simple, Ansible-based all-in-one server setup that can be used for your own XMPP server as well.

Snikket Hosting is now publicly available! The launch is about providing new ways to get started with Snikket, not replacing the options that are already available. If you are already self-hosting Snikket, or planning to, nothing is changing for you. Though please do donate to support the project, even a little helps!

European Union news:

• Designated gatekeepers must now comply with all obligations under the Digital Markets Act
• Interoperable Europe act: Council adopts new law for more efficient digital public services across the EU

Software News

Clients and Applications

• The JMP newsletter introduces the JMP eSIM Adapter, allowing users to use eSIMs from any provider in any device. It explains how to download and write eSIMs to the adapter, including using an open source Android app or PC/SC reader. Cheogram Android also receives an update with new features and a visual refresh. There’s a clarification on contact integration, with Google now requiring the app claim to upload the contacts even though it does not.
• Converse 10.1.7 with StropheJS fix for XEP-0474: SASL SCRAM Downgrade Protection.
• Psi+ 1.5.1716 through 1.5.1669 have been released.

Servers

• Openfire has been released in version 4.8.1.
  • Openfire inVerse plugin version 10.1.7.1 has been released.
  • Openfire PionTurn plugin 1.0.0 has been released! It creates a TURN/STUN server for Openfire.

Libraries & Tools

• A new release of go-xmpp 0.1.4.
• A new release of go-sendxmpp 0.9.0.
• overpush: A self-hosted, drop-in replacement for Pushover, that uses XMPP as delivery method. It offers the same API for submitting messages, so that existing setups (e.g. Grafana) can continue working and only require changing the API URL. Release article.

Extensions and specifications

The XMPP Standards Foundation develops extensions to XMPP in its XEP series in addition to XMPP RFCs.

Developers and other standards experts from around the world collaborate on these extensions, developing new specifications for emerging practices, and refining existing ways of doing things. Proposed by anybody, the particularly successful ones end up as Final or Active - depending on their type - while others are carefully archived as Deferred. This life cycle is described in XEP-0001, which contains the formal and canonical definitions for the types, states, and processes. Read more about the standards process. Communication around Standards and Extensions happens in the Standards Mailing List (online archive).

Proposed

The XEP development process starts by writing up an idea and submitting it to the XMPP Editor. Within two weeks, the Council decides whether to accept this proposal as an Experimental XEP.

• Message Displayed Synchronization
  • This specification allows multiple clients of the same user to synchronize the displayed state of their chats.

New

• Version 0.1.0 of XEP-0485 (PubSub Server Information)
  • Promoted to Experimental. (dg)
• Version 0.1.0 of XEP-0486 (MUC Avatars)
  • Promoted to Experimental (XEP Editor: dg)
• Version 0.1.0 of XEP-0487 (Host Meta 2 - One Method To Rule Them All)
  • Promoted to Experimental (XEP Editor: dg)
• Version 0.1.0 of XEP-0488 (MUC Token Invite)
  • Promoted to Experimental (XEP Editor: dg)
• Version 0.1.0 of XEP-0489 (Reporting Account Affiliations)
  • Promoted to Experimental (XEP Editor: dg)
• Version 0.1.0 of XEP-0490 (Message Displayed Synchronization)
  • Promoted to Experimental (XEP Editor: dg)

Deferred

If an experimental XEP is not updated for more than twelve months, it will be moved off Experimental to Deferred. If there is another update, it will put the XEP back onto Experimental.

• No XEPs deferred this month.

Updated

• Version 1.25.0 of XEP-0001 (XMPP Extension Protocols)
  • Add note that editorial changes do not affect Deferred state (XEP Editor: dg)
• Version 1.26.0 of XEP-0060 (Publish-Subscribe)
  • Add examples for publishing item without ID (melvo)
• Version 1.1.1 of XEP-0313 (Message Archive Management)
  • Add XEP-0136 to superseded specifications (gdk)
• Version 0.5.0 and 0.6.0 of XEP-0333 (Displayed Markers (was: Chat Markers))
  • Remove <received/> to not replicate functionality.
  • Remove <acknowledged/> because it was not implemented in the last 10 years and apparently is not needed.
  • Remove Disco feature. Opting in via <markable/> is enough (dg)
  • Add Business Rule about opportunistic Displayed Markers in 1:1 chats (dg)
• Version 0.5.0 of XEP-0334 (Message Processing Hints)
  • Incorporate last call feedback from 2017.
  • Differences between this specification and XEP-0079 have been clarified.
  • A note about handling of hints found in error stanzas has been added. (mw)
• Version 0.4.1 of XEP-0388 (Extensible SASL Profile)
  • Add missing elements to XML Schema
  • Add missing XMPP Registrar Considerations (dg)
• Version 0.3.0 of XEP-0398 (User Avatar to vCard-Based Avatars Conversion)
  • Add text to explain that both and are valid implementations.
  • Add Security Considerations for both variants (dg)
• Version 0.4.1 of XEP-0424 (Message Retraction)
  • Fix schema.
  • Add missing for attribute in fallback element (Example 4). (nc)
• Version of XEP-0425 (Moderated Message Retraction)
  • Remove the dependency on XEP-0422 Message Fastening
  • Rename to ‘Moderated Message Retraction’ and focus only on the retraction use-case
  • Ensure compatibility with clients that only implement XEP-0424
  • Clarify the purpose of the <reason/> element
• Version 0.3.0 and 0.3.1 of XEP-0447 (Stateless file sharing)
  • Describe how to use for multiple files, with body text, without any source in original message and compatibility with various current deployed protocols. (lmw)
  • Fix example for multiple files. (lmw)

Last Call

Last calls are issued once everyone seems satisfied with the current XEP status. After the Council decides whether the XEP seems ready, the XMPP Editor issues a Last Call for comments. The feedback gathered during the Last Call can help improve the XEP before returning it to the Council for advancement to Stable.

• XEP-0333: Displayed Markers (was: Chat Markers)
• XEP-0334: Message Processing Hints
• XEP-0360: Nonzas (are not Stanzas)
• XEP-0386: Bind 2
• XEP-0388: Extensible SASL Profile
• XEP-0392: Consistent Color Generation

Stable

• Version 1.0.0 of XEP-0392 (Consistent Color Generation)
  • Accept as Stable as per Council Vote from 2024-03-27. (XEP Editor (dg))

Deprecated

• No XEP deprecated this month.

Rejected

• XEP-0360: Nonzas (are not Stanzas)

Spread the news

Please share the news on other networks:

• Mastodon
• Twitter
• YouTube
• LinkedIn
• Lemmy instance (unofficial)
• Reddit (unofficial)
• XMPP Facebook page (unofficial)

Also check out our RSS Feed!

Looking for job offers or want to hire a professional consultant for your XMPP project? Visit our XMPP job board.

Newsletter Contributors & Translations

This is a community effort, and we would like to thank translators for their contributions. Volunteers are welcome! Translations of the XMPP Newsletter will be released here (with some delay):

• English (original): xmpp.org
  • General contributors: Adrien Bourmault (neox), Alexander “PapaTutuWawa”, Arne, cal0pteryx, emus, Federico, Jonas Stein, Kris “poVoq”, Licaon_Kter, Ludovic Bocquet, Mario Sabatino, melvo, MSavoritias (fae,ve), nicola, Simone Canaletti, XSF iTeam
• French: jabberfr.org and linuxfr.org
  • Translators: Adrien Bourmault (neox), alkino, anubis, Arkem, Benoît Sibaud, mathieui, nyco, Pierre Jarillon, Ppjet6, Ysabeau
• Italian: notes.nicfab.eu
  • Translators: nicola

Help us to build the newsletter

This XMPP Newsletter is produced collaboratively by the XMPP community. Each month’s newsletter issue is drafted in this simple pad. At the end of each month, the pad’s content is merged into the XSF Github repository. We are always happy to welcome contributors. Do not hesitate to join the discussion in our Comm-Team group chat (MUC) and thereby help us sustain this as a community effort. You have a project and want to spread the news? Please consider sharing your news or events here, and promote it to a large audience.

Tasks we do on a regular basis:

• gathering news in the XMPP universe
• short summaries of news and events
• summary of the monthly communication on extensions (XEPs)
• review of the newsletter draft
• preparation of media images
• translations
• communication via media accounts

License

This newsletter is published under CC BY-SA license.

We are happy to announce the release of Smack 4.4.8, our XMPP-client library for JVMs and Android. For a high-level overview of what’s changed in Smack 4.4.8, check out Smack’s changelog

Smack 4.4.8 contains mostly small fixes. However, we fixed one nasty bug in Smack’s reactor causing an, potentially endless, busy loop. Smack’s new connection infrastrucutre makes heavy use of the reactor, that enables tausands of connections being served by only a handful of threads.

As always, this Smack patchlevel release is API compatible within the same major-minor version series (4.4) and all Smack releases are available via Maven Central.

We would like to use this occasion to point at that Smack now ships with a NOTICE file. Please note that this adds some requirements when using Smack as per the Apache License 2.0. The content of Smack’s NOTICE file can conveniently be retrieved using Smack.getNoticeStream().

1 post - 1 participant

Read full topic

Looking to learn more about the basics of RabbitMQ? This powerful message broker plays a key role in modern, distributed systems. 

This post will break down its fundamentals and highlight its importance in the world of modern, distributed systems.

An introduction to RabbitMQ

RabbitMQ emerged from the need to create a scalable, robust messaging system that was able to handle high volumes of communications between applications, all while maintaining both data and performance. 

It is now a popular open-source messaging broker, with queue software written in Erlang. One of its key strengths is its ability to support and adhere to Application Programming Interface (API) protocols, for example, AMQP, HTTP AND STOMP. 

What are APIs you ask? 

They define the rules and conventions that allow for the interaction and communication of different software. For developers, APIs are the go-between that allows them to access a software or services functionality, without the need for a full understanding of the ins and outs of that particular system. 

In turn, these protocols offer a standard method of transmitting commands and data. The result? Seamless integration and interoperability between different systems.

Let’s circle back to one previously mentioned protocol, the Advanced Message Queuing Protocol or AMQP. This protocol was made to ensure that messages are reliably delivered between applications, no matter where the platform it is running on is located. AMQP has precise rules for the delivery, formatting and confirmation of messages. This ensures that every message sent through an AMQP-based system, like RabbitMQ, reaches its intended location.

Here’s an illustration better explaining the AMQP system:

Source: The Heart of RabbitMQ

What is RabbitMQ used for?

Developers use RabbitMQ to efficiently process high-throughput and reliable background jobs and facilitate the integration and communication between applications. It is also great at managing complex routing to consumers by integrating various applications and services.

RabbitMQ is also a great solution for web servers that require a rapid-request response. It also effectively distributes workloads between workers, handling over 20,000 messages per second. It can manage background jobs and longer-running tasks, for example, PDF conversion and file scanning.

How does RabbitMQ work?

Think of RabbitMQ as a middleman. It collects messages from a producer (publisher) and passes them on to receivers (consumers). Using a messaging queue, it then holds messages until the consumers can process them. 

Here’s a better overview of these core systems:

Source: RabbitMQ tutorials

Key features of RabbitMQ

As one of the most powerful and flexible messaging systems, RabbitMQ offers several key features, including:

Security: Various security features in RabbitMQ are designed to protect systems from unauthorised access and potential data breaches. With authentication and authorisation support, administrators can control which users or applications have access to certain queues or exchanges. It also supports SSL/TLS encryption, to ensure clear communication between brokers and clients.

Reliability: Reliable message delivery by supporting features, such as message acknowledgement and persistent message storage.

Scalable and fault-tolerant: RabbitMQ provides features for building scalable and fault-tolerant messaging systems. It also supports clustering, whereby adding more nodes to the cluster allows the system to handle higher message volumes. It’s then able to distribute the workload across multiple nodes, making for efficient utilisation of resources. In the case of a node failure, other nodes in the cluster can continue to handle messages without interruption.

Extended features: RabbitMQ is not limited to the AMQP protocol, but is very versatile and can support a host of others, such as MQTT and STOMP.

Enterprise and the Cloud: RabbitMQ is lightweight and easy to deploy on the public as well as private clouds using pluggable authentication authorisation.

Tools and Plugins: RabbitMQ offers a host of tools and plugins, ideal for integration and wider support.

Common use cases for RabbitMQ

We’ve already highlighted the versatility of RabbitMQ in modern distributed systems. With its robust features and flexible architecture, here are some most common use cases:

Legacy applications: RabbitMQ integrates with legacy systems by using available or custom plugins. You can connect consumer apps to legacy apps for example, connecting JMS apps using the Java Message Service (JMS) plug-in and JMS client library. 

Distributed systems: RabbitMQ serves as a messaging infrastructure in distributed systems. It fosters asynchronous communication between different components, facilitating the scalability and decoupling of the system.

IoT applications: When used in Internet of Things (IoT) applications, RabbitMQ can handle the exchange of messages between devices and backend systems, allowing for reliable and efficient communication, control and real-time monitoring of IoT devices.

Chat applications: For real-time communication in chat applications, RabbitMQ manages messaging exchanges between users, facilitating group chat and instant messaging functionalities. 

Task/job queues: RabbitMQ manages task queues and distributes work across multiple workers. This means that tasks are processed efficiently and at scale, reducing bottlenecks and utilising resources. 

Event-driven architectures: RabbitMQ is great for carrying out event-driven architectures.

It allows various system components to respond to events and seamlessly interact with each other. 
Microservices communication: A common use of RabbitMQ is enabling asynchronous and reliable communication between microservices. Messages are delivered, even if some services are unavailable.

To conclude 

As businesses seek to adopt distributed architectures and microservices-based applications, RabbitMQ remains a go-to choice for improved adaptability and seamless integration across systems. If you’d like to discuss how RabbitMQ can improve your applications, get in touch with the Erlang Solutions team.

The post A Guide to RabbitMQ appeared first on Erlang Solutions.

Harrier is our Military Messaging client. It provides a modern, secure web UI that supports SMTP, STANAG 4406 and ACP 127. Harrier allows authorised users to access role-based mailboxes and respond as a role within an organisation rather than as an individual.

You can find out more about Harrier here.

Server Administration and Monitoring

Harrier 4.0 adds a Web interface for server administrators to configure Harrier. Key points:

• Secure bootstrap
• Sensible defaulting of parameters to facilitate startup
• Per domain and global configuration options
• Security features, including TLS, HSM, S/MIME and Security Labels/Security Policy
• Full configuration of all Harrier options and capabilities

In addition to configuration, the Web user interface provides a monitoring capability to show server activity and key operational parameters.

UI Enhancements

A number of improvements made to the Harrier UI including:

• Variable size compose windows, retaining user preferences and stacking multiple windows
• HTML Message editing:
  • Font bold/italic/underline/colour
  • Lists and Bullets
  • Reply to HTML messages
• Undo and redo in message editor
• Organizations in from selection has configurable default and alphabetic sort.
• Active role shown on browser tab. Facilitates working with multiple roles in different tabs.
• Extended message search capabilities to include:
  • Filter by precedence
  • Free text search in choice of: body; subject; SIC; action; info; from

Security Enhancements

The following security enhancements added:

• Per domain S/MIME signing policy (never/if possible/always). Model is administrator choice rather than user selection.
• Policy control of using S/MIME header signing.
• Policy choice to alert users to unsigned messages.
• Policy choice to allow encryption.
• Policy choice of encryption by enveloping or triple wrap.
• Message Decrypt on initial access. The primary goal of S/MIME encryption is end to end protection. Some clients leave messages encrypted, which can lead to problems over time if keys become unavailable or are changed. Decryption prevents these issues. Note that for triple wrap, the inner signature is retained.
  

Other Enhancements

• Server option to force user confirmation of message send (audit logged). Important in some scenarios to confirm message responsibility.
• Option to configure multiple address books in different directories.
• Revalidation of recipients before message release.
• Timezone option to be Zulu or Local.

🚀 Introducing ejabberd 24.02: A Huge Release!

ejabberd 24.02 has just been release and well, this is a huge release with 200 commits and more in the libraries. We’ve packed this update with a plethora of new features, significant improvements, and essential bug fixes, all designed to supercharge your messaging infrastructure.


– 🌐 Matrix Federation Unleashed: Imagine seamlessly connecting with Matrix servers – it’s now possible! ejabberd breaks new ground in cross-platform communication, fostering a more interconnected messaging universe. We have still some ground to cover and for that we are waiting for your feedback.
– 🔐 Cutting-Edge Security with TLS 1.3 & SASL2: In an era where security is paramount, ejabberd steps up its game. With support for TLS 1.3 and advanced SASL2 protocols, we increase the overall security for all platform users.
– 🚀 Performance Enhancements with Bind 2: Faster connection times, especially crucial for mobile network users, thanks to Bind 2 and other performance optimizations.
– 🔄 User gains better control over on their messages: The new support for XEP-0424: Message Retraction allows users to manage their message history and remove something they posted by mistake.
– 🔧 Optimized server pings by relying on an existing mechanism coming from XEP-0198
– 📈 Streamlined API Versioning: Our refined API versioning means smoother, more flexible integration for your applications.
– 🧩 Enhanced Elixir, Mix and Rebar3 Support

If you upgrade ejabberd from a previous release, please review those changes:

• Update the SQL schema
• Update API commands as explained below, or use API versioning
• Mix or Rebar3 used by default instead of Rebar to compile ejabberd
• Authentication workaround for Converse.js and Strophe.js

A more detailed explanation of those topics and other features:

Matrix federation

ejabberd is now able to federate with Matrix servers. Detailed instructions to setup Matrix federation with ejabberd will be detailed in another post.

Here is a quick summary of the configuration steps:

First, s2s must be enabled on ejabberd. Then define a listener that uses mod_matrix_gw:

listen:
  -
    port: 8448
    module: ejabberd_http
    tls: true
    certfile: "/opt/ejabberd/conf/server.pem"
    request_handlers:
      "/_matrix": mod_matrix_gw

And add mod_matrix_gw in your modules:

modules:
  mod_matrix_gw:
    matrix_domain: "domain.com"
    key_name: "somename"
    key: "yourkeyinbase64"

Support TLS 1.3, Bind 2, SASL2

• RFC 9266 Channel Bindings for TLS 1.3: This enhances security and reliability.
• XEP-0386: Bind 2: This is going to reduce the connection time for clients. This is especially important if you are using XMPP to connect from a mobile network.
• XEP-0388: Extensible SASL Profile – SASL2
• XEP-0440: SASL Channel-Binding Type Capability: These updates are aimed at bolstering our authentication mechanisms.
• XEP-0474: SASL SCRAM Downgrade Protection

Support for XEP-0424 Message Retraction

With the new support for XEP-0424: Message Retraction, users of MAM message archiving can control their message archiving, with the ability to ask for deletion.

Support for XEP-0198 pings

If stream management is enabled, let mod_ping trigger XEP-0198 <r/>equests rather than sending XEP-0199 pings. This avoids the overhead of the ping IQ stanzas, which, if stream management is enabled, are accompanied by XEP-0198 elements anyway.

Update the SQL schema

The table archive has a text column named origin_id (see commit 975681). You have two methods to update the SQL schema of your existing database:

If using MySQL or PosgreSQL, you can enable the option update_sql_schema and ejabberd will take care to update the SQL schema when needed: add in your ejabberd configuration file the line update_sql_schema: true

If you are using other database, or prefer to update manually the SQL schema:

• MySQL default schema:

ALTER TABLE archive ADD COLUMN origin_id text NOT NULL DEFAULT '';
ALTER TABLE archive ALTER COLUMN origin_id DROP DEFAULT;
CREATE INDEX i_archive_username_origin_id USING BTREE ON archive(username(191), origin_id(191));

• MySQL new schema:

ALTER TABLE archive ADD COLUMN origin_id text NOT NULL DEFAULT '';
ALTER TABLE archive ALTER COLUMN origin_id DROP DEFAULT;
CREATE INDEX i_archive_sh_username_origin_id USING BTREE ON archive(server_host(191), username(191), origin_id(191));

• PostgreSQL default schema:

ALTER TABLE archive ADD COLUMN origin_id text NOT NULL DEFAULT '';
ALTER TABLE archive ALTER COLUMN origin_id DROP DEFAULT;
CREATE INDEX i_archive_username_origin_id ON archive USING btree (username, origin_id);

• PostgreSQL new schema:

ALTER TABLE archive ADD COLUMN origin_id text NOT NULL DEFAULT '';
ALTER TABLE archive ALTER COLUMN origin_id DROP DEFAULT;
CREATE INDEX i_archive_sh_username_origin_id ON archive USING btree (server_host, username, origin_id);

• MSSQL default schema:

ALTER TABLE [dbo].[archive] ADD [origin_id] VARCHAR (250) NOT NULL;
CREATE INDEX [archive_username_origin_id] ON [archive] (username, origin_id)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON);

• MSSQL new schema:

ALTER TABLE [dbo].[archive] ADD [origin_id] VARCHAR (250) NOT NULL;
CREATE INDEX [archive_sh_username_origin_id] ON [archive] (server_host, username, origin_id)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON);

• SQLite default schema:

ALTER TABLE archive ADD COLUMN origin_id text NOT NULL DEFAULT '';
CREATE INDEX i_archive_username_origin_id ON archive (username, origin_id);

• SQLite new schema:

ALTER TABLE archive ADD COLUMN origin_id text NOT NULL DEFAULT '';
CREATE INDEX i_archive_sh_username_origin_id ON archive (server_host, username, origin_id);

Authentication workaround for Converse.js and Strophe.js

This ejabberd release includes support for XEP-0474: SASL SCRAM Downgrade Protection, and some clients may not support it correctly yet.

If you are using Converse.js 10.1.6 or older, Movim 0.23 Kojima or older, or any other client based in Strophe.js v1.6.2 or older, you may notice that they cannot authenticate correctly to ejabberd.

To solve that problem, either update to newer versions of those programs (if they exist), or you can enable temporarily the option disable_sasl_scram_downgrade_protection in the ejabberd configuration file ejabberd.yml like this:

disable_sasl_scram_downgrade_protection: true

Support for API versioning

Until now, when a new ejabberd release changed some API command (an argument renamed, a result in a different format…), then you had to update your API client to the new API at the same time that you updated ejabberd.

Now the ejabberd API commands can have different versions, by default the most recent one is used, and the API client can specify the API version it supports.

In fact, this feature was implemented seven years ago, included in ejabberd 16.04, documented in ejabberd Docs: API Versioning… but it was never actually used!

This ejabberd release includes many fixes to get API versioning up to date, and it starts being used by several commands.

Let’s say that ejabberd 23.10 implemented API version 0, and this ejabberd 24.02 adds API version 1. You may want to update your API client to use the new API version 1… or you can continue using API version 0 and delay API update a few weeks or months.

To continue using API version 0:
– if using ejabberdctl, use the switch --version 0. For example: ejabberdctl --version 0 get_roster admin localhost
– if using mod_http_api, in ejabberd configuration file add v0 to the request_handlers path. For example: /api/v0: mod_http_api

Check the details in ejabberd Docs: API Versioning.

ejabberd commands API version 1

When you want to update your API client to support ejabberd API version 1, those are the changes to take into account:
– Commands with list arguments
– mod_http_api does not name integer and string results
– ejabberdctl with list arguments
– ejabberdctl list results

All those changes are described in the next sections.

Commands with list arguments

Several commands now use list argument instead of a string with separators (different commands used different separators: ; : \\n ,).

The commands improved in API version 1:
– add_rosteritem
– oauth_issue_token
– send_direct_invitation
– srg_create
– subscribe_room
– subscribe_room_many

For example, srg_create in API version 0 took as arguments:

{"group": "group3",
 "host": "myserver.com",
 "label": "Group3",
 "description": "Third group",
 "display": "group1\\ngroup2"}

now in API version 1 the command expects as arguments:

{"group": "group3",
 "host": "myserver.com",
 "label": "Group3",
 "description": "Third group",
 "display": ["group1", "group2"]}

mod_http_api not named results

There was an incoherence in mod_http_api results when they were integer/string and when they were list/tuple/rescode…: the result contained the name, for example:

$ curl -k -X POST -H "Content-type: application/json" -d '{}' "http://localhost:5280/api/get_loglevel/v0"
{"levelatom":"info"}

Staring in API version 1, when result is an integer or a string, it will not contain the result name. This is now coherent with the other result formats (list, tuple, …) which don’t contain the result name either.

Some examples with API version 0 and API version 1:

$ curl -k -X POST -H "Content-type: application/json" -d '{}' "http://localhost:5280/api/get_loglevel/v0"
{"levelatom":"info"}

$ curl -k -X POST -H "Content-type: application/json" -d '{}' "http://localhost:5280/api/get_loglevel"
"info"

$ curl -k -X POST -H "Content-type: application/json" -d '{"name": "registeredusers"}' "http://localhost:5280/api/stats/v0"
{"stat":2}

$ curl -k -X POST -H "Content-type: application/json" -d '{"name": "registeredusers"}' "http://localhost:5280/api/stats"
2

$ curl -k -X POST -H "Content-type: application/json" -d '{"host": "localhost"}' "http://localhost:5280/api/registered_users/v0"
["admin","user1"]

$ curl -k -X POST -H "Content-type: application/json" -d '{"host": "localhost"}' "http://localhost:5280/api/registered_users"
["admin","user1"]

ejabberdctl with list arguments

ejabberdctl now supports list and tuple arguments, like mod_http_api and ejabberd_xmlrpc. This allows ejabberdctl to execute all the existing commands, even some that were impossible until now like create_room_with_opts and set_vcard2_multi.

List elements are separated with , and tuple elements are separated with :.

Relevant commands:
– add_rosteritem
– create_room_with_opts
– oauth_issue_token
– send_direct_invitation
– set_vcard2_multi
– srg_create
– subscribe_room
– subscribe_room_many

Some example uses:

ejabberdctl add_rosteritem user1 localhost testuser7 localhost NickUser77l gr1,gr2,gr3 both
ejabberdctl create_room_with_opts room1 conference.localhost localhost public:false,persistent:true
ejabberdctl subscribe_room_many user1@localhost:User1,admin@localhost:Admin room1@conference.localhost urn:xmpp:mucsub:nodes:messages,u

ejabberdctl list results

Until now, ejabberdctl returned list elements separated with ;. Now in API version 1 list elements are separated with ,.

For example, in ejabberd 23.10:

$ ejabberdctl get_roster admin localhost
jan@localhost jan   none    subscribe       group1;group2
tom@localhost tom   none    subscribe       group3

Since this ejabberd release, using API version 1:

$ ejabberdctl get_roster admin localhost
jan@localhost jan   none    subscribe       group1,group2
tom@localhost tom   none    subscribe       group3

it is still possible to get the results in the old syntax, using API version 0:

$ ejabberdctl --version 0 get_roster admin localhost
jan@localhost jan   none    subscribe       group1;group2
tom@localhost tom   none    subscribe       group3

ejabberdctl help improved

ejabberd supports around 200 administrative commands, and probably you consult them in the ejabberd Docs -> API Reference page, where all the commands documentation is perfectly displayed…

The ejabberdctl command-line script already allowed to consult the commands documentation, consulting in real-time your ejabberd server to show you exactly the commands that are available. But it lacked some details about the commands. That has been improved, and now ejabberdctl shows all the information, including arguments description, examples and version notes.

For example, the connected_users_vhost command documentation as seen in the ejabberd Docs site is equivalently visible using ejabberdctl:

$ ejabberdctl help connected_users_vhost
  Command Name: connected_users_vhost

  Arguments: host::binary : Server name

  Result: connected_users_vhost::[ sessions::string ]

  Example: ejabberdctl connected_users_vhost "myexample.com"
           user1@myserver.com/tka
           user2@localhost/tka

  Tags: session

  Module: mod_admin_extra

  Description: Get the list of established sessions in a vhost

Experimental support for Erlang/OTP 27

Erlang/OTP 27.0-rc1 was recently released, and ejabberd can be compiled with it. If you are developing or experimenting with ejabberd, it would be great if you can use Erlang/OTP 27 and report any problems you find. For production servers, it’s recommended to stick with Erlang/OTP 26.2 or any previous version.

In this sense, the rebar and rebar3 binaries included with ejabberd are also updated: now they support from Erlang 24 to Erlang 27. If you want to use older Erlang versions from 20 to 23, there are compatible binaries available in git: rebar from ejabberd 21.12 and rebar3 from ejabberd 21.12.

Of course, if you have rebar or rebar3 already installed in your system, it’s preferable if you use those ones, because probably they will be perfectly compatible with whatever erlang version you have installed.

Installers and ejabberd container image

The binary installers now include the recent and stable Erlang/OTP 26.2.2 and Elixir 1.16.1. Many other dependencies were updated in the installers, the most notable is OpenSSL that has jumped to version 3.2.1.

The ejabberd container image and the ecs container image have gotten all those version updates, and also Alpine is updated to 3.19.

By the way, this container image already had support to run commands when the container starts… And now you can setup the commands to allow them fail, by prepending the character !.

Summary of compilation methods

When compiling ejabberd from source code, you may have noticed there are a lot of possibilities. Let’s take an overview before digging in the new improvements:

• Tools to manage the dependencies and compilation:
  • Rebar: it is nowadays very obsolete, but still does the job of compiling ejabberd
  • Rebar3: the successor of Rebar, with many improvements and plugins, supports hex.pm and Elixir compilation
  • Mix: included with the Elixir programming language, supports hex.pm, and erlang compilation
• Installation methods:
  • make install: copies the files to the system
  • make prod: prepares a self-contained OTP production release in _build/prod/, and generates a tar.gz file. This was previously named make rel
  • make dev: prepares quickly an OTP development release in _build/dev/
  • make relive: prepares the barely minimum in _build/relive/ to run ejabberd and starts it
• Start scripts and alternatives:
  • ejabberdctl with erlang shell: start/foreground/live
  • ejabberdctl with elixir shell: iexlive
  • ejabberd console/start (this script is generated by rebar3 or mix, and does not support ejabberdctl configurable options)

For example:
– the CI dynamic tests use rebar3, and Runtime tries to test all the possible combinations
– ejabberd binary installers are built using: mix + make prod
– container images are built using: mix + make prod too, and started with ejabberdctl foreground

Several combinations didn’t work correctly until now and have been fixed, for example:
– mix + make relive
– mix + make prod/dev + ejabberdctl iexlive
– mix + make install + ejabberdctl start/foregorund/live
– make uninstall buggy has an experimental alternative: make uninstall-rel
– rebar + make prod with Erlang 26

Use Mix or Rebar3 by default instead of Rebar to compile ejabberd

ejabberd uses Rebar to manage dependencies and compilation since ejabberd 13.10 4d8f770. However, that tool is obsolete and unmaintained since years ago, because there is a complete replacement:

Rebar3 is supported by ejabberd since 20.12 0fc1aea. Among other benefits, this allows to download dependencies from hex.pm and cache them in your system instead of downloading them from git every time, and allows to compile Elixir files and Elixir dependencies.

In fact, ejabberd can be compiled using mix (a tool included with the Elixir programming language) since ejabberd 15.04 ea8db99 (with improvements in ejabberd 21.07 4c5641a)

For those reasons, the tool selection performed by ./configure will now be:
– If --with-rebar=rebar3 but Rebar3 not found installed in the system, use the rebar3 binary included with ejabberd
– Use the program specified in option: --with-rebar=/path/to/bin
– If none is specified, use the system mix
– If Elixir not found, use the system rebar3
– If Rebar3 not found, use the rebar3 binary included with ejabberd

Removed Elixir support in Rebar

Support for Elixir 1.1 was added as a dependency in commit 01e1f67 to ejabberd 15.02. This allowed to compile Elixir files. But since Elixir 1.4.5 (released Jun 22, 2017) it isn’t possible to get Elixir as a dependency… it’s nowadays a standalone program. For that reason, support to download old Elixir 1.4.4 as a dependency has been removed.

When Elixir support is required, better simply install Elixir and use mix as build tool:

./configure --with-rebar=mix

Or install Elixir and use the experimental Rebar3 support to compile Elixir files and dependencies:

./configure --with-rebar=rebar3 --enable-elixir

Added Elixir support in Rebar3

It is now possible to compile ejabberd using Rebar3 and support Elixir compilation. This compiles the Elixir files included in ejabberd’s lib/ path. There’s also support to get dependencies written in Elixir, and it’s possible to build OTP releases including Elixir support.

It is necessary to have Elixir installed in the system, and configure the compilation using --enable-elixir. For example:

apt-get install erlang erlang-dev elixir
git clone https://github.com/processone/ejabberd.git ejabberd
cd ejabberd
./autogen.sh
./configure --with-rebar=rebar3 --enable-elixir
make
make dev
_build/dev/rel/ejabberd/bin/ejabberdctl iexlive

Elixir versions supported

Elixir 1.10.3 is the minimum supported, but:
– Elixir 1.10.3 or higher is required to build an OTP release with make prod or make dev
– Elixir 1.11.4 or higher is required to build an OTP release if using Erlang/OTP 24 or higher
– Elixir 1.11.0 or higher is required to use make relive
– Elixir 1.13.4 with Erlang/OTP 23.0 are the lowest versions tested by Runtime

For all those reasons, if you want to use Elixir, it is highly recommended to use Elixir 1.13.4 or higher with Erlang/OTP 23.0 or higher.

make rel is renamed to make prod

When ejabberd started to use Rebar2 build tool, that tool could create an OTP release, and the target in Makefile.in was conveniently named make rel.

However, newer tools like Rebar3 and Elixir’s Mix support creating different types of releases: production, development, … In this sense, our make rel target is nowadays more properly named make prod.

For backwards compatibility, make rel redirects to make prod.

New make install-rel and make uninstall-rel

This is an alternative method to install ejabberd in the system, based in the OTP release process. It should produce exactly the same results than the existing make install.

The benefits of make install-rel over the existing method:
– this uses OTP release code from rebar/rebar3/mix, and consequently requires less code in our Makefile.in
– make uninstall-rel correctly deletes all the library files

This is still experimental, and it would be great if you are able to test it and report any problem; eventually this method could replace the existing one.

Just for curiosity:
– ejabberd 13.03-beta1 got support for make uninstall was added
– ejabberd 13.10 introduced Rebar build tool and code got more modular
– ejabberd 15.10 started to use the OTP directory structure for ‘make install’, and this broke make uninstall

Acknowledgments

We would like to thank the contributions to the source code, documentation, and translation provided for this release by:

• Holger Weiß for the support XEP-0198 pings and XEP-0425 fixes
• Mr. EddX, updated the Bulgarian translation
• Sketch6580, updated the Chinese translation
• Jan Aschenbrenner, updated the Czech translation
• Ranforingus, updated the Dutch translation
• Ermete Melchiorre, updated the Italian translation
• Mako N, updated the Japanese translation
• Silvério Santos, updated the Portuguese translation
• Олександр Кревський, updated the Ukrainian translation

And also to all the people contributing in the ejabberd chatroom, issue tracker…

Improvements in ejabberd Business Edition

Customers of the ejabberd Business Edition, in addition to all those improvements and bugfixes, also get:

Push

• Fix clock issue when signing Apple push JWT tokens
• Share Apple push JWT tokens between nodes in cluster
• Increase allowed certificates chain depth in GCM requests
• Use x:oob data as source for image delivered in pushes
• Process only https urls in oob as images in pushes
• Fix jid in disable push iq generated by GCM and Webhook service
• Add better logging for TooManyProviderTokenUpdated error
• Make get_push_logs command generate better error if mod_push_logger not available
• Add command get_push_logs that can be used to retrieve info about recent pushes and errors reported by push services
• Add support for webpush protocol for sending pushes to safari/chrome/firefox browsers

MAM

• Expand mod_mam_http_access API to also accept range of messages

MUC

• Update mod_muc_state_query to fix subject_author room state field
• Fix encoding of config xdata in mod_muc_state_query

PubSub

• Allow pubsub node owner to overwrite items published by other persons (p1db)

ChangeLog

This is a more detailed list of changes in this ejabberd release:

Core

• Added Matrix gateway in mod_matrix_gw
• Support SASL2 and Bind2
• Support tls-server-end-point channel binding and sasl2 codec
• Support tls-exporter channel binding
• Support XEP-0474: SASL SCRAM Downgrade Protection
• Fix presenting features and returning results of inline bind2 elements
• disable_sasl_scram_downgrade_protection: New option to disable XEP-0474
• negotiation_timeout: Increase default value from 30s to 2m
• mod_carboncopy: Teach how to interact with bind2 inline requests

Other

• ejabberdctl: Fix startup problem when having set EJABBERD_OPTS and logger options
• ejabberdctl: Set EJABBERD_OPTS back to "", and use previous flags as example
• eldap: Change logic for eldap tls_verify=soft and false
• eldap: Don’t set fail_if_no_peer_cert for eldap ssl client connections
• Ignore hints when checking for chat states
• mod_mam: Support XEP-0424 Message Retraction
• mod_mam: Fix XEP-0425: Message Moderation with SQL storage
• mod_ping: Support XEP-0198 pings when stream management is enabled
• mod_pubsub: Normalize pubsub max_items node options on read
• mod_pubsub: PEP nodetree: Fix reversed logic in node fixup function
• mod_pubsub: Only care about PEP bookmarks options when creating node from scratch

SQL

• MySQL: Support sha256_password auth plugin
• ejabberd_sql_schema: Use the first unique index as a primary key
• Update SQL schema files for MAM’s XEP-0424
• New option sql_flags: right now only useful to enable mysql_alternative_upsert

Installers and Container

• Container: Add ability to ignore failures in execution of CTL_ON_* commands
• Container: Update to Erlang/OTP 26.2, Elixir 1.16.1 and Alpine 3.19
• Container: Update this custom ejabberdctl to match the main one
• make-binaries: Bump OpenSSL 3.2.1, Erlang/OTP 26.2.2, Elixir 1.16.1
• make-binaries: Bump many dependency versions

Commands API

• print_sql_schema: New command available in ejabberdctl command-line script
• ejabberdctl: Rework temporary node name generation
• ejabberdctl: Print argument description, examples and note in help
• ejabberdctl: Document exclusive ejabberdctl commands like all the others
• Commands: Add a new muc_sub tag to all the relevant commands
• Commands: Improve syntax of many commands documentation
• Commands: Use list arguments in many commands that used separators
• Commands: set_presence: switch priority argument from string to integer
• ejabberd_commands: Add the command API version as a tag vX
• ejabberd_ctl: Add support for list and tuple arguments
• ejabberd_xmlrpc: Fix support for restuple error response
• mod_http_api: When no specific API version is requested, use the latest

Compilation with Rebar3/Elixir/Mix

• Fix compilation with Erlang/OTP 27: don’t use the reserved word ‘maybe’
• configure: Fix explanation of --enable-group option (#4135)
• Add observer and runtime_tools in releases when --enable-tools
• Update “make translations” to reduce build requirements
• Use Luerl 1.0 for Erlang 20, 1.1.1 for 21-26, and temporary fork for 27
• Makefile: Add install-rel and uninstall-rel
• Makefile: Rename make rel to make prod
• Makefile: Update make edoc to use ExDoc, requires mix
• Makefile: No need to use escript to run rebar|rebar3|mix
• configure: If --with-rebar=rebar3 but rebar3 not system-installed, use local one
• configure: Use Mix or Rebar3 by default instead of Rebar2 to compile ejabberd
• ejabberdctl: Detect problem running iex or etop and show explanation
• Rebar3: Include Elixir files when making a release
• Rebar3: Workaround to fix protocol consolidation
• Rebar3: Add support to compile Elixir dependencies
• Rebar3: Compile explicitly our Elixir files when --enable-elixir
• Rebar3: Provide proper path to iex
• Rebar/Rebar3: Update binaries to work with Erlang/OTP 24-27
• Rebar/Rebar3: Remove Elixir as a rebar dependency
• Rebar3/Mix: If dev profile/environment, enable tools automatically
• Elixir: Fix compiling ejabberd as a dependency (#4128)
• Elixir: Fix ejabberdctl start/live when installed
• Elixir: Fix: FORMATTER ERROR: bad return value (#4087)
• Elixir: Fix: Couldn’t find file Elixir Hex API
• Mix: Enable stun by default when vars.config not found
• Mix: New option vars_config_path to set path to vars.config (#4128)
• Mix: Fix ejabberdctl iexlive problem locating iex in an OTP release

Full Changelog

https://github.com/processone/ejabberd/compare/23.10…24.02

ejabberd 24.02 download & feedback

As usual, the release is tagged in the Git source code repository on GitHub.

The source package and installers are available in ejabberd Downloads page. To check the *.asc signature files, see How to verify ProcessOne downloads integrity.

For convenience, there are alternative download locations like the ejabberd DEB/RPM Packages Repository and the GitHub Release / Tags.

The ecs container image is available in docker.io/ejabberd/ecs and ghcr.io/processone/ecs. The alternative ejabberd container image is available in ghcr.io/processone/ejabberd.

If you consider that you’ve found a bug, please search or fill a bug report on GitHub Issues.

Some may have predicted it, but now it happened: the chinese government banned Monal from the chinese appstore. Below is the complete email we got from Apple regarding this ban. We got that mail twice, once on Wed, 27 Mar 2024 15:46:18 +0100 and a second time on Thu, 28 Mar 2024 17:01:19 +0100.
The macOS version of Monal is still available in the appstore and with homebrew, though.

Here is the full mail, a translation of the CAC articles can be found over here for reference.

Hello,

We are writing to notify you that your application, per demand from the CAC (Cyberspace Administration of China), will be removed from the China App Store because it includes content that is illegal in China, which is not in compliance with the App Review Guidelines:

5. Legal Apps must comply with all legal requirements in any location where you make them available (if you’re not sure, check with a lawyer). We know this stuff is complicated, but it is your responsibility to understand and make sure your app conforms with all local laws, not just the guidelines below. And of course, apps that solicit, promote, or encourage criminal or clearly reckless behavior will be rejected.

According to the CAC, your app violates Articles 3 of the Provisions on the Security Assessment of Internet-based Information Services with Attribute of Public Opinions or Capable of Social Mobilization (具有舆论属性或社会动员能力的互联网信息服务安全评估规定).

If you need additional information regarding this removal or the laws and requirements in China, we encourage you to reach out directly to the CAC (Cyberspace Administration of China).

While your app has been removed from the China App Store, it is still available in the App Stores for the other territories you selected in App Store Connect. The TestFlight version of this app will also be unavailable for external and internal testing in China and all public TestFlight links will no longer be functional.

Best regards,

App Review

A security vulnerability was intentionally added to a widely used open-source project known as ‘xz’. This project is packaged in many operating systems, and a lot of software depends upon it. The vulnerability has been assigned the identifier CVE-2024-3094.

Systems with the vulnerable package may allow an attacker to gain unauthorized access to the system via SSH, if your system’s SSH server was linked to the affected packages.

Thankfully, the vulnerability was discovered before it reached most operating systems. However if you are using a pre-release version of any Debian or Red Hat distribution, you may be affected and should install the available security updates and check for any signs of unauthorized access.

Snikket server

The Snikket server software builds upon Debian base images. We can confirm that Snikket uses the stable Debian release, and does not have the vulnerable packages.

Snikket Hosting

The Snikket Hosting platform is run on Debian servers. We also use the stable Debian release, and can confirm this vulnerability has not affected our service.

More information

Although the vulnerability does not affect Snikket itself, always ensure you install all available security updates for your host system to keep it secure.

• Debian Security Advisory
• Red Hat CVE report