How to set up a fully compliant XMPP server from scratch

2022-04-21

This is the English version of the guide I posted in the Linux User forum. This version also fixes some errors I made in the original version.

Introduction

In this guide I'll show you how to set up a fully compliant XMPP server step by step, from scratch. In contrast with other guides that explain the basics for the setup of a minimal server, this one aims to help new sysadmins to set up an XMPP server that complies up to 100% to the XMPP protocol.

There are a few XMPP servers you can pick from. This guide focuses on Prosody, which is one of the most popular options for an XMPP server, and it's quite flexible. Moreover, these instructions are given for servers that run Debian (or Debian-based distributions). If you use another distribution for your server, use the appropriate commands for your distribution and refer to the documentation of the software you are going to use.

What is XMPP

XMPP (also known as Jabber) is a protocol for instant messaging. Compared with other IM services, XMPP is decentralized with lots of servers running appropriate software than similar services which are based on a centralized server.

What is needed for the server

To set up the XMPP server, the following are required:

A server (an older computer or a basic VPS is enough)
A domain, because some DNS records must be created
An SSL certificate, or more, depending on your server's settings. If you don't already have a certificate, you can generate a certificate with Certbot.
A web server. It won't be needed for XMPP itself, but for some modules that require it. In this guide Nginx will be used.
And Prosody, of course.

Installation and configuration of Prosody

To install Prosody, run sudo apt install prosody prosody-modules mercurial.

By running it, Prosody will be downloaded with some modules, which will be very useful later. mercurial will be used later to download some additional community modules that will be needed.

To download the community modules, run hg clone https://hg.prosody.im/prosody-modules/ prosody-modules.

(Personally, I have the prosody-modules directory at /usr/lib/prosody/modules/ for convenience. You can put it in whatever directory you want to.)

To set up prosody, edit the /etc/prosody/prosody.cfg.lua file as root with the text editor of your choice.

The following should be configured:

Define the user(s) who will have administration rights by giving their JIDs. You can add more users if you need to.
admins = { "username@domain.tld" }
Define the plugin paths for the modules that were downloaded earlier. You can set the modules path, as well as the prosody-modules directory.
plugin_paths = { "/usr/lib/prosody/modules", "/dir/of/prosody-modules" }

Define the modules you want to be enabled. Some of them are already enabled by default.

modules_enabled {
  -- Generally required
    "roster";
    "saslauth";
    "tls";
    "dialback";
    "disco";

  -- Not essential, but recommended
    "carbons";
    "pep";
    "private";
    "blocklist";
    "vcard4";
    "vcard_legacy"

  -- Nice to have
    "version";
    "uptime";
    "time";
    "ping";
    "register";
    "mam";
    "csi_simple";

  -- Admin interfaces
    "admin_adhoc";
    --"admin_telnet";

  -- HTTP modules
    "bosh";
    --"websocket";
    "http_files";

  -- Other specific functionality
    "posix";
    [...]
    "proxy65";

  -- Add if you have downloaded the community modules
    "cloud_notify";
    "smacks";
    "turncredentials";
    "vcard_muc";
    "external_services";
    "bookmarks";
    "server_contact_info";
    "http_upload_external";
}

(Note: in the Lua programming language, the two dashes at the beginning of a line declare the specific line as a comment.)

Declare if users can register in the server.
allow_registration = false
If the server will be public and you want to allow users to register themselves, you can set this value to true.
Set up authentication
authentication = "internal_hashed"
Set up storage
storage = "internal"
You can use an SQL database if you want to.
Set up the certificates directory
certificates = "certs"
This line declares that the certificates can be found at /etc/prosody/certs.

Set up VirtualHost with the domain and the subdomains needed

VirtualHost "domain.tld"
  ssl = {
    key = "certs/domain.tld.key"
    certificate = "certs/domain.tld.key"
  }
  disco_items = {
    { "upload.domain.tld", "File upload" };
    { "muc.domain.tld", "MUC" };
  }
}

Add some settings for BOSH (useful to connect to your server from web clients)

consider_bosh_secure = true;
cross_domain_bosh = true;
https_ssl = {
  certificate = "/etc/letsencrypt/live/domain.tld/fullchain.pem";
  key = "/etc/letsencrypt/live/domain.tld/privkey.pem";
}

Add contact info

contact_info = {
  abuse = { "mailto:abuse@domain.tld" };
  admin = { "mailto:admin@domain.tld" };
  feedback = { "mailto:feedback@domain.tld" };
}

Add the components needed

MUC:

Component "muc.domain.tld" "muc"
  restrict_room_creation = false
  modules_enabled {
    "vcard_muc",
    "muc_mam",
  }

Uploads:

Component "upload.domain.tld" "http_upload_external"
  http_upload_external_base_url = "https://upload.domain.tld/"
  http_upload_external_secret = "secret"
  http_upload_external_file_size_limit = 104857600 -- 100 Mib

Add the external service configuration for the turnserver (for audio and video calls through a compatible XMPP client):

external_services = {
  {
    type = "stun",
    transport = "udp",
    host = "turn.domain.tld",
    port = 3478
  }, {
    type = "turn",
    transport = "udp",
    host = "turn.domain.tld",
    port = 3478,
    secret = "secret"
  }
}

Save the file.
Create the admin account by running sudo prosodyctl adduser username@domain.tld.
With the same command we can create normal accounts as well.
If a certificate already exists, run the command sudo prosodyctl -root cert import /etc/letsencrypt/live. If there is not any, create one with certbot.
At the DNS configuration settings for the domain, add the _xmpp and _xmpps SRV records, as they are mentioned here. You can also make SRV records for the subdomains you need, like for MUC, for example (the configuration for uploads and the turnserver will be mentioned below). It may take a few minutes for the records to be deployed. Also forward the ports (TCP only) on your router and configure your firewall accordingly, if it is enabled.

If the steps above are done, enable the prosody service and start it with the following commands:
sudo systemctl enable prosody
sudo systemctl start prosody

After starting Prosody, try to connect with your JID and your password to an XMPP client (I suggest using Dino on desktop and Conversations on Android). If you can connect to your server, it means that everything went alright. Otherwise, run the sudo prosodyctl check command. It's very helpful to troubleshoot any issues on Prosody.

If your server is functional, You might want to check compliance.conversations.im, where you can see your server's compliance with the XMPP protocol. It's suggested to make a testing account, which can also be useful later.

Upload server configuration

In order to upload files on the XMPP server, some things have to be configured first.

As you can see above, the upload component and the http_upload_external module were added in the configuration file. That module has a few implementations, as you can see here. In this guide, Prosody Filer (which is the implementation in Go) will be used, as it is the easiest to configure.

First, install golang on your system with the sudo apt install golang command.
Clone the Prosody Filer repository by running git clone https://github.com/ThomasLeister/prosody-filer.
Go to the prosody-filer directory, and run the build.sh script to compile Prosody Filer.
Copy prosody-filer and config.example.toml to a directory of choice, for example, in /var/www/upload.
Rename config.example.toml to config.toml and edit it like this:
```
listenport = "[::]:5050"
secret = "secret"
storeDir = "/var/www/upload/uploads/"
uploadSubDir = ""
```
The secret must be the same that was set up in Prosody's configuration, in the upload component.

Create the systemd service for Prosody Filer at the /etc/systemd/system/prosody-filer.service directory and write the following to it:

[Unit]
Description=Prosody file upload server

[Service]
Type=simple
ExecStart=/var/www/upload/prosody-filer
Restart=always
WorkingDirectory=/var/www/upload

[Install]
WantedBy=multi-user.target

Run the following commands to start Prosody Filer: sudo systemctl reload-daemon sudo systemctl enable prosody-filer sudo systemctl start prosody-filer You can check if Prosody Filer is running with the sudo systemctl status prosody-filer command.

Create a configuration file at /etc/nginx/sites-available/xmpp-upload and write the following to it:

server {
  listen 80;
  listen [::]:80;
  listen 443 ssl;
  listen [::]:443 ssl;

  server_name uploads.domain.tld;

  ssl_certificate /etc/letsencrypt/live/uploads/domain.tld/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/uploads/domain.tld/privkey.pem;

  client_max_body_size 50m;

  location /upload/ {
    if ( $request_method = OPTIONS ) {
      add_header Access-Control-Allow-Origin '*';
      add_header Access-Control-Allow-Methods 'PUT, GET, OPTIONS, HEAD';
      add_header Access-Control-Allow-Headers 'Authorization, Content-Type';
      add_header Content-Length 0;
      add_header Content-Type text/plain;
      return 200;
    }
    proxy_pass http://[::]:5050/upload/;
      proxy_request_buffering off;
  }
}

Create a symlink of this configuration by running: sudo ln -s /etc/nginx/sites-available/xmpp-upload /etc/nginx/sites-enabled.
Enable and start Nginx by running the following commands: sudo systemctl enable nginx sudo systemctl start nginx

If there are no errors, restart Prosody by running sudo systemctl restart prosody. Then you can check if file uploading works from your XMPP client.

Turnserver configuration for audio and video calls

To improve your XMPP server a bit, you can easily set up a turnserver to be able to make and accept voice and video calls from friends.

First, install Coturn by running sudo apt install coturn.
Change the TURNSERVER_ENABLED value in the /etc/default/coturn file from 0 to 1.
Enable Coturn by running sudo systemctl enable coturn and sudo systemctl start coturn.
Make the following changes in the /etc/turnserver.conf (uncommenting the following options):
```
listening-port=3478
listening-ip=0.0.0.0
external-ip=[Your external IP address]
min-port=49152
max-port=65535
static-auth-secret=secret
server-name=turn.domain.tld
user=test:test123
realm=domain.tld
```
The user option here is useful in case you want to test the turnserver. Also, the static-auth-secret must be the same with the secret that has been set at Prosody's configuration, similarly to upload's secret.
Restart Coturn by running sudo systemctl restart coturn.
Forward the 3478 port on your router (TCP and UDP) and set up your firewall accordingly, if you have that enabled.
Create an A record on the DNS configuration settings with the host set as turn.domain.tld with value 127.0.0.1 and TTL set as 3600 (or automatically).

You can restart Coturn and Prosody and try the calls function with another user, like the testing one, as I mentioned that earlier in the guide.

Finishing the setup

If the set up process didn't have any issues, it means that your server is ready and probably complies fully with the XMPP protocol. But XMPP's capabilities don't stop there, as you can improve your XMPP server with lots of extensions that you can try for your needs.

gettie's space