Hosting your own Matrix Synapse server using Ansible

Posted on Sep 11, 2021

Why

There are many reasons for hosting your own matrix server. Here are some of them:

Control of your data: Maybe you need something similar to closed source platforms like Microsoft Teams or Slack, but you don’t want them to have all your personal data.
Bridges: Matrix has the ability to set up bridges to other platforms like WhatsApp, Telegram, Microsoft Teams and many others.
WhatsApp alternative: Matrix has a lot of functionalities, but you can just tell your friends to switch to your Matrix instance (or any other instance, thanks to its federation capabilities) and stop using the Facebook proprietary software for your personal and private communications.
Learning purposes: Why not? Just try it.

How

There is an official ansible playbook for deploying the full Synapse server. You can follow its great documentation using this link. There is a problem though: the default setup works if you have only matrix and no other services/reverse proxy installed in your server.

DNS settings

First of all, you will need a domain. Depending on your use case, you will need to create three or more DNS records for your domain (check the documentation). Here are the DNS records I have set up:

matrix: CNAME pointing to my server name.
element: CNAME pointing to my server name.
_matrix-identity._tcp: SRV record for my front facing proxy.
_matrix._tcp: SRV record port 8448 for federation and Synapse server delegation.

Hosts

Let’s start making changes to the ansible related files (following the documentation):

You have to create a directory inventory/hostvars/<host>. The host part is how you name the matrix server on your Ansible inventory. In my case host equals matrix.
Move the vars to the directory created.
My setup will be a bit more complicated since I will be dedicating the matrix server only to the matrix services. The reverse proxy will be configured in another server, which will be the one obtaining the certificates. This is why my vars file looks like this:

# The bare domain name which represents your Matrix identity.
# Matrix user ids for your server will be of the form (`@user:<matrix-domain>`).
#
# Note: this playbook does not touch the server referenced here.
# Installation happens on another server ("matrix.<matrix-domain>").
#
# If you've deployed using the wrong domain, you'll have to run the Uninstalling step,
# because you can't change the Domain after deployment.
#
# Example value: example.com
matrix_domain: example.com

# A shared secret (between Coturn and Synapse) used for authentication.
# You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`).
matrix_coturn_turn_static_auth_secret: 'pwgen -s 64 1'

# A secret used to protect access keys issued by the server.
# You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`).
matrix_synapse_macaroon_secret_key: 'pwgen -s 64 1'

# A Postgres password to use for the superuser Postgres user (called `matrix` by default).
#
# The playbook creates additional Postgres users and databases (one for each enabled service)
# using this superuser account.
matrix_postgres_connection_password: 'pwgen -s 64 1'

# Do not retrieve SSL certificates. This shall be managed by another webserver or other means.
matrix_ssl_retrieval_method: none

# Do not try to serve HTTPS, since we have no SSL certificates.
# Disabling this also means services will be served on the HTTP port
# (`matrix_nginx_proxy_container_http_host_bind_port`).
matrix_nginx_proxy_https_enabled: false

# Coturn relies on SSL certificates that have already been obtained.
# Since we don't obtain any certificates (`matrix_ssl_retrieval_method: none` above), it won't work by default.
# An alternative is to tweak some of: `matrix_coturn_tls_enabled`, `matrix_coturn_tls_cert_path` and `matrix_coturn_tls_key_path`.
matrix_coturn_enabled: false

Configure your hosts file. Mine looks like this:

[matrix_servers]
matrix

I can do it this way because I have my .ssh/config configured this way:

Host matrix
    HostName IP
    User user
    Port port

Installing

Deploy the matrix services using the playbook:

ansible-playbook -i inventory/hosts setup.yml --tags=setup-all

Start the matrix services using the playbook:

ansible-playbook -i inventory/hosts setup.yml --tags=start

Reverse proxy

As I said before, my reverse proxy will be on another server (known as front end from now on). I had to make configurations on three different servers for this to work.

Front end

This server will be in charge of reverse proxying all matrix requests. It will also need to abtain a certificate for HTTPS to work.

I use nginx as my reverse proxy. This is the site I have configured for matrix:

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name matrix.*;

    # Your SSL settings
    include /config/nginx/ssl.conf;

    location /.well-known/matrix {
		proxy_pass http://<matrix-server-ip>:80/.well-known/matrix;
		proxy_set_header X-Forwarded-For $remote_addr;
    }


    location / {

        # Your proxy settings
        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        set $upstream_proto http;
        proxy_pass $upstream_proto://<matrix-server-ip>:80;

    }
}
server {
    listen 8448 ssl default_server;
    listen [::]:8448 ssl default_server;
    server_name matrix.*;

    include /config/nginx/ssl.conf;
    include /config/nginx/proxy.conf;
    location / {
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_pass http://<matrix-server-ip>:8448;
    }
}

Finally, this is the site configured for Element, the matrix client:

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name element.*;

    include /config/nginx/ssl.conf;

    location / {
        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        set $upstream_proto http;
        proxy_pass $upstream_proto://<matrix-server-ip>:80;

    }
}

After that, you can get your LetsEncrypt certifcate as usual executing:

certbot certonly --cert-name domain.example --webroot -w /srv/www/domain.example/html -d matrix.domain.example -d element.domain.example

Matrix proxy

You will need to make some manual configuration on your matrix server. This is the part that the ansible installation playbook does not cover and you need to figure out.

First, edit /matrix/nginx-proxy/conf.d/matrix-domain.conf. By default, it will listen for requests to matrix.example.com. When the front end nginx server forwards the petition, it will be the matrix-server-ip, so the matrix server will not process the request. That is why we need to make changes.

In the first server block:

Change the line listen 8080; to listen 8080 default_server;
Change the line server_name matrix.example.org to server_name _;

In the second server block:

Change the line listen 8448; to listen 8448 default_server;
Change the line server_name matrix.example.org to server_name _;

After that, restart the matrix proxy:

docker restart matrix-nginx-proxy

Base domain proxy

In my case, the installation is even more complicated because I have my base domain pointed to yet another different server. Matrix uses the base domain to access federation files. This problem is also covered on the main documentation.

For fixing this problem, I had to add the following to the nginx configuration on the server pointed by my base domain (that being example.org ):

location /.well-known/matrix {
		proxy_pass https://matrix.example.org/.well-known/matrix;
		proxy_set_header X-Forwarded-For $remote_addr;
	}

This means that a request to this subdirectory of my base domain webserver will be then forwarded to my front end server and then finally to the matrix one.

Debugging

Finally, you can check that everything is working using this command:

ansible-playbook -i inventory/hosts setup.yml --tags=self-check

If the check fails, it will do so in a particular step and will give you a little feedback, helping you troubleshoot any misconfigurations.

Creating users

You can check the official documentation for registering users.

For manually create a user, you can execute the following:

ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=<your-username> password=<your-password> admin=<yes|no>' --tags=register-user