Install Mastodon in Docker Swarm
Mastodon is an open-source, federated (i.e., decentralized) social network, inspired by Twitter's "microblogging" format, and used by upwards of 4.4M early-adopters, to share links, pictures, video and text.
Why would I run my own instance?
That's a good question. After all, there are all sorts of public instances available, with a range of themes and communities. You may want to run your own instance because you like the tech, because you just think it's cool
You may also have realized that since Mastodon is federated, users on your instance can follow, toot, and interact with users on any other instance!
If you're not into that much effort / pain, you're welcome to join our instance
Mastodon requirements
Ingredients
Already deployed:
- Docker swarm cluster with persistent shared storage (Alternatively, see the Kubernetes recipe here)
- Traefik configured per design
New:
- DNS entry for your epic new social network, pointed to your keepalived IP
- An S3-compatible bucket for serving media (I use Backblaze B2)
- An SMTP gateway for delivering email notifications (I use Mailgun)
- A business card, with the title "I'm CEO, Bitch"
Setup data locations
First, we create a directory to hold the Mastodon docker-compose configuration:
mkdir /var/data/config/mastodon
Then we setup directories to hold all the various data:
mkdir -p /var/data/runtime/mastodon/redis
mkdir -p /var/data/runtime/mastodon/elasticsearch
mkdir -p /var/data/runtime/mastodon/postgres
Why /var/data/runtime/mastodon and not just /var/data/mastodon?
The data won't be able to be backed up by a regular filesystem backup, because it'll be in use. We still need to store it somewhere though, so we use /var/data/runtime, which is excluded from automated backups. See Data Layout for details.
Setup Mastodon environment
Create /var/data/config/mastodon/mastodon.env something like the example below..
# This is a sample configuration file. You can generate your configuration
# with the `rake mastodon:setup` interactive setup wizard, but to customize
# your setup even further, you'll need to edit it manually. This sample does
# not demonstrate all available configuration options. Please look at
# https://docs.joinmastodon.org/admin/config/ for the full documentation.
# Note that this file accepts slightly different syntax depending on whether
# you are using `docker-compose` or not. In particular, if you use
# `docker-compose`, the value of each declared variable will be taken verbatim,
# including surrounding quotes.
# See: https://github.com/mastodon/mastodon/issues/16895
# Federation
# ----------
# This identifies your server and cannot be changed safely later
# ----------
LOCAL_DOMAIN=example.com # (1)!
# Redis
# -----
REDIS_HOST=redis
REDIS_PORT=6379
# PostgreSQL
# ----------
DB_HOST=db
DB_USER=postgres
DB_NAME=postgres
DB_PASS=tootmeupbuttercup # (2)!
DB_PORT=5432
# Elasticsearch (optional)
# ------------------------
ES_ENABLED=false # (3)!
ES_HOST=es
ES_PORT=9200
# Authentication for ES (optional)
ES_USER=elastic
ES_PASS=password
# Secrets
# -------
# Make sure to use `rake secret` to generate secrets
# -------
SECRET_KEY_BASE=imafreaksecretbaby # (4)!
OTP_SECRET=imtoosecretformysocks
# Web Push
# --------
# Generate with `rake mastodon:webpush:generate_vapid_key`
# docker run -it tootsuite/mastodon bundle exec rake mastodon:webpush:generate_vapid_key
# --------
VAPID_PRIVATE_KEY= # (5)!
VAPID_PUBLIC_KEY=
# Sending mail # (6)!
# ------------
SMTP_SERVER=smtp.mailgun.org
SMTP_PORT=587
SMTP_LOGIN=
SMTP_PASSWORD=
SMTP_FROM_ADDRESS=notifications@example.com
# File storage (optional) # (7)!
# -----------------------
S3_ENABLED=true
S3_BUCKET=files.example.com
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
S3_ALIAS_HOST=files.example.com
# IP and session retention
# -----------------------
# Make sure to modify the scheduling of ip_cleanup_scheduler in config/sidekiq.yml
# to be less than daily if you lower IP_RETENTION_PERIOD below two days (172800).
# -----------------------
IP_RETENTION_PERIOD=31556952
SESSION_RETENTION_PERIOD=31556952
- Set this to the FQDN you plan to use for your instance.
- It doesn't matter what this is set to, since we're using
POSTGRES_HOST_AUTH_METHOD=trust, but I've left it in for completeness and consistency with Mastodon's docs - Only enable this if you have enough resources for an Elasticsearch instance for full-text indexing
- Generate these with
docker run -it tootsuite/mastodon bundle exec rake secret - Generate these with
docker run -it tootsuite/mastodon bundle exec rake mastodon:webpush:generate_vapid_key - You'll need to complete this if you want to send email
- You'll need to complete this if you want to host media elsewhere
Mastodon Docker Swarm config
Create a docker swarm config file in docker-compose syntax (v3), something like the example below.. example:
Fast-track with premix! π
I automatically and instantly share (with my sponsors) a private "premix" git repository, which includes necessary docker-compose and env files for all published recipes. This means that sponsors can launch any recipe with just a git pull and a docker stack deploy π.
π Update: Premix now includes an ansible playbook, so that sponsors can deploy an entire stack + recipes, with a single ansible command! (more here)
version: '3.5'
services:
db:
image: postgres:14-alpine
networks:
- internal
healthcheck:
test: ['CMD', 'pg_isready', '-U', 'postgres']
volumes:
- /var/data/runtime/mastodon/postgres:/var/lib/postgresql/data
environment:
- 'POSTGRES_HOST_AUTH_METHOD=trust'
redis:
image: redis:6-alpine
networks:
- internal
healthcheck:
test: ['CMD', 'redis-cli', 'ping']
volumes:
- /var/data/runtime/mastodon/redis:/data
# es:
# image: docker.elastic.co/elasticsearch/elasticsearch:7.17.4
# environment:
# - "ES_JAVA_OPTS=-Xms512m -Xmx512m -Des.enforce.bootstrap.checks=true"
# - "xpack.license.self_generated.type=basic"
# - "xpack.security.enabled=false"
# - "xpack.watcher.enabled=false"
# - "xpack.graph.enabled=false"
# - "xpack.ml.enabled=false"
# - "bootstrap.memory_lock=true"
# - "cluster.name=es-mastodon"
# - "discovery.type=single-node"
# - "thread_pool.write.queue_size=1000"
# networks:
# - internal
# healthcheck:
# test: ["CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1"]
# volumes:
# - /var/data/runtime/mastodon/elasticsearch:/usr/share/elasticsearch/data
# ulimits:
# memlock:
# soft: -1
# hard: -1
# nofile:
# soft: 65536
# hard: 65536
# ports:
# - '9200:9200'
web:
image: tootsuite/mastodon
env_file: /var/data/config/mastodon/mastodon.env
command: bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000"
networks:
- internal
- traefik_public
healthcheck:
test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:3000/health || exit 1']
volumes:
- /var/data/mastodon:/mastodon/public/system
deploy:
labels:
# traefik
- traefik.enable=true
- traefik.docker.network=traefik_public
# traefikv2
- "traefik.http.routers.mastodon.rule=Host(`mastodon.example.com`)"
- "traefik.http.routers.mastodon.entrypoints=https"
- "traefik.http.services.mastodon.loadbalancer.server.port=3000"
streaming:
image: tootsuite/mastodon
env_file: /var/data/config/mastodon/mastodon.env
command: node ./streaming
networks:
- internal
- traefik_public
healthcheck:
test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:4000/api/v1/streaming/health || exit 1']
deploy:
labels:
# traefik
- traefik.enable=true
- traefik.docker.network=traefik_public
# traefikv2
- "traefik.http.routers.mastodon.rule=Host(`mastodon.example.com`) && PathPrefix(`/api/v1/streaming`))"
- "traefik.http.routers.mastodon.entrypoints=https"
- "traefik.http.services.mastodon.loadbalancer.server.port=3000"
sidekiq:
image: tootsuite/mastodon
env_file: /var/data/config/mastodon/mastodon.env
command: bundle exec sidekiq
networks:
- internal
volumes:
- /var/data/mastodon:/mastodon/public/system
healthcheck:
test: ['CMD-SHELL', "ps aux | grep '[s]idekiq\ 6' || false"]
## Uncomment to enable federation with tor instances along with adding the following ENV variables
## http_proxy=http://privoxy:8118
## ALLOW_ACCESS_TO_HIDDEN_SERVICE=true
# tor:
# image: sirboops/tor
# networks:
# - internal
#
# privoxy:
# image: sirboops/privoxy
# volumes:
# - /var/data/mastodon/privoxy:/opt/config
# networks:
# - internal
networks:
traefik_public:
external: true
internal:
driver: overlay
ipam:
config:
- subnet: 172.16.9.0/24
Note
Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you're creating/removing stacks a lot. See my list here.
Pre-warming
Unlike most recipes, we can't just deploy Mastodon into Docker Swarm, and trust it to setup its database itself. We have to "pre-warm" it using docker-compose, per the official docs (Docker Swarm is not officially supported)
Start with docker-compose
From the /var/data/config/mastodon directory, run the following to start up the Mastodon environment using docker-compose. This will result in a broken environment, since the database isn't configured yet, but it provides us the opportunity to configure it.
docker-compose -f mastodon.yml up -d
The output should look something like this:
root@raphael:/var/data/config/mastodon# docker-compose -f mastodon.yml up -d
WARNING: Some services (streaming, web) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
WARNING: The Docker Engine you're using is running in swarm mode.
Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.
To deploy your application across the swarm, use `docker stack deploy`.
Creating mastodon_sidekiq_1 ... done
Creating mastodon_db_1 ... done
Creating mastodon_redis_1 ... done
Creating mastodon_streaming_1 ... done
Creating mastodon_web_1 ... done
root@raphael:/var/data/config/mastodon#
Create database
Run the following to create the database. You can expect this to take a few minutes, and produce a lot of output:
cd /var/data/config/mastodon
docker-compose -f mastodon.yml run --rm web bundle exec rake db:migrate
Create admin user
Next, decide on your chosen username, and create your admin user:
cd /var/data/config/mastodon
docker-compose -f mastodon.yml run --rm web bin/tootctl accounts \
create <username> --email <email address> --confirmed --role admin
The password will be output on completion1:
root@raphael:/var/data/config/mastodon# docker-compose -f mastodon.yml run --rm web bin/tootctl accounts create batman --email batman@batcave.org --confirmed --role admin
WARNING: Some services (streaming, web) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
OK
New password: c6eb8e0d10cd6f0aa874b7a384177a08
root@raphael:/var/data/config/mastodon#
Turn off docker-compose
We've setup the essestials now, everything else can be configured either via the UI or via the .env file, so tear down the docker-compose environment with:
docker-compose -f mastodon.yml down
The output should look like this:
root@raphael:/var/data/config/mastodon# docker-compose -f mastodon.yml down
WARNING: Some services (streaming, web) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
Stopping mastodon_streaming_1 ... done
Stopping mastodon_web_1 ... done
Stopping mastodon_db_1 ... done
Stopping mastodon_redis_1 ... done
Stopping mastodon_sidekiq_1 ... done
Removing mastodon_streaming_1 ... done
Removing mastodon_web_1 ... done
Removing mastodon_db_1 ... done
Removing mastodon_redis_1 ... done
Removing mastodon_sidekiq_1 ... done
Removing network mastodon_internal
Network traefik_public is external, skipping
root@raphael:/var/data/config/mastodon#
Launch Mastodon!
Launch the Mastodon stack by running:
docker stack deploy mastodon -c /var/data/config/mastodon/mastodon.yml
Now hit the URL you defined in your config, and you should see your beautiful new Mastodon instance! Login with your configured credentials, navigate to Preferences, and have fun tweaking and tooting away!
Once you're done, "toot" me by mentioning funkypenguin@so.fnky.nz in a toot! 
Tip
If your instance feels lonely, try using some relays to bring in the federated firehose!
Summary
What have we achieved? Even though we had to jump through some extra hoops to setup database and users, we now have a fully-swarmed Mastodon instance, ready to federate with the world!
Summary
Created:
- Mastodon configured, running, and ready to toot!
Chef's notes π
-
Or, you can just reset your password from the UI, assuming you have SMTP workingΒ β©
Tip your waiter (sponsor) π
Did you receive excellent service? Want to compliment the chef? (..and support development of current and future recipes!) Sponsor me on Github / Ko-Fi / Patreon, or see the contribute page for more (free or paid) ways to say thank you! π
Employ your chef (engage) π€
Is this too much of a geeky PITA? Do you just want results, stat? I do this for a living - I'm a full-time Kubernetes contractor, providing consulting and engineering expertise to businesses needing short-term, short-notice support in the cloud-native space, including AWS/Azure/GKE, Kubernetes, CI/CD and automation.
Learn more about working with me here.
Flirt with waiter (subscribe) π
Want to know now when this recipe gets updated, or when future recipes are added? Subscribe to the RSS feed, or leave your email address below, and we'll keep you updated.