Docker Swarm mode, Traefik and Gitlab - Part 1
Being able to develop an application of any kind and automatically deploy it is the norm nowadays. I have been using dokku in my personal deployment stack for several years now. And at work, we are using Kubernetes. I love the idea of Kubernetes but wanted to try something else, and came across docker swarm mode. In this blog post series, I’m gonna describe the process of setting up a docker swarm cluster, putting traefik in front as a reverse proxy and automatically deploying via gitlab without downtime.
Because this is a big topic, I won’t be able to cover it all in one post. I’m gonna split it into 2 parts.
- First part (this one) will cover the setup of a docker swarm cluster and traefik.
- The next part will cover how to integrate with gitlab and automatically deploy without downtime.
Some clarification before we start, when I use the term “docker swarm”, I mean “running docker engine in swarm mode”. Docker swarm is a separate product and not in active development anymore. I also use the old tool
docker-machine to provision virtualbox for demonstration purposes, the new Docker Desktop should be used instead of you are using Mac or Windows.
Set up a docker swarm cluster⌗
First, we need 3 servers, I will use 1 as the
manager and the other 2 are
worker. I’m gonna use
virtualbox to create them locally.
docker-machine create --driver virtualbox local-manager docker-machine create --driver virtualbox local-worker-1 docker-machine create --driver virtualbox local-worker-2
Let’s verify that they are created correctly
docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM DOCKER local-manager - virtualbox Running tcp://192.168.99.100:2376 v19.03.5 local-worker-1 - virtualbox Running tcp://192.168.99.101:2376 v19.03.5 local-worker-2 - virtualbox Running tcp://192.168.99.102:2376 v19.03.5
Great, they are all up and running, now we need to ssh into the manager node and initialize the swarm
docker-machine ssh local-manager docker swarm init --advertise-addr 192.168.99.100
After running that command, docker will show us how workers node can join the swarm
docker swarm join --token some-token 192.168.99.100:2377
Now run that in the other 2 worker nodes, and after that we can verify that we have everything set up correct by running
docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION p1mx...* local-manager Ready Active Leader 19.03.5 sqvu... local-worker-1 Ready Active 19.03.5 p5kj... local-worker-2 Ready Active 19.03.5
Awesome! now let’s deploy something, in the manager node, put this
version: "3.7" services: whoami: image: jwilder/whoami:latest ports: - 8000:8000 deploy: replicas: 6
Then run the deploy command. Only the manager node can deploy stuff to the cluster
docker stack deploy -c whoami.yml test
Wait few seconds for the deployment, after that you should be able to use
docker service ls to see the current service(s)
ID NAME MODE REPLICAS IMAGE PORTS qo... test_whoami replicated 6/6 jwilder/whoami:latest *:8000->8000/tcp
There are 2 deployment modes:
- replicated mode deploys the image in many containers. We can also specify constraints to tell swarm where to deploy stuff. For example, in production, we might have 1 server dedicated for background job, we can tell swarm to deploy background job image to that particular server through constraints
- global mode deploys exactly 1 container per each node in the swarm cluster. This also abides by the constraints
I don’t want to go too much into details because docker’s documentation has everything. Moving on, now that we have the image ready, we can visit any IP address of any node at port 8000 to access
whoami (which btw just prints the container ID). Swarm will automatically send our request to an available node.
Imagine that if this is our application, we would want to push the image to a docker registry, and then after we have published the new image tag, we can run
docker stack deploy to roll out the update. However, there are 2 problems here
- Due to the exposed port (8000), anyone can access any node using that port. We probably don’t want that. We need one single reverse proxy to hide all the internal port.
- Our update process is not “zero downtime”. The default update config of docker swarm is
stop-firstwhich means old nodes are stopped first before starting new nodes. This is not desired because we might not be able to start new nodes due to reasons.
I will cover those 2 points in the next part.
Meet traefik, a reverse proxy⌗
I definitely don’t choose traefik because of its funny name
Now that we have our simple whoami service running, we want to close its internal port from the outside world and instead only expose the reverse proxy (traefik) to handle all the public traffic. Traefik will then pick the appropriate server to redirect the incoming requests.
In the manager node, create a
proxy.yml file with the following content and run the deployment command
version: "3.7" services: traefik: image: traefik:v2.2 command: - --api=true # enable the management api - --api.dashboard=true # enable the monitoring dashboard - --api.insecure=true # allow insecure access to the dashboard - --providers.docker=true # use docker - --providers.docker.swarmMode=true # in swarm mode - --providers.docker.exposedbydefault=false # but don't pick up services automatically - --entrypoints.web.address=:80 # define `web` entry point listening at port 80 ports: - 80:80 - 8080:8080 # dashboard volumes: # must mount the docker socket so that traefik can listen to changes - /var/run/docker.sock:/var/run/docker.sock deploy: # this basically says that only deploy 1 per node in every manager node # and reserve 128MB of ram to it, also limit the memory to 256MB mode: global placement: constraints: - "node.role == manager" resources: reservations: memory: 128M limits: memory: 256M
Then, verify that we have it up and running
docker service ls ID NAME MODE REPLICAS IMAGE PORTS qo... test_whoami replicated 6/6 jwilder/whoami:latest *:8000->8000/tcp av... proxy_traefik global 1/1 traefik:v2.2 *:80->80/tcp
Great! We can now access traefix via any node’s IP. I will assign the manager’s IP to
docker-swarm.local, simply add this to
But instead of the container id, we have this…
Well, we have just set up traefik, we haven’t tell it to which node it should send the traffic to. This can be verified by accessing the dashboard. We only have the internal stuff running at the moment.
whoami.yml to tell traefik that it should send traffic to this service
version: "3.7" services: whoami: image: jwilder/whoami:latest # ports: # - 8000:8000 we don't have to expose the internal port anymore deploy: replicas: 6 labels: # the most import label to tell traefik that it should pick up this service - "traefik.enable=true" # by default, traefik picks up the first exposed port, we can explicitly set it # to something else here - "traefik.http.services.whoami.loadbalancer.server.port=8000" # tell traefik to send all requests to `docker-swarm.local` to this service - "traefik.http.routers.whoami.rule=Host(`docker-swarm.local`)" # the default entry point is `web` which is HTTP - "traefik.http.routers.whoami.entrypoints=web"
Run the deploy command again
docker stack deploy -c whoami.yml test
Now we should see this with
docker service ls
ID NAME MODE REPLICAS IMAGE PORTS av... proxy_traefik global 1/1 traefik:v2.2 *:80->80/tcp ws... test_whoami replicated 6/6 jwilder/whoami:latest
Now let’s try again… Hey! it still doesn’t work what the heck
“Gateway Timeout” usually means that traefik can’t communicate with the docker containers it pick up. Let’s check the networks.
docker network ls NETWORK ID NAME DRIVER SCOPE 9e1475d38cdf bridge bridge local 0fd5af803547 docker_gwbridge bridge local 6ae776ce9f3e host host local n9or85idy165 ingress overlay swarm 7e699eddc6c1 none null local ua27p80ckoka proxy_default overlay swarm 3l6cuiaqx1tb test_default overlay swarm pxhs290h4t41 test_proxy_default overlay swarm
Apparently, each stack has its own network in addition to
ingress which is the default network for the whole cluster. We need to make them all connect to the same network. We can either re-use one of the existing network or create a new one. Let’s create a new one
docker network create --driver=overlay --attachable whoami
After we have created a new network, we need to put traefik into the same network, otherwise it can’t talk to the docker containers
services: traefik: image: traefik:v2.2 networks: - whoami # the rest of the config networks: whoami: external: true
And then we need to tell
whoami to use the new network instead of its default one
services: whoami: image: jwilder/whoami:latest networks: - whoami # the rest of the config networks: whoami: external: true
docker stack deploy -c proxy.yml proxy docker stack deploy -c whoami.yml test
To re-deploy everything, and check
docker-swarm.local again, we should be able to see the container ID
The dashboard also shows proper containers (all 6 of them!)
And that’s everything we need to set up a swarm cluster and traefik. However this is just a basic setup, we still need to properly integrate it with a CI/CD service (gitlab for example) and make it whenever we deploy something, there will be no downtime. I will cover those 2 points in the next part.