Adding Nginx and HTTPS via Letsencrypt to Docker Compose setup

I have a website served by a Docker host. It is setup to allow deployment on the remote host with a single command:

$ docker-compose --context=remote up

But at the moment this site only runs http and I want https. There are quite a few steps to this, and I get help from a few different places. So this will serve as my notes for whenever I need to perform this setup process again.

This is what we need to do:

• It is assumed that DNS records point to the Docker host.
• Setup reverse proxy application Nginx Certbot on the host VM.
• Initialize the Letsencrypt certificate on the server.
• Setup networking between the proxy containers and the application containers.
• Launch the proxy containers from the host VM.
• Launch the application containers from the development machine through docker context.

Three of my sources for this article refer to the same project. Nginx Certbot is the name of a bundle with a Docker Compose-setup and a shell script for automating the ssh certification renewal process. I had no idea how any of this worked until yesterday, and if any of this helps anyone, it’s all thanks to Philipp’s work.

To install the reverse proxy, log in to the host VM and clone the repo:

$ shh user$remoteip
$ git clone https://github.com/wmnnd/nginx-certbot.git
$ cd nginx-certbot

Now the readme states the following:

Modify configuration:

• Add domains and email addresses to init-letsencrypt.sh
• Replace all occurrences of example.org with primary domain (the first one you added to init-letsencrypt.sh) in data/nginx/app.conf

Go ahead and do that, except for the very last occurrence of example.com in line 12 of data/nginx/app.conf:

server {
    listen 443 ssl;
    server_name example.org;
    server_tokens off;

    ssl_certificate /etc/letsencrypt/live/example.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.org/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    location / {
        proxy_pass  http://example.org; <----- this one
        proxy_set_header    Host                $http_host;
        proxy_set_header    X-Real-IP           $remote_addr;
        proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    }
}

See, what this piece of configuration does is it provides a proxy for the address in the proxy_pass. That means that if we switch this out, and write our own address, then we’ll end up looping back around to the http version of our website (which is itself a redirect). So leave that as it is for now.

Having finished configurations, go ahead and run ./init-letsencrypt.sh. This script does a few things. First it creates a dummy certificate, then it requests an update to this certificate, then it authenticates with Letsencrypt, then it overwrites the dummy certificate.

Next, with that out of the way, we actually do have to edit the proxy_pass parameter in the app.conf-file. Go ahead and change it to myapp:5000.

    location / {
        proxy_pass  http://myapp:5000/;
        proxy_set_header    Host                $http_host;
        proxy_set_header    X-Real-IP           $remote_addr;
        proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    }

Then, on the host VM as well as the development machine, create the Docker network my_network:

$ docker network create my_network

On the host machine, add this network to the compose-file:

$ diff --git a/docker-compose.yml b/docker-compose.yml
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -12,8 +12,6 @@ services:
       - "80:80"
       - "443:443"
     command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'"
+    networks:
+      - my_network
   certbot:
     image: certbot/certbot
     restart: unless-stopped
@@ -21,9 +19,3 @@ services:
       - ./data/certbot/conf:/etc/letsencrypt
       - ./data/certbot/www:/var/www/certbot
     entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"
+    networks:
+      - my_network
+
+networks:
+  my_network:
+    external: true

On the development machine, likewise, add the network to the compose-file:

--- a/docker-compose.yml
+++ b/docker-compose.yml
version: '3.7'

services:
  myapp:
    environment:
      PYTHONUNBUFFERED: 1
      LOGLEVEL: INFO
    build:
      context: ./app
      dockerfile: Dockerfile
    ports:
       - 5000:5000
     container_name: app
+    networks:
+      - my_network
   worker:
     environment:
       PYTHONUNBUFFERED: 1
       LOGLEVEL: INFO
     build:
       context: ./worker
       dockerfile: Dockerfile
     ports:
       - 5001:5000
     container_name: worker
+    networks:
+      - my_network
   redis:
     image: 'redis:5.0.6-alpine'
     container_name: redisinstance
     command: redis-server
     ports:
       - '6379:6379'
+    networks:
+      - my_network
+
+networks:
+  my_network:
+    external: true

Next time we launch these two compose files on the Docker host, we get two sets of containers running on the same machine. They are not part of the same bundle, but thanks to the network linking they are still able to talk to each other. We could also go more granular and put the backend on its own network to separate concerns.

To recap: We set the proxy_pass to myapp:5000, where myapp is the name of our entry point in the application’s compose-file, and port 5000 is the port that the app is exposed on within the Docker network. The network itself is usually automatically created, but because we need the environment from two separate compose-files to communicate with each other, we need to create an external network shared between the two before we can refer to myapp from the reverse proxy.

With all that set up, it is time to try and deploy both projects on the host. Run up on both machines:

$ docker-compose up -d

And the site is live. The ssl certificates makes it possible to serve the site through https, and the ssl certificate itself gets automatically renewed through Letsencrypt.

Sources:

• https://docs.docker.com/compose/reference/up/
• https://gabrieltanner.org/blog/docker-compose
• https://medium.com/@pentacent/nginx-and-lets-encrypt-with-docker-in-less-than-5-minutes-b4b8a60d3a71
• https://github.com/wmnnd/nginx-certbot
• https://github.com/wmnnd/nginx-certbot/issues/58#issuecomment-579695486