HTTPS with NGINX

QR Code linking to this page

Written by David Evans, first published (last updated )

NGINX makes setting up HTTPS very easy (especially when combined with certbot), but some optional features can be a bit trickier to set up. Here we’ll focus on hosting multiple sites with different certificates, and setting up TLS Session Resumption.

The basics

First, the basics. To set up nginx with HTTPS there are a few simple initial steps:

  1. Generate private parameters for Diffie-Hellman key exchanges. Using Diffie-Hellman for the initial key exchange means that the per-connection encryption keys will remain secret, even if an attacker compromises the certificate’s private key (this is called Perfect Forward Secrecy):

    openssl dhparam -out dhparam.pem 2048;
    sudo mv dhparam.pem /etc/nginx/dhparam.pem;
    sudo chmod 0600 /etc/nginx/dhparam.pem;
    sudo chown root:root /etc/nginx/dhparam.pem;
    
  2. Configure nginx (in the http block, not inside individual server blocks; we’ll see why below) with a set of secure ciphers and the generated parameters. This is typically done by adding a new file to /etc/nginx/conf.d/:

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_ciphers HIGH:!aNULL:!MD5:!PSK:!CAMELLIA:!SHA1:!SHA256:!SHA384:!AES128:!ARIA128:!AES256-GCM-SHA384:!AES256-CCM8:!AES256-CCM:!ARIA256-GCM-SHA384;
    ssl_dhparam /etc/nginx/dhparam.pem;
    

    Here we start with the HIGH suite of ciphers and turn off the known insecure ones. Another option is to use a tool like TLS Configurator to get a specific set of allowed ciphers based on the versions of tools being used.

  3. Configure a listener for HTTP traffic to serve acme-challenge files (required if using certbot and its webroot verification to get a certificate), and redirect other requests to HTTPS:

    server {
      listen 80; # or use a non-privileged port and map it using iptables or nftables
      listen [::]:80;
      root /var/www/http;
    
      location / {
        return 301 https://$host$request_uri;
      }
    
      location /.well-known/acme-challenge/ {
      }
    }
    
    nginx -s reload
    
  4. Obtain a certificate from certbot (here using the simple webroot verifier):

    certbot certonly \
      --non-interactive \
      --agree-tos \
      --register-unsafely-without-email \
      --keep-until-expiring \
      --cert-name my-cert \
      --webroot \
      -w /var/www/http \
      -d example.com;
    
  5. Configure a listener for the site to be served via HTTPS:

    server {
      listen 443 ssl; # or use a non-privileged port and map it using iptables or nftables
      listen [::]:443 ssl;
    
      ssl_certificate /etc/letsencrypt/live/my-cert/fullchain.pem;
      ssl_certificate_key /etc/letsencrypt/live/my-cert/privkey.pem;
    
      # (rest of site config here)
    }
    
    nginx -s reload
    

Now the site is being served over HTTPS. By adding more server blocks and setting explicit server_names, it’s possible to include multiple sites, and they can either share a certificate which has multiple domains (add more -d flags to the certbot call), or each use their own certificate (call certbot multiple times with different --cert-name values to get a certificate for each site).

I like to use SSLLabs to test the resulting setup, since it can flag insecure cipher choices as well as many other configuration issues.

TLS Session Resumption

Our current setup requires clients to negotiate a new encrypted channel every time they connect. The process looks like this:

title HTTPS Connections; begin TLS 1.2 Client as C2; begin Server as S2; begin TLS 1.3 Client as C3; begin "Server " as S3; # TLS 1.2; C2 -> +S2: SYN; -S2 -> +C2: ACK + SYN; -C2 -> +S2: "ACK + Client Hello\n+ Domain name"; -S2 -> +C2: "Server Hello\n+ Certificate"; state over C2: Check certificate; -C2 -> +S2: "Premaster Secret\n+ Finished"; & note right of S2: "Premaster Secret is encrypted\nusing the Server's public key."; -S2 -> C2: Finished; simultaneously:; # TLS 1.3; C3 -> +S3: SYN; -S3 -> +C3: ACK + SYN; -C3 -> +S3: "ACK + Client Hello\n+ Domain name"; -S3 -> +C3: "Server Hello + Certificate\n+ Signature + Finished"; state over C3: Check certificate & signature; -C3 -> S3: "Client DH Parameter\n+ Finished"; divider line: Encrypted channel established; C2 -> +S2: Request; -S2 --> C2: Response; C2 -> +S2: Request; -S2 --> C2: Response; text right of C2: etc.; simultaneously:; C3 -> +S3: Request; -S3 --> C3: Response; C3 -> +S3: Request; -S3 --> C3: Response; text right of C3: etc.; terminators fade

TLS Session Resumption saves some round-trips and processing time when clients reconnect to download new content. Without Session Resumption, the client would need to perform the full TLS handshake again (2 round-trips before requesting any data), but with Session Resumption, the client can request the new data after just a single round trip (or even 0 round trips if using ssl_early_data, but beware of the potential for “replay attacks” with that).

There are 2 implementations of TLS Session Resumption:

TLS 1.3 only supports the latter approach, under the new name of Pre-Shared Keys rather than Session Tickets. TLS 1.2 supports both.

It is important to note that Session Tickets introduce a potential vulnerability: if an attacker gets access to the root key stored in the server, they can decrypt all Session Tickets, giving them the details they need to decrypt all connections, past and future (until the key is cycled). This breaks Forward Secrecy1. For this reason, the common recommendation is to cycle the root key frequently (at least once per day, and some sites rotate it much more often) to limit the impact of a compromise.

Many sources recommend disabling Session Tickets in nginx because it did not originally support automatic cycling of the root keys, but this behaviour now exists in widely available versions and the advice to disable Session Tickets is increasingly out-of-date.

To enable Session Tickets on a single server (using a randomly generated root key which will automatically be rotated), add the following to the http block configuration (e.g. a file in /etc/nginx/conf.d/):

ssl_session_cache shared:SSL:5M;
ssl_session_timeout 1h;
ssl_session_tickets on;

A breakdown of these options:

The default behaviour is to generate a random root key, and regenerate it at intervals of ssl_session_timeout. NGINX then stores 2 of these keys at a time: one to use for new connections, and the other (the previous key) to validate existing tickets which have not yet expired.

Load balancing

If you are load-balancing multiple nginx servers, they must share the root keys between them so that tickets issued by one will be recognised by the others. If you do not do this, the site will still be served without any apparent issues, but Session Resumption will simply not work whenever clients reach a different server when resuming their connection (falling back to a full handshake).

To share keys, they must be generated externally and passed in to nginx via ssl_session_ticket_key. Then you can use whatever means you prefer for sharing the key files between servers.

The keys can be generated and cycled with:

rm previous.key
mv current.key previous.key
mv upcoming.key current.key
openssl rand 80 > upcoming.key

(this may be scheduled using e.g. crontab).

And loaded with:

ssl_session_ticket_key current.key;
ssl_session_ticket_key previous.key;
ssl_session_ticket_key upcoming.key;

This particular approach keeps 3 keys in use at a time: the current key is used for new connections (because it is listed first in the nginx config). The previous key can still be used to validate existing tokens (issued before this key was cycled, but which have not yet expired), and the upcoming key adds some leeway to allow time for the keys to propagate between all servers (all servers in the cluster will be able to accept connections using the next key, even if they have not yet cycled to it themselves).

Testing Session Resumption

SSLLabs can be used to confirm that Session Resumption is working, but it will not check that the keys are being rotated.

To confirm that key rotation is working with your setup, you can make a series of TLS 1.2 requests to the server and inspect the responses:

openssl s_client -connect example.com:443 -tls1_2 -servername example.com </dev/null 2>/dev/null | grep -A14 'TLS session ticket:'

(change both occurrences of example.com to your domain)

This will print out the opaque session ticket blob. Fortunately, nginx follows a convention of using the first 16 bytes of this opaque blob to store the key ID. If you make multiple requests in succession, you should see that although most of the data changes at random, the first 16 bytes (the first line of output) is constant. When the key rotates, this line will change, so you should see it change at intervals of ssl_session_timeout.

Troubleshooting

The key changes with every request

This probably means you are using a load-balanced cluster of nginx servers and you are not sharing the keys between them correctly.

The key never changes

This probably means the keys are not cycling at all, not being reloaded by nginx, or the ssl_session_timeout is set too high.

Multiple sites

When working with multiple sites served by a single nginx instance, it’s often desirable to block access to the raw IP address (by default, nginx will serve whichever site was defined first).

But when doing this it’s useful to understand how requests get routed to the correct site (confusingly called a server in the nginx config).

Classically (in HTTP), the Host header is used to decide which site should respond to a request. However with HTTPS the Host header (along with all other headers and the requested URL) is not sent until the TLS connection is established. But the server does not know which certificate to send to establish the TLS connection until it knows which site the client wishes to connect to. A “Chicken or egg” problem.

To solve this, clients send the hostname (but not the full URL path) as part of the initial “client hello” packet2. This is read by nginx to decide which site (and therefore which certificate) to use.

When resuming a session, the approach is slightly different: the request does not (necessarily) include a hostname in the first packet. This means that nginx has to fall-back to using whichever site is its default3 for the purpose of resuming the TLS session, then can switch to the correct site once the secure channel is established and the Host header has been sent. This is not a problem, because the encrypted session token is tied to the server‘s private key, not the specific site’s certificate.

All of this means that we can set up a default “catch-all” site to block access, but we must be aware that it will also be responsible for handling TLS Session Resumption requests:

server {
	listen 80 default_server;
	listen [::]:80 default_server;
	listen 443 ssl default_server;
	listen [::]:443 ssl default_server;

	ssl_reject_handshake on;

	access_log off;
	lingering_close off;
	return 444; # nginx special code: close without response
}

Note: for TLS Session Resumption to continue working with this configuration, it is important that both the ssl_protocols [...] configuration and the ssl_session_cache [...] configuration are defined at the root level (inside the http block, not a server block), as described above. If they are defined only at the server level, TLS Session Resumption will fail. The ssl_certificate and ssl_certificate_key config lines can (and should) continue to be defined at the server level, allowing you to specify different certificates for each site (and they do not need to be specified for this new fallback site).

With this configuration, requests for unknown domains and requests which do not specify a domain at all will be rejected immediately, with no SSL handshake and no response (this avoids the need for a “stub” self-signed certificate, used by some approaches). But requests to resume an existing session will succeed, then get re-assigned to the correct site.

Bonus HTTPS configuration

There are some more (simpler) HTTPS features which can also be enabled:

More information / further reading

I found the following sources useful while figuring this stuff out:

Footnotes

  1. Forward Secrecy is the idea that if an attacker passively records encrypted network traffic, they should not be able to decrypt it after stealing secrets in the future. ↩︎

  2. Since this is sent before the encrypted channel is established, it is in plain text and can be observed by all nodes along the route; an obvious privacy concern. It is also sent by clients regardless of the server’s configuration, so even servers which only serve a single site will receive the host in the unencrypted “client hello” packet, with no ability to stop it being sent (but, equally, traffic to servers which serve only a single site is identifiable by the IP address anyway). Recently, “encrypted client hello” has emerged as a way to avoid the plain-text privacy issue, but it requires a very recent version of OpenSSL and is beyond the scope of this article to configure. ↩︎

  3. Decided by the presence of default_server, or first site defined if none have this. ↩︎