No Hair Blog: HTTPS using relayd reverse proxy and SNI with virtual httpd hosts
HTTPS using relayd reverse proxy and SNI with virtual httpd hosts

I decided to add HTTPS to my backwater website using Let's Encrypt as the certificate authority. I had set this up as a test using a self-signed certificate. This works but is flagged by Mozilla and other browsers. So, I set it up with a real CA. Then, I ran into the problem of the relayd reverse proxy in front of httpd.

In order for the relayd reverse proxy to do inspection, rewriting, redirection, or sanitization (blocking of malformed or malicious requests) of HTTPS requests, relayd has to close the tls 'tunnel', decrypt the HTTPS traffic, and be able to read the headers, etc. Until recently, the certificates were handled by httpd; relayd did not support SNI (Server Name Indication) or, if a certificate was employed at the relayd level to decrypt the HTTPS traffic, it had to be keyed to the IP address of the relayd process (see here). To get this to work with multiple hosts on one server, you had to have each virtual host on a different external IP address. One hack included creating links from website certificate files such as 'www.example.cm.crt/pem/key' to '127.0.0.1.crt/pem/key' which could be then used by relayd. This works for a single website but if multiple domains are being served on a single httpd server behind a single relayd proxy, HTTPS can only be enabled on one per localhost address. Another hack is to use the 'subjAltName' to list multiple domain names in one key/certificate pair and refer this back to relayd with a soft link to a key and certificate named for the relayd IP address. This works but key management gets a bit complicated.

On OpenBSD 6.6 and later, relayd now supports SNI and can now use certificates keyed to the website name. Here's one way to do it

A. Setup acme client and get certificates:

First, create the directories to hold the keys and certificates if they do not exist.

doas mkdir -p -w 700 /etc/ssl/private
doas mkdir -p -m 755 /var/www/acme

If they already exist, check the permissions.

Now, edit acme-client.conf for our two domains/virtual hosts on httpd.

doas cp /etc/examples/acme-client.conf /etc/acme-client.conf

Then edit acme-client.conf like this, substituting the domains for which you will request certificates.

#
# $OpenBSD: acme-client.conf
#

authority letsencrypt {
        api url "https://acme-v02.api.letsencrypt.org/directory"
        account key "/etc/acme/letsencrypt-privkey.pem"
}

authority letsencrypt-staging {
        api url "https://acme-staging-v02.api.letsencrypt.org/directory"
        account key "/etc/acme/letsencrypt-staging-privkey.pem"
}

domain www.example.net {
        alternative names { example.net }
        domain key "/etc/ssl/private/www.example.net.key"
        domain certificate "/etc/ssl/www.example.net.crt"
        domain full chain certificate "/etc/ssl/www.example.net.fullchain.pem"
        sign with letsencrypt
}

domain www.example.cc {
        alternative names { example.cc }
        domain key "/etc/ssl/private/www.example.cc.key"
        domain certificate "www.example.cc.crt"
        domain full chain certificate "/etc/ssl/www.example.cc.fullchain.pem"
        sign with letsencrypt
}

Now let's make some configuration changes in httpd.conf to enable Let's Encrypt run the challenge and confirm control/ownership of the webserver. This involves adding a location directive.

# www.example.net virtual server
server "www.example.net" {
        alias "example.net"
        listen on $ext_addr port $ext_port
        log style combined
        root "/htdocs/vhosts/example.net"
        
        location "/.well-known/acme-challenge/*"  {
        	root "/acme"
        	request strip 2
        	}
        }

# www.example.cc virtual server
server "www.example.cc" {
        alias "example.cc"
        listen on $ext_addr port $ext_port
        log style combined
        root "/htdocs/vhosts/example.cc"
        
        location "/.well-known/acme-challenge/*"  {
        	root "/acme"
        	request strip 2
        	}
        }

Now restart httpd and then create a new account and request keys from letsencrypt.org.

acme-client -v www.example.net
acme-client -v www.example.cc

Check the output to be sure there are no errors. Check /etc/ssl and /etc/ssl/private to see if the keys, certificates, and pem files are there.

[As an aside, if you weren't using relayd and you want httpd to handle the HTTPS requests, you would go back at this point and add to httpd.conf in the server{} block:

tls {
  certificate "/etc/ssl/www.example.net.pem"
  key "/etc/ssl/private/www.example.net.key"
  }

And so on for other virtual hosts. However, we are going to let relayd handle the certificates so httpd only need to serves plain old unencrypted HTTP traffic.]

B. Setup httpd:

As an example of the setup, my webserver is set up like this:

            HTTP              HTTPS       HTTP
          port 80            port 80   port 8080      (No change from usual)
HTTP request -> pf redirection -> relayd -> httpd

             HTTPS             HTTPS       HTTP
           port 443          port 443   port 8080
HTTPS request -> pf redirection -> relayd -> httpd

relayd decrypts the HTTPS traffic and httpd only serves HTTP. This is like the setup for a TLS acceleration relay. Unecrypted HTTP requests are still returned; redirection with HTTPS only is not currently enforced.

C. Setup pf:

The pf stanzas are:

# redirect HTTP traffic to relayd listening on 127.0.0.1:80
pass in proto tcp from any to $ext_if port www rdr-to 127.0.0.1 port 80
pass out on $lan_if divert-reply

# redirect HTTPS traffic to relayd listening on 127.0.0.1:443
pass in proto tcp from any to $ext_if port https rdr-to 127.0.0.1 port 443
pass out on $lan_if divert-reply

D: Setup relayd:

Now relayd.conf goes something like this:

##
## $OpenBSD: relayd.conf
##

##
## Macros
##
httpd_ip="127.0.0.1"

##
## Global Options
##
# interval 10
# timeout 1000
prefork 3
log all

##
## Tables
##
table <webserver1> {127.0.0.1}

##
## Redirections
##

##
## Protocols
##

#
# Filtering rules for HTTP reverse proxy
#

http protocol http_reverseproxy {

#       # TCP performance options 
        tcp {nodelay, sack, socket buffer 65536, backlog 100 }

#       # Return HTTP/HTML error pages
        return error

#       # allow logging of remote client ips to internal web server log
        match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
        match request header append "Forwarded" value "$REMOTE_ADDR"
        match request header append "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"

#       # Change timeout
        match header set "Keep-Alive" value "$TIMEOUT"

#       # Anonymize our webserver's name/type
        match response header set  "Server" value "Microsoft IIS 9 beta 1"
        
#       # Block malicious requests:
#	# block requests for 'php', 'wp-', and 'dat'
#	# pass GET and HEAD, drop all other HTTP requests
	block request quick path "/*.php"
	block request quick path "/*.wp-"
        block request quick path "/*.dat"
        pass request quick method "HEAD" forward to <webserver1>
        pass request quick method "GET" forward to <webserver1>
        match request label "HTTP Request Not Allowed"
        block request
		
        }
        
#
# Filtering rules for HTTPS reverse proxy
#
      
http protocol https_reverseproxy {

#       # TCP performance options 
        tcp {nodelay, sack, socket buffer 65536, backlog 100 }

#       # Return HTTP/HTML error pages
        return error
        
        tls keypair "www.example.net"
        tls yeypair "www.example.cc"
        tls { no tlsv1.0, ciphers "HIGH" }

#       # allow logging of remote client ips to internal web server log
        match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
        match request header append "Forwarded" value "$REMOTE_ADDR"
        match request header append "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"

#       # Change timeout
        match header set "Keep-Alive" value "$TIMEOUT"
    
#       # Anonymize our webserver's name/type
        match response header set  "Server" value "Microsoft IIS 9 beta 1"
        
#       # Block malicious requests:
#	# block requests for 'php', 'wp-', and 'dat'
#	# pass GET and HEAD, drop all other HTTP requests
	block request quick path "/*.php"
	block request quick path "/*.wp-"
        block request quick path "/*.dat"
        pass request quick method "HEAD" forward to <webserver1>
        pass request quick method "GET" forward to <webserver1>
        match request label "HTTP Request Not Allowed"
        block request

        }

##
## Relays
##

relay http_80 {

        # listen on pf-rdr redirected port for http traffic
        listen on 127.0.0.1 port 80

        # apply web filters listed above
        protocol http_reverseproxy

        # forward to httpd
        forward to $httpd_addr port 8080

        }

relay https_443 {

        # listen on pf-rdr redirected port for https traffic
        # bound for www.example.net
        listen on 127.0.0.1 port 443 tls

        # apply web filters listed above
        protocol https_reverseproxy

        # forward to httpd
        forward to $httpd_addr port 8080

        }
        

##
## Routers
##

##
## End of /etc/relayd.conf
##

The 'tls keypair' identifies the certificate and key to be used by relayd (in lieu of the default certificate keyed to the ip address). Multiple keypairs can be used, one for each host. The first keypair is considered the default and will be used for requests not matching any key/certificate domain names. httpd.conf is unchanged.

[As an aside, with the 'block request path' and 'block request url' commands you can do a pretty adequate job of filtering spam and malicious requests so they never even reach httpd.]

E. Optional - add the CAA records to your DNS server.

For BIND:

example.net.  CAA 0 issue "letsencrypt.org"
examaple.com.  CAA 0 issue "letsencrypt.org"

For unbound:

local-data: "example.net. CAA 0 issue letsencrypt.org"
local-data" "examaple.com. CAA 0 issue letsencrypt.org"

Then test the website and validate with www.ssllabs.com


Posted by Gordon, No Hair Blog, Nov 19, 2019

© nohair.net and the author

For comments, corrections, and addenda, email: gordon[AT]nohair.net

Blog | Entries | Tags | Home