Caddy and TLS on OpenBSD

Since I want https to work properly on my local network I need to allow caddy to manage TLS certificates for me. There are two options, either I configure caddy to use lets-encrypt to get real trusted certificates so I don’t have to deal with browsers telling me that “this certificate isn’t trusted” or I just let caddy create it’s own certificates and I have to add exceptions (or import the root certificate) on all my client browsers.

At first I just went with the second option as it was quicker to get up and running, but that was always meant to be temporary as the user experience of needing to install certificates everywhere is annoying.

In order to get caddy to provide actual certificates, I need to configure it to respond to Acme challenges. Per the caddy docs there are two ways to do this:

1. HTTP/TLS-ALPN challenge - this requires that my caddy instance be reachable over the public internet, which is not something I’d like to do since it’s an internal service for internal clients.

2. DNS challenge - this requires that I install a custom build of caddy which may be a pain in the butt since I’m running on OpenBSD. On the other hand, it might be super easy. Only one way to find out.

Caddy’s documentation says that I can download a pre-built binary from the caddy download page. Surprisingly, they seem to have native support for OpenBSD! I searched for my dns provider (he.net) selected it and waited for my custom build to download.

I copied the binary onto my OpenBSD home server, replaced the existing caddy binary, fixed the owner and group, ran chmod on the file so it was executable, crossed my fingers and restarted the daemon:

scp caddy_openbsd_amd64_custom gabe@nuc:~/
ssh gabe@nuc
doas mv /usr/local/bin/caddy /usr/local/bin/caddy-old
doas mv ./caddy_openbsd_amd64_custom /usr/local/bin/caddy
doas chown root:bin /usr/local/bin/caddy
doas chmod 755 /usr/local/bin/caddy
doas rcctl restart caddy

Ok, that was really easy. The only downside to this approach is now anytime I upgrade my system this special binary is going to be replaced by the updated caddy from packages and I’ll have to do this again. Not the end of the world, but adds extra compelxity and is yet another place where something could go wrong down the line. I wonder if OpenBSD provides a flavour of the caddy binary with he support already baked in? My guess is no, since there are too many dns providers to do this, but it’s worth a check.

After I got the caddy daemon up and running, I needed to create a new TXT record in my lifewaza.com DNS configuration and enable it for DYNDNS, which lets me create an API key for it. Then I pasted that API key into my Caddyfile, and restarted caddy.

        acme_dns he <HE.NET DYNDNS KEY HERE>

So far, nothing seems to have changed, and I don’t see any errors in my logs or when running the caddy deamon in debug mode. I’m still seeing the old certificates when I connect to my internal services. I’ll have to poke around a bit to figure out why.

After digging a bit, it turns out that the tls internal directive I had specified before means “use the internal tls system” which is obvious now, but when I was looking over my config I assumed it meant “use caddy for handling tls”. So, the fix was just removing tls internal from each hostname entry. I generally do things iteratively (in case I break something along the way, it’s usually easy to undo) and I’m glad I did today because once I got this working for one hostname I realized that doing this host by host would require a custom DNS TXT record for each host, which sounds annoying to setup (I’m lazy). I asked Claude if there was a way to avoid this, and it told me I could configure caddy to use a wildcard entry like so:

# entry for wildcard dns
*.lifewaza.com {
        abort
}

Well, that’s not really what Claude said, Clause said a bunch of stuff that was true but also wasn’t the simplest/cleanest way to do things so eventually I asked Claude to link me to the caddy docs where I found this gem: “Since Caddy 2.10, Caddy will prefer to use an applicable wildcard certificate over requesting a separate certificate for a subdomain. This means that you can list your subdomains as usual, and they will automatically begin using the defined wildcard certificate:”

*.example.com {
	tls {
		dns <provider_name> [<params...>]
	}
	abort
}

# This will use the above certificate
foo.example.com {
	respond "Foo!"
}

Which is exactly what I was trying to do!

Once, I set that up, I needed to change the DNS TXT record I had made to allow it to be for a wildcard, so I updated it to this:

_acme-challenge.lifewaza.com.	300	IN	TXT	"Placeholder"

Then I restarted caddy, and started getting Let’s Encrypt signed certificates for my internal services!

Nice.

Here’s my updated Caddyfile:

# see https://caddyserver.com/docs/
{
        # bind locally-only by default
        default_bind [::1] 127.0.0.1

        # pf handles redirection from port 80 and 443
        http_port 10080
        https_port 10443

        # admin API endpoint on unix socket
        admin unix//var/caddy/admin.sock|0220

        # don't try to install internal CA to system
        skip_install_trust

        acme_dns he <HE.NET DYNDNS KEY HERE>
}

# entry for wildcard dns
*.lifewaza.com {
        abort
}

# services hosted on this box (nuc)
grafana.lifewaza.com {
        reverse_proxy 127.0.0.1:3000
}

prometheus.lifewaza.com
        reverse_proxy 127.0.0.1:9090
}

freshrss.lifewaza.com {
        reverse_proxy 127.0.0.1:8088
}

llama.lifewaza.com {
        reverse_proxy 127.0.0.1:8090
}

unifi.lifewaza.com {
        # Main UniFi Controler access
        reverse_proxy 127.0.0.1:8443 {
                header_up Host {hostport}
                transport http {
                        tls_insecure_skip_verify
                }
        }
}

# services hosted on truenas
actual.lifewaza.com {
        reverse_proxy 192.168.1.7:31012
}

karakeep.lifewaza.com {
        reverse_proxy 192.168.1.7:30147
}

dawarich.lifewaza.com {
        reverse_proxy 192.168.1.7:30161
}

immich.lifewaza.com {
        reverse_proxy 192.168.1.7:30041
}

dozzle.lifewaza.com {
        reverse_proxy 192.168.1.7:30064
}

jellyfin.lifewaza.com {
        reverse_proxy 192.168.1.7:8096
}

mealie.lifewaza.com {
        reverse_proxy 192.168.1.7:30111
}

# truenas box
truenas.lifewaza.com {
        reverse_proxy 192.168.1.7:443 {
                transport http {
                        tls_insecure_skip_verify
                }
        }
}