HN Distilled

Essential insights from Hacker News discussions

Nginx introduces native support for ACME protocol

Original Article

Hacker News Discussion

This discussion revolves around the integration of native ACME (Automated Certificate Management Environment) support, specifically for Let's Encrypt certificates, into Nginx. The advent of this feature is being compared and contrasted with existing solutions, primarily Caddy and Apache's mod_md, and the broader ecosystem of certificate management tools like Certbot, ACME-sh, and Traefik.

Native ACME Support in Nginx and its Comparison to Caddy

A significant portion of the discussion centers on Nginx's new ACME integration and how it stacks up against Caddy, which has had this functionality built-in for a long time. Many users express a cautious optimism, acknowledging Nginx's move but also pointing out Caddy's established strengths.

  • Caddy's Established Advantage: Caddy is frequently cited as the benchmark for ease of use and polished ACME integration. Users praise its "It Just Works" philosophy and simple configuration.

    • "When I tried using Caddy with something serious for the first time, I thought I was missing something. I thought, these docs must be incomplete, there has to be more to it, how does it know to do X based on Y, this is never going to work... But it DID work. There IS almost nothing to it." - bityard
    • "Caddy has been great for me. I don't think you should switch if your current setup works but give it a try in a new project." - orphea
    • "Caddy has been great for me. I don't think you should switch if your current setup works but give it a try in a new project." - orphea
    • "What I really like about Caddy is their better syntax. I actually use nginx (via nginx proxy manager) and Traefik but recently I did one project with Caddy and found it very nice." - elashri
    • "I use Caddy as my main reverse proxy into containers with CloudFlare based DNS let’s encrypt. The syntax is intuitive and just works. I’ve used Traefik in the past with Kubernetes and while powerful the setup and grok ability has quite a bit steeper learning curve." - nodesocket
    • "Nginx is so 2007." - reactordev
    • "After discovering Caddy, I don't use Nginx any longer. Just a much better development experience." - RagnarD
    • "Caddy is just for developers that want to publish/test the thing they write. For power users or infra admins, nginx is still much more valuable. And yes, I use Caddy in my home lab and it's nice and all but it's not really flexible as nginx is." - darkwater
    • "We use Caddy across hundreds of apps with 10s of millions of requests per day in production." - j-krieger

  • Nginx's Catch-up Effort: The integration is seen as a positive step for Nginx users who may not want to switch to Caddy but desire native ACME support.

    • "It's interesting that mod_md is so unknown: https://httpd.apache.org/docs/2.4/mod/mod_md.html But also hey, now we have built-in ACME support in all the mainstream web servers: Nginx, Caddy and Apache2!" - KronisLV
    • "This is pretty big. Caddy had this forever but not everybody wants to use caddy. It'll probably eat into the user share of software like Traefik." - dizhn
    • "I figured either somehow Let's Encrypt doesn't work out, or, everybody bakes in ACME within 2-3 years. The idea that you can buy software in 2025 which has TLS encryption but expects you to go sort out the certificate. It's like if cars had to be refuelled periodically by taking them to a weird dedicated building which is not useful to anything else rather than just charging while you're asleep like a phone and... yeah you know what I get it now." - tialaramex

  • Configuration Simplicity: Caddy's configuration is frequently highlighted as a major advantage, even for complex setups.

    • "You set literally the bare minimum of configuration you could possibly need, and Caddy figures out the rest and uses sane defaults. The docs are VERY good, there is a nice community around it." - bityard
    • "I agree. That, and the sane defaults are almost always nearly perfect for me. Here is the entire configuration for a TLS-enabled HTTP/{1.1,2,3} static server: ... That's the whole thing." - kstrauser

  • Plugin System and Customization: A point of concern with Caddy is its plugin system and the need for custom builds for certain features like DNS challenges.

    • "If I had any complaint at all, it would be that the plugin system is slightly goofy." - bityard
    • "The other thing I really don't like is if you install via a package manager to get automated updates, you don't get any of the plugins. If you want plugins you have to build it yourself or use their build service, and you don't get automatic updates." - Saris

Limitations and Future Considerations for Nginx's ACME Support

The discussion also delves into the limitations of Nginx's current implementation and what users hope to see in the future. The primary missing piece for many is support for the DNS-01 challenge, which is crucial for wildcard certificates and internal services not exposed to the internet.

  • HTTP-01 vs. DNS-01 Challenges: The current Nginx support only handles HTTP-01 challenges, leading to requests for DNS-01 support.

    • "This integration doesn’t support the dns-01 challenge. So wildcard certs are out of the question at this point." - cpach
    • "The current preview implementation supports HTTP-01 challenges to verify the client’s domain ownership. ... Seriously. PKI in IT is a PITA and I want someone to SOLVE IT without requiring AD CAs or Yet Another Hyperspecific Appliance (YAHA). If your load balancer, proxy server, web server, or router appliance can’t mint me a basic Acme certificate via DNS-01 challenges, then you officially suck and I will throw your product out for something like Caddy the first chance I get." - stego-tech
    • "The only temptation [for HTTP-01] is Wildcards." - Dylan16807
    • "Wildcard certificates are probably the most important answer: they’re not available via HTTP challenge." - chrismorgan

  • Wildcard Certificates and Internal Services: DNS-01 is identified as essential for obtaining wildcard certificates and for managing certificates for internal services.

    • "For wildcards you need a Caddy build that includes the dns plugin for your specific provider." - nadanke
    • "DNS-01 is also good for services on a private network." - cpach
    • "DNS-01 is a practical requirement here because only one of the terminating servers would be able to respond during an HTTP or ALPN challenge." - jcgl

  • DNS Provider API Complexity and Delegation: The need for Nginx to interact with various DNS provider APIs for DNS-01 challenges complicates the implementation. Solutions involving DNS delegation or standardized protocols like TSIG are discussed.

    • "But you have to have your dns api key loaded and many dns providers don’t allow api keys per zone." - clvx
    • "No, you have to build Caddy with plugins. We provide xcaddy to make it easy.... It's still kinda annoying because now you need to manage the binary for yourself but when I tried it with Hetzner it worked fine." - francislavoie
    • "The problem I continue to encounter is that delegating this to colleagues or other teams is that - inevitably - someone thinks they're clever bypassing part or all of the procedure to, say, generate a wildcard cert and share its private key component with whoever asks for a cert, instead of going through approved processes." - stego-tech
    • "You can sidestep that by delegating the ACME keys to your own name server." - account42
    • "DNS challenge is complicated by the fact that every registrar has their own API. HTTP is easier for nginx because it’s a single flow and it already does HTTP." - pointlessone

  • Comparison with Traefik and other Tools: Traefik is mentioned as another popular reverse proxy with ACME support, though some find its learning curve steeper than Caddy's. The discussion briefly touches on other tools like Certbot, ACME-sh, and LEGO.

    • "I’ve used Traefik in the past with Kubernetes and while powerful the setup and grok ability has quite a bit steeper learning curve." - nodesocket
    • "Certbot is a giant swiss army chainsaw that can do everything middlingly well, if you don't mind vibecoding your encryption intrastructure. But a clean solution it usually isn't." - creshal
    • "Certbot also fights automation and provisioning with e.g. Andible by modifying config files to remember command line options if you ever need to do anything manually in an emergency. It is a terrible piece of software. I use dehydrated which I'd much friendlier to automation." - jeltz

Critiques of Certbot and the Broader ACME Ecosystem

A notable undercurrent in the discussion is a critique of Certbot and the overall experience of managing certificates, particularly for automation and deployment within diverse environments.

  • Certbot's Usability and Integration Issues: Certbot receives significant criticism for its perceived complexity, its dependency on Snap packages (controversial for some), and its aggressive modification of configuration files.

    • "Certbot is a utility that can only be installed via snap. That crap won’t make it to our servers, and many other people view it the same way I do." - 9dev
    • "Maybe it's better these days, but even as an experienced systems administrator, I found certbot incredibly annoying to use in practice. They tried to make it easy and general-purpose for beginners to web hosting, but they did it with a lot of magic that does Weird Stuff to your host and server configuration." - bityard
    • "Certbot also fights automation and provisioning with e.g. Andible by modifying config files to remember command line options if you ever need to do anything manually in an emergency. It is a terrible piece of software." - jeltz
    • "Whoever first recommended using that mode [standalone] in anything other than some sort of emergency situation needs to be given a firm kick in the butt. Certbot also has a mode that mangles your apache or nginx config files in an attempt to wire up certificates to your virtual hosts. Whoever wrote the nginx integration also needs a butt kick, it's terrible." - kijin

  • Desire for Streamlined PKI: There's a general desire for simpler, more automated Public Key Infrastructure (PKI) management, with native integrations in web servers seen as a major step in that direction.

    • "The thinking goes is that this would allow more certificates to be issued internally and easily, but without the extra layer of management involved with a fully bespoke internal CA." - stego-tech
    • "If I can get PKI to be as streamlined as the rest of my tech stack internally, and without forking over large sums for Microsoft Server licenses and CALs, I’d be a very happy dinosaur that’s a lot less worried about tracking the myriad of custom cert renewals and deployments." - stego-tech

  • Versioning and Release Practices: Some users expressed dissatisfaction with projects remaining in perpetual "major version 0," suggesting it indicates immaturity or laziness and a potential for breaking changes.

    • "This is really cool, but I find projects that have thousands of people depending on it not cutting a stable release really distasteful. ... Don't consume major version 0 software, it'll bite you one day. Convince your maintainers to release stable cuts if they've been sitting on major version 0 for years. It's just lazy and immature practice abusing semantic versioning." - andrewmcwatters

Security Considerations and Best Practices

The conversation touches on security best practices, particularly concerning API keys for DNS providers and the implications of wildcard certificates.

  • API Key Security and Granularity: The importance of fine-grained API permissions for DNS providers is highlighted, with users expressing frustration when providers don't offer per-zone or per-subdomain API keys.

    • "The thread requesting it is a masterclass in passive aggressiveness: https://community.cloudflare.com/t/restrict-scope-api-tokens-to-a-subdomain/156702/28" - teruakohatu
    • "One of Traefik's shortcomings with ACME is that you can only use one api key per DNS provider." - chaz6
    • "This is one of the main reasons Caddy stopped using lego for ACME and I wrote our own ACME stack." - mholt

  • Wildcard Certificate Security: Concerns are raised about the security implications of wildcard certificates in scenarios like mutual TLS authentication, where a single wildcard certificate could be misused to impersonate multiple services.

    • "One problem with wildcards is that any service with *.foo.com can pretend to be any other service. This is an issue if you're using mutual TLS authentication and want to trust the server's certificate." - cyberax

  • Delegation and Decentralization of DNS: Users discuss strategies for managing DNS challenges without relying on broad API access, such as delegating specific subdomains or using separate DNS providers. This reflects a desire for more decentralized control over DNS management for certificate issuance.

    • "No, you have to build Caddy with plugins. We provide xcaddy to make it easy. Sign up for notifications on github for releases, and just write yourself a tiny bash script to build the binary with xcaddy, and restart the service." - francislavoie
    • "You can cname _acme-challenge.foo.com to foo.bar.com. Now, if when you do the DNS challenge, you make a TXT at foo.bar.com with the challenge response, through CNAME redirection, the TXT record is picked up as if it were directly at _acme-challenge.foo.com." - dwood_dev
    • "It's a bit like the 'Mozilla' in browser user-agents; I hope we can stop at one digit sacrificed, rather than ending up like user-agents did, though. In other words, 0ver, unironically." - juped