DEV Community

Cover image for Don't Get Rate-Limited: Use Let's Encrypt Staging
Nick Taylor
Nick Taylor Subscriber

Posted on • Edited on • Originally published at nickyt.co

Don't Get Rate-Limited: Use Let's Encrypt Staging

While getting Pomerium Core set up this week for a demo, I ran into Let's Encrypt rate limits. Here’s how using Let's Encrypt staging can save you time when setting up auto-provisioned certificates.

For context, I work at Pomerium. This was part of a local demo stack I was building with Docker Compose. Pomerium is an identity-aware proxy (think of it as an application gateway) that secures access to your apps with built-in policy enforcement, SSO, microsegmentation etc. All the Zero Trust security goodies.

I had autocert enabled for Pomerium in my config.yaml:

autocert: true
Enter fullscreen mode Exit fullscreen mode

What is autocert?

autocert is a Pomerium feature that automatically provisions and renews TLS certificates for your routes using Let’s Encrypt.
No manual cert management, no copying files around — just tell Pomerium to handle it and it will issue valid certificates on the fly.

This works great… until you accidentally annoy Let’s Encrypt while testing.

What happened

I was adding a few test routes (verify.demo.maisonlab.dev, authenticate.demo.maisonlab.dev, etc.) and restarting Pomerium repeatedly to tweak my config.

Then this showed up in the logs:

HTTP 429 urn:ietf:params:acme:error:rateLimited - too many certificates (5) already issued for this exact set of domains in the last 168h0m0s
Enter fullscreen mode Exit fullscreen mode

Rate limited. No more certs for verify.demo.maisonlab.dev for 7 days.

Why this happened

First, I had the wrong volume path for the autocert cache:

services:
  pomerium:
    image: pomerium:mcp
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./config.yaml:/pomerium/config.yaml:ro
-      - ./pomerium-cache:/pomerium/.cache
+      - ./pomerium-cache:/data/autocert
+      # this is where Pomerium stores autocert data
    networks:
      - pom-network

# ...
Enter fullscreen mode Exit fullscreen mode

Without the correct cache volume, Pomerium wasn’t persisting issued certs between Docker compose restarts. Each time I restarted the Pomerium container, it tried to re-request a new cert and Let’s Encrypt counted every attempt.

Second, even failed attempts count toward Let’s Encrypt’s certificate provisioning quota. 🫠

The fix: use staging while testing

If you’re working on a new Pomerium setup, use Let’s Encrypt’s staging environment while testing certificate provisioning:

autocert: true
# autocert_ca defaults to https://acme-v02.api.letsencrypt.org/directory
autocert_ca: https://acme-staging-v02.api.letsencrypt.org/directory
Enter fullscreen mode Exit fullscreen mode

See the autocert_ca docs. When you point Pomerium at the staging certificate authority (CA):

  • You can test your flow without worrying about production rate limits.
  • The only catch: staging certs aren’t trusted by browsers (you’ll get a warning), but that’s fine for local testing.

Once your config is good and your cert cache is persisting properly, flip back to production by removing the autocert_ca line.

TL;DR

If you see:

HTTP 429 urn:ietf:params:acme:error:rateLimited
Enter fullscreen mode Exit fullscreen mode

You’re in rate-limit jail for 7 days.

Fred Armissen as a dictator saying Straight to jail!

  • Make sure your autocert volume is correct for the Pomerium container, i.e. (/data/autocert).
  • Use Let’s Encrypt staging while testing.
  • Flip back to production once you’re happy.

One last note: This isn’t a Pomerium-specific quirk. It’s how Let’s Encrypt’s rate limits work in general. If you’re automating certificate issuance with any tool, the same caution applies when testing.

Hope this saves someone else a bit of time (and avoids a few unnecessary retries).
If you’re building out your own Pomerium setup and hit this, now you know what’s going on. Also, if you're using Pomerium, hit me up! I'd love to know how you're using it or what issues you run into, if any.

If you want to stay in touch, all my socials are on nickyt.online.

Photo by Andrew Wulf on Unsplash

Top comments (8)

Collapse
 
nevodavid profile image
Nevo David

Been there with getting stuck in rate-limit jail - feels brutal, but this makes it way easier!

Collapse
 
nickytonline profile image
Nick Taylor

Awesome!

Hackerman from Kung Fury putting on a Nintendo Power glove

Collapse
 
nadeem_zia_257af7e986ffc6 profile image
nadeem zia

Nice work

Collapse
 
nickytonline profile image
Nick Taylor

Thanks!

Multiple TV and movie characters giving a thumbs up

Collapse
 
dotallio profile image
Dotallio

Great tip on the staging CA - I’ve lost time hitting the same kind of rate limit in dev too. Has anyone run into any weird issues with the staging certs other than the browser warning?

Collapse
 
nickytonline profile image
Nick Taylor

Only one I've experience so far is the browser warning.

Collapse
 
alexcarter profile image
Alex Carter

Hello

Collapse
 
nickytonline profile image
Nick Taylor

Obiwan saying hello there!

Some comments may only be visible to logged-in visitors. Sign in to view all comments.