The Rate Limit I Forgot to Ask For

I asked an AI assistant to build a public endpoint for LoteríaYa: return the latest lottery results as JSON, no login required. It worked. I pasted the URL into the browser, saw the JSON come back clean, closed the tab, and moved on to the next thing.

That sentence, "it worked," is the one that should have slowed me down. It worked for exactly one request, from one browser, from me. Nobody had asked what happens when that same URL gets hit a thousand times in a second, from a script instead of a browser tab. I hadn't asked either, which is the actual point of this post.

The endpoint that already worked

LoteríaYa shows results for lotteries and chances across Colombia, published within roughly 15 minutes of each draw. Some of that lives on public pages, some of it lives on endpoints that pages and other clients hit directly. None of it sits behind a login. That's by design: results are public information, and putting a signup wall in front of a lottery result would be a strange kind of gatekeeping nobody asked for.

Public and unauthenticated is a fine choice. Public and unauthenticated and unlimited is a different thing entirely, and that's the part I hadn't built yet. The endpoint answered every request the same way, as fast as the database could return an answer, with no concept of "too many, too fast." If a script decided to call it 5,000 times a minute, it wasn't going to say no. It was going to try to answer 5,000 times, every time.

What a rate limit actually decides

A rate limit is a cap on how many times something can be called in a given stretch of time. Past that number, the response stops being data and becomes a controlled error instead: a 429, "too many requests," maybe with a hint about when to try again.

Simple enough as a sentence. What took me longer to get right is that a rate limit is really three separate decisions bundled into one feature, and skipping any of them quietly breaks the whole thing.

The first decision is the key: what are you counting requests against? For an endpoint with no login, the obvious answer is IP address. It's not a perfect answer. A whole office or a mobile carrier's NAT can put hundreds of real users behind one IP, so an aggressive per-IP cap can punish innocent people sharing an address. For endpoints that do require some form of identity, an API key or a token is a better key: it tracks the actual caller instead of a network address that might represent one person or a thousand. For LoteríaYa's public results, IP is the one that made sense, since there's no token to key against in the first place.

The second decision is the window: how do you define "per minute"? This one sounds trivial until you try to implement it, and it's where I actually got it wrong the first time.

The third is the response: what do you send back once the limit is hit, and does the caller get any information about when it's safe to try again? Skip this and your rate limit still works, technically, but it turns a legitimate client that briefly hit a burst into someone guessing blindly when to retry.

The window problem, and what I got wrong first

My first pass at this used what's called a fixed window: pick a window, say 60 seconds, allow N requests inside it, reset the counter when the window ends. It's the simplest thing to build and the first thing most people, including the version of me writing this a few weeks ago, reach for.

It has a real gap at the edges. If your limit is 100 requests per 60-second window and someone sends 100 requests at second 59 of one window, then another 100 at second 1 of the next window, they've just sent 200 requests in about two seconds and never technically broken the rule. The window reset gave them a second bucket to refill before the first one had even finished draining.

A sliding window fixes that by not resetting to zero on a clock tick. Instead of a hard reset, it tracks requests against a rolling timeframe, so the count at any given moment reflects the last 60 seconds, not "the last 60 seconds since the top of this particular minute." It costs a bit more to compute and to store, since you need something more than a single integer that resets, but it closes the boundary gap that a fixed window leaves wide open.

There's a second failure that had nothing to do with which window algorithm I picked, and everything to do with where I put the counter. My very first version kept the request count in memory, inside the function handling the request. That's fine on a single long-running server. It falls apart on serverless, where each invocation can land on a different instance with its own memory, and where an idle function can get spun down and lose its state entirely. I'd built a rate limiter that, in practice, mostly limited nothing, because half the time the counter it was checking wasn't the same counter the last five requests had touched. The count needs to live somewhere shared and persistent across every instance answering that route, which for me meant moving it into Redis instead of a variable in the function.

Infrastructure layer or application layer: you still have to pick

Once the counter lived somewhere real, the next question was where to put the check itself, and this is where the AI-assistant framing from earlier stops being a throwaway line and becomes an actual design decision.

You can enforce a rate limit at the infrastructure layer, before a request ever reaches your application code. Cloudflare's rate limiting rules, or edge-level protections on platforms like Vercel, sit in front of your server and can reject a request based on IP and path without spinning up your function at all. This is the cheapest place to block abuse, because you're not paying compute time to answer a request you were always going to refuse. It's also coarser: it typically knows about IPs and paths, not about which specific token made the request or what that token is allowed to do.

You can also enforce it at the application layer, inside your own code, usually in middleware that runs before your route handler. This is where you can be precise: different limits for different routes, limits keyed by API key instead of IP, custom logic for what counts as "one request" if some endpoints are heavier than others. The cost is that the request still reaches your server and your database connection before you decide to reject it, which is exactly the cost you're trying to avoid in the first place.

For LoteríaYa I ended up doing both, at different layers for different reasons. The results endpoints, which are the ones a script would actually want to hammer, get a rate limit at the application layer, keyed by IP, using a sliding window backed by Redis (Upstash's rate limiting library, specifically, since it was already the closest fit for a Next.js app on serverless functions and I didn't want to run my own Redis instance). Anything closer to raw abuse, like an obvious flood from a single IP across many different paths at once, is the kind of thing better handled by the edge before it costs me a database round trip at all.

What actually shipped

The version running now checks, per IP, on a sliding window, and returns a 429 with a Retry-After header when a caller goes past the limit. The specific numbers I use for the window and the request count aren't the interesting part of this post, and honestly they're still numbers I expect to tune as real traffic teaches me more than my guesses did. What mattered was closing the gap between "an endpoint that answers" and "an endpoint that knows when to stop answering the same caller."

None of that came from the AI assistant that wrote the endpoint in the first place, and I don't think it should have. I asked for a route that returns results. It built a route that returns results: the query, the response shape, the error handling for a bad request. A rate limit wasn't in the request, so it wasn't in the output. That's not a flaw in the tool. It's the same reason it wouldn't add a lock to a door I only asked it to build, and expecting it to guess at security work I never specified is asking an assistant to read a mind I hadn't made up yet.

So here's the concrete version of the lesson, the one I'd actually put in a prompt next time instead of learning after the fact: when you ask an AI to expose a public endpoint, add a second instruction in the same request. Tell it who the limit applies to (IP, user, or token), what window to use (and whether a fixed window's edge case matters for your case), and what to return once that limit is hit. Three details, one extra line in the prompt. Skip it, and the endpoint you shipped isn't fast. It's just open, and it's a matter of time before something other than your own browser finds that out first.