I Deployed a Backend API on Vercel and Here Is What Broke

The API was returning different data on consecutive requests. Same endpoint, same query parameters, same database. But the response was inconsistent.

I spent three days assuming it was a bug in my code before I realized the problem had nothing to do with my code at all.

The project

A few months earlier, I'd started building a straightforward backend, a JSON API with no rendering involved, no pages, just endpoints returning data to a frontend elsewhere. I was already using Next.js for other work; the deploy was one click, there was a generous free tier, and Vercel felt like the obvious choice. It wasn't a hard decision at the time. It barely felt like a decision at all.

The API needed to respond quickly and consistently, and to keep some lightweight state in memory between requests: a cache of recent lookups, a simple counter or two. None of that seemed unusual for a backend. I built it the way I'd build any Express app, just inside a Next.js API route instead.

Then the inconsistent responses started, and I went looking for the bug in the wrong place for three days.

How serverless functions actually work

Here is the mental model I did not have at the time, and wish I did.

When a request hits a Vercel serverless function, one of two things happens. If a warm instance is available from a recent request, that instance handles the new one. If no warm instance is available, Vercel spins up a fresh one from scratch: it loads the Node.js runtime, imports every module your function depends on, runs any module-level initialization code, and only then handles the request.

On a long-running server, all of that initialization happens exactly once, when the server first starts. On serverless, depending on traffic, it can happen on almost every request.

A warm instance is one that handled a recent request and has not yet been shut down. Vercel keeps instances warm for a period after their last request specifically to avoid cold starts on rapid, successive traffic. But if your API goes idle for even a few minutes, that warmth is gone, and the next request pays the full cold start cost from scratch.

Two consequences of this matter a lot more than they sound like they should. For an internal tool used sporadically through the day, the first person to use it after lunch always waits noticeably longer than everyone who uses it right after them. For a customer-facing API, it's worse than that: your slowest responses land on the users who have been waiting the longest between visits, which is often precisely the group you'd least want to leave with a bad impression.

This is also the answer to the mystery at the start of this post.

Problem 1: my in-memory state did not persist

I had a cache sitting at the module level, a Map storing recent database lookups so repeat requests for the same resource would not need to hit the database again.

// This does not work reliably on Vercel serverless functions.
// The cache resets on every cold start.

const cache = new Map();

export default async function handler(req, res) {
  if (cache.has(req.query.id)) {
    return res.json(cache.get(req.query.id));
  }
  const data = await fetchFromDatabase(req.query.id);
  cache.set(req.query.id, data);
  res.json(data);
}

It worked perfectly in local development, where the server stays warm, and the process never restarts. In production, module-level variables like this cache do not reliably persist between invocations, exactly as the lifecycle above explains. The cache would appear to work for a burst of requests hitting the same warm instance, then quietly reset the moment a fresh instance spun up, then work again. That is what produced the inconsistent responses. Not a bug in my query logic, but a cache that existed in one instance and not another, both serving the same endpoint.

The same logic on a long-running Express server behaves the way I originally expected:

// This works reliably on a long-running Express server,
// because the process stays alive between requests.

const cache = new Map();

app.get('/data/:id', async (req, res) => {
  if (cache.has(req.params.id)) {
    return res.json(cache.get(req.params.id));
  }
  const data = await fetchFromDatabase(req.params.id);
  cache.set(req.params.id, data);
  res.json(data);
});

Same Map, same logic. The only difference is that the process hosting it never disappears between requests.

Caching was not the only thing this broke. Rate limiting failed the same way, for the same underlying reason. A common pattern is tracking how many requests an IP address has made in a rolling window, using an in-memory counter:

// This rate limit is only correct on a long-running server.

const requestCounts = new Map();

function isRateLimited(ip) {
  const count = requestCounts.get(ip) || 0;
  if (count >= 10) return true;
  requestCounts.set(ip, count + 1);
  return false;
}

On a long-running server, this works exactly as written. On serverless, each function instance keeps its own counter. Ten concurrent requests might land on ten different warm instances, each with its own counter starting at zero, each perfectly willing to accept one more request. A limit meant to be ten requests per minute becomes, in effect, ten requests per minute per instance, which is not a limit at all once traffic is high enough to spin up several instances at once.

Persistent connections break for the same root reason. Real-time features like live notifications, chat, or collaborative editing depend on holding a connection open and remembering who is on the other end of it between messages. A serverless function shuts down between requests, so there is no process left alive to hold that connection or remember that state. It is not that this is difficult on serverless; it is that the model itself has nowhere to keep it.

Problem 2: cold start latency

Beyond the mechanics above, the practical effect on latency is what actually shows up in monitoring. When a function has been idle, spinning up the runtime, loading modules, and getting ready to respond takes real time. For a simple always-on API expected to answer in milliseconds, that adds latency that is not just slower; it is unpredictable, swinging based on traffic shape rather than staying consistent. That unpredictability, more than the raw latency number, is what makes a serverless API genuinely hard to reason about or monitor sensibly.

Problem 3: database connection overhead

A long-running server opens a connection pool to the database once, when it starts, and every request reuses a connection from that pool.

// Connection pool initialized once, when the server starts.
// Reused across every request that follows.

const pool = new Pool({
  connectionString: process.env.DATABASE_URL,
  max: 10,
});

app.get('/users', async (req, res) => {
  const result = await pool.query('SELECT * FROM users');
  res.json(result.rows);
});

A serverless function does not get that luxury. Each cold start either opens a fresh connection or competes with other invocations over a shared pool. For a backend hitting PostgreSQL on nearly every request, that adds real overhead per request and, under real load, can exhaust the database's connection limit outright.

Problem 4: Next.js overhead for a backend that renders nothing

The React runtime, the routing layer, the build tooling, none of it does useful work for an API that only returns JSON. It adds weight to every cold start and complexity to a codebase that a plain Express server simply does not carry.

What I did instead

I moved the API to a long-running Express server on Render. The server stays warm, module-level variables persist the way I originally expected, the connection pool stays open, and response times are consistent instead of dependent on traffic shape.

Formgrid runs exactly this way today: a long-running Express server on Render, serving over 400 signups and 12 paying customers, handling around 1,000 daily visitors, for under $20 a month, with consistent sub-100ms API responses, because the server is always warm and the connection pool is always ready.

When Vercel and serverless are genuinely the right call

This is not an argument that serverless is bad. It is an argument that it is a specific tool for specific workloads, and mine was not one of them. Serverless is the right choice for:

• Static sites and marketing pages
• Server-side rendered pages built with Next.js
• Edge functions serving geographically distributed, low-latency responses
• Bursty, unpredictable traffic where you want to pay per request rather than for idle time
• Webhooks and event-driven functions that run occasionally, not continuously
• Rapid prototyping, where deployment simplicity matters more than performance tuning

If your workload looks like any of those, Vercel is a genuinely good choice, and I would reach for it again without hesitation.

Where to actually host an always-on Node.js API

This is the question I actually wanted answered before I made my original decision, so here is the honest comparison I wish I had read first.

Render: free tier available, but it spins down after inactivity, which reintroduces the same cold start problem you're trying to avoid. The paid plan, around $7 a month, keeps the server always on. Deployment from GitHub is simple. This is what Formgrid runs on.

Railway: a free tier with a small monthly credit. Simple deployment, a good developer experience, and slightly more flexibility than Render for more complex setups.

Fly.io: a free tier is available, runs containers globally, a solid option if you want geographic distribution without taking on the full complexity of serverless.

Hetzner: not free, but extremely cheap, starting around 4 euros a month for a VPS. Full control, no abstraction layer, and it demands more DevOps knowledge, but it gives you the most flexibility and the best price-to-performance ratio for a simple always-on API.

DigitalOcean App Platform: a free tier available, managed containers, an experience fairly similar to Render.

I left Vercel off this list on purpose. The whole point of this post is that it is the wrong tool for this particular job.

The actual lesson

The mistake was never choosing Vercel specifically. The mistake was choosing a tool without first checking whether its execution model matched my problem.

Before picking infrastructure for an API now, I ask myself a short set of questions:

1. Does this API need to maintain state between requests?
2. Does it need consistent, sub-100ms response times regardless of traffic patterns?
3. Does it hold long-lived connections to a database or an external service?
4. Is the traffic always on, or genuinely bursty?

If the answers are yes, yes, yes, and always on, a long-running server is almost certainly the right call, no matter how convenient serverless feels at deploy time.

If you have hit this same wall, or made the same call and lived to tell about it, I would genuinely like to hear what you learned. Reach me at allen@formgrid.dev.