How to Deploy an AI Agent to Production (Jobs, Not Requests)

To deploy an AI agent to production, stop shipping it as an HTTP request handler. Accept the job, return a run id, and execute the loop in a background worker that saves its state after every step. That one change fixes the timeouts, the duplicate side effects, and the runs that die at step 7 and start over at step 1. Everything else in this post is detail on top of that move.

TL;DR

• An agent run is a job, not a request. The endpoint writes a run row and returns a run id. A worker owns the loop.
• The duration math decides it for you. Vercel Functions cap at 300 seconds on Hobby, 800 seconds on Pro and Enterprise (1800 seconds in beta), and AWS Lambda at 900 seconds per invocation.
• Anthropic's docs tell you to use streaming or the Batch API for long-running requests, especially over 10 minutes, and the SDKs validate that non-streaming requests are not expected to exceed a 10-minute timeout.
• Checkpoint after every step. A crash should cost you one step, not the whole run.
• Ship the boring layer before the clever one: retries with backoff, a step ceiling, a cost ceiling, an idempotency key on every write, and a trace per run.

The duration math nobody runs first

Most agents get built inside a request handler because that is where the code already lives. It works on your laptop against a three-step task, then dies the first time a real user asks something that needs 25 tool calls.

Look at the ceilings. Vercel Functions with fluid compute default to 300 seconds on every plan, with Hobby capped there, Pro and Enterprise able to reach 800 seconds, and an 1800 second extended maximum in beta that requires function-level configuration. AWS Lambda allows 1 to 900 seconds per invocation. Vercel's Edge runtime is tighter still: it must begin sending a response within 25 seconds to keep streaming, and can stream for up to 300 seconds.

Now price a real run. Twenty tool calls, three of them a slow third-party API at 4 seconds each, two model calls that retry after a 429, and a final synthesis pass with a large max_tokens. Five to twelve minutes is normal. That fits in an 800 second window right up until the day it does not, and the failure mode is a 504 with no partial work saved.

There is a second ceiling that catches people even inside a generous function limit. Anthropic's error documentation warns against a large max_tokens without streaming, because some networks drop idle connections after a variable period, which can cause the request to fail or time out with no response at all. The SDKs validate that non-streaming Messages API requests are not expected to exceed a 10-minute timeout, and they set a TCP keep-alive. So even the single longest model call in your loop wants streaming, not just the loop around it.

Deploy it as a job, not a route

The fix is structural, and it is small. Split the thing that accepts work from the thing that does work.

In practice that is four pieces:

1. The entry point. A normal function. It authenticates the caller, validates input, writes a runs row with status queued, enqueues the job, and returns { run_id }. It should finish in tens of milliseconds and never call a model.
2. The queue. Anything durable that supports retries and a visibility timeout. A Postgres table with SELECT ... FOR UPDATE SKIP LOCKED is enough to start, and it keeps the run state and the queue in one transaction, which removes a whole class of bugs.
3. The worker. A long-lived process, container, or durable workflow. It owns the loop, the tool calls, and the checkpointing. Give it a hard step ceiling and a hard cost ceiling.
4. The settle step. On success or failure, write the terminal status, the final output, the token counts, and fire the webhook or notification. The client polls the run id or listens for that event.

The client-facing behavior barely changes. You go from one hanging request to a fast POST plus a poll, and you get the ability to show progress, which users prefer anyway.

Checkpoint after every step

A background worker without checkpoints is just a slower way to lose work. The rule is that state gets written after each step completes, not at the end of the run.

A workable runs row holds the run id, the status, the step index, the serialized message list, an array of tool calls with arguments and results, running input and output token counts, and an updated_at you can alert on when a run goes quiet. Serialize the messages exactly as the API returned them. Reconstructing or filtering assistant content on resume is how you end up with 400 errors on the next turn.

Two rules make resume safe:

Idempotency keys on every external write. Derive the key from the run id plus the step index, for example run_8f21:step_7, and pass it to Stripe, your mailer, or your own API. A replayed step then becomes a no-op instead of a second charge. Any tool without an idempotency path is a tool you should not let an agent retry automatically.

A step ceiling and a cost ceiling that stop the loop. Not warnings, stops. Count tokens as you go and terminate the run with status budget_exceeded when it crosses the line. I wrote the cost side of this in how to cut your AI API bill, and the ceiling belongs in the worker regardless of what you spend.

Handle the four failures that actually happen

Every deployed agent hits the same short list. Handle them explicitly and most incidents turn into a slower run instead of an outage.

Rate limits and overload. Anthropic's SDKs retry transient failures including connection errors, rate limits, and 5xx server errors twice by default with exponential backoff, honoring retry-after when present, with a configurable max-retries option. Add your own cap on retries per run so a degraded upstream cannot eat the budget, and reduce worker concurrency under sustained 429s rather than retrying from 40 workers at once. The docs also note that a sharp increase in your own usage can trip acceleration limits, so ramp traffic gradually. More on this in how to handle LLM rate limits.

Tool failures. A tool that returns a 500 should return a structured error into the conversation, not throw and kill the loop. Give the model one useful sentence about what failed so it can pick a different path, and count the failure toward the step ceiling.

Payload size. The Messages API accepts requests up to 32 MB, the Batch API up to 256 MB, and the Files API up to 500 MB. Separately, Vercel Functions cap the request or response body at 4.5 MB, which is the limit most people hit first when an agent tries to return a large document through an API route. Write big outputs to object storage and return a URL.

Stuck runs. A run with no updated_at movement for longer than your longest legitimate step is stuck. Sweep for those on a schedule and either resume or fail them. Without a sweeper, a worker that died holding a lock leaves a run sitting in running forever.

Secrets, permissions, and blast radius

The agent runs with whatever credentials you hand the worker, and it will use them exactly as far as they reach.

Give each agent its own credential rather than the shared service key. Scope database access to the tables it needs, and put writes behind stored procedures or an API layer instead of raw table access. Keep the model provider key in the worker's environment only, never in anything the model can read back. Log which credential a run used alongside the trace, since "which key touched this record" is the first question in any incident.

Human approval is the other half of blast radius. Anything irreversible (payments, external emails, deletes, publishing) belongs behind a gate rather than inside the autonomous path. The mechanics of building those gates are in human in the loop AI agent.

The pre-launch checklist

Run this before the first real user touches it.

• Kill the worker mid-run and confirm the run resumes at the correct step.
• Replay a single step twice and confirm no duplicate side effect.
• Force a 429 and confirm backoff, then confirm the retry cap fires.
• Set the cost ceiling to something tiny and confirm the run stops instead of warning.
• Confirm every run writes a trace you can read end to end. If you cannot answer "what did it do at step 4" in under a minute, fix that first (agent observability covers the tracing setup).
• Deploy to internal traffic or one low-risk account first and read the traces by hand for a week. That slow rollout pattern is the whole idea behind THE CANARY METHOD, and it catches the failures no test suite predicted.

If you want the next one of these when it lands, they go out through the newsletter before anywhere else.

The bottom line

Deploying an AI agent is mostly a queueing problem wearing an AI costume. The model call is the part that already works. What breaks in production is duration, state, and side effects, and all three get solved by the same move: accept the job, return a run id, run the loop in a worker, and write state after every step. Do that first, then add the ceilings, the idempotency keys, and the traces. An agent that resumes from step 7 and stops at a budget line is worth more than a smarter one that starts over every time something times out.