Run API jobs in batches

If nobody is waiting on the answer, you shouldn’t pay full price for it. The Anthropic Message Batches API takes a list of ordinary requests, runs them asynchronously, and bills every token at half the standard rate. This guide sets up a project for that from scratch: a spend cap, an API key handled so it never leaks, a clean Python environment, and a batch run that records what each request cost.

It assumes you have API credits in the Claude Console. That is a different thing from a Claude subscription — see subscriptions, APIs, and local models .

1. Give the project its own workspace and cap

In the Console, a workspace is the unit you cap and report on. Make one per project:

1. Settings → Workspaces → Create, named for the project.
2. Open its Limits tab and set a monthly spend limit. That is the hard ceiling: requests in the workspace stop billing past it, whatever your code does.
3. Settings → API keys → Create key, and pick the new workspace. Name the key for the job.

Usage and cost now show up per workspace, so the experiment’s spend is never mixed with anything else.

2. Store the key without ever showing it

API keys leak in boring ways: echoed to a terminal to “check it’s there”, passed as a command-line flag (anyone on the machine can read it with ps), or pasted into an AI assistant’s chat, which keeps a transcript. Avoid all three: put the key in a file only you can read, and have programs read the file.

Run this in your own terminal — not inside an agent session:

mkdir -p ~/.config/anthropic && (umask 077; read -rs -p "API key: " k && printf '%s' "$k" > ~/.config/anthropic/research.key; unset k); echo

read -s doesn’t echo what you paste, and umask 077 makes the file readable only by you (mode 600). Check it without printing the value:

stat -c 'mode=%a bytes=%s' ~/.config/anthropic/research.key    # macOS: stat -f 'mode=%Lp bytes=%z'

To check that the key works without spending anything, list the models. The header goes to curl on stdin (-K -), so the key never appears in the process list:

{ printf 'header = "x-api-key: '; tr -d '\r\n' < ~/.config/anthropic/research.key; printf '"\n'; } | curl -s -o /dev/null -w 'HTTP %{http_code}\n' -K - -H "anthropic-version: 2023-06-01" https://api.anthropic.com/v1/models

HTTP 200 means the key is good. 401 means it isn’t — fix the file rather than printing it.

If you work with an AI coding agent, tell it the path to the key file, never the key. It can load the file in code and you never have to show it the value.

3. A Python environment just for this project

uv (opens in new tab) makes a virtual environment — a folder holding this project’s own packages, separate from the system Python — in a couple of seconds:

cd ~/my-batch-project
uv venv .venv
uv pip install --python .venv/bin/python anthropic

Run scripts with .venv/bin/python. Deleting .venv/ removes everything.

4. Shape the work for a batch

A batch is a list of independent, single-shot requests. That has three consequences:

• No back-and-forth inside a batch. If the work needs a loop (ask, read the answer, ask again), run it in rounds: a fan-out batch of independent sub-questions, then a second batch or one normal call that combines the answers.
• Results come back in any order. Give every request a custom_id and match on it.
• You wait. Most batches finish within an hour; the maximum is 24 hours. Results stay available for 29 days.

A batch holds up to 100,000 requests or 256 MB. Prompt caching still works, and its discount stacks with the batch discount: put shared context (a long document, instructions) in the system prompt with cache_control. Cache hits inside a batch are best-effort, because requests run concurrently.

5. Submit, wait, collect — and record the cost

import csv, time
from pathlib import Path
import anthropic

key = Path("~/.config/anthropic/research.key").expanduser().read_text().strip()
client = anthropic.Anthropic(api_key=key)

questions = {
    "q1": "What are the main open problems in X?",
    "q2": "Which datasets are standard for evaluating Y?",
}

batch = client.messages.batches.create(requests=[
    {
        "custom_id": cid,
        "params": {
            "model": "claude-opus-5-5",
            "max_tokens": 16000,
            "messages": [{"role": "user", "content": text}],
        },
    }
    for cid, text in questions.items()
])
print("submitted", batch.id)

while (b := client.messages.batches.retrieve(batch.id)).processing_status != "ended":
    time.sleep(60)

with open("usage.csv", "w", newline="") as f:
    w = csv.writer(f)
    w.writerow(["custom_id", "status", "input", "output", "cache_read", "cache_write"])
    for r in client.messages.batches.results(batch.id):
        if r.result.type == "succeeded":
            m, u = r.result.message, r.result.message.usage
            Path(f"{r.custom_id}.md").write_text(
                "".join(blk.text for blk in m.content if blk.type == "text"))
            w.writerow([r.custom_id, "ok", u.input_tokens, u.output_tokens,
                        u.cache_read_input_tokens or 0, u.cache_creation_input_tokens or 0])
        else:
            w.writerow([r.custom_id, r.result.type, 0, 0, 0, 0])

errored results with an invalid_request error need the request fixed; other errors and expired results can simply be resubmitted.

6. Measure whether it was worth it

• Per request: usage.csv gives exact token counts. Multiply by the batch rates on the pricing page (opens in new tab) to get the cost of each request.
• Against a control: send three to five of the same prompts through the normal Messages API. Compare cost, answer quality, and wall-clock time. The time you waited is what you paid for the discount.
• Ground truth: the Console’s usage and cost pages, filtered to the project’s workspace.

7. Wrap it in a command: api-batch

Once the pattern works, the script above turns into a small CLI that treats each study as a named project — the way amux treats agent sessions. This is what we run in the lab:

api-batch init     thesis --keyname research-key       # project dir, key file PATH, budget
api-batch add      thesis background.docx              # docx -> markdown + images (pandoc)
api-batch ask      thesis -f questions.txt             # one question per line
api-batch estimate thesis --model opus --effort medium # free: token count + worst-case cost
api-batch submit   thesis --model opus --effort medium # bills; refuses past the budget
api-batch ls                                           # every project, state, spent/budget
api-batch collect  thesis                              # answers + usage.csv + ledger
api-batch show     thesis                              # read the latest run's answers
api-batch synth    thesis "Combine the answers into a two-page summary"

The parts worth copying into your own version:

• A budget per project. submit sums the ledger of past batches, adds the worst case for this one (every answer at max_tokens), and refuses if the total passes the cap. The provider’s workspace limit is still the real backstop; this just fails earlier and more politely.
• Key by path, never by value. The project stores the key file’s name; the CLI reads the file at run time. Nothing about the key is printed, logged, or passed on a command line.
• A ledger. Every collect appends one line per batch (date, id, model, requests, cost, minutes), so “what has this project cost so far” is a cat.
• Remote. api-batch -H <host> <cmd> runs the same command over SSH on another machine, and ls --all lists projects across all of them. Each machine holds its own copy of the key file.
• A human presses submit. estimate is free and an agent may run it freely; submit spends money, so in our setup the agent shows the estimate and stops, and a person runs submit. We learned this the usual way: a sentence that merely described the submit command was read as a go-ahead, and two identical batches ran.

What one real run looked like: a two-page .docx with two screenshots, one question, Claude Fable 5.1 at medium effort. Estimate said $0.42 worst case; the batch finished in 2 minutes and cost $0.15 for a 1,500-word answer (5,032 output tokens; the 3,600 input tokens were a cache write).

The CLI is a single Python file with no dependencies beyond the provider SDK (it creates its own venv on first run). It is open source under the BSD 2-Clause license: code at github.com/iblues76/api-batch (opens in new tab) , documentation at api-batch.software.nuilab.org (opens in new tab) .

Where to go next

• How API access differs from a subscription: subscriptions, APIs, and local models .
• Splitting work between a frontier API and a local model: local and frontier models .