The costs of badly spending

The work buckette puts a budget around is not cheap work. An agent session that reads a large codebase costs a few dollars to a few tens of dollars. A nightly job that calls a paid API costs hundreds. A sweep of evaluations or a training run on rented GPUs costs thousands. Each one is started by code, and nobody approves its spending one call at a time.

Most runs go fine. The ones that go badly tend to go badly in the same few ways, and each way has a price. This post puts a number on four of them, and says what buckette does in each case. The numbers are illustrative: round prices and sizes, chosen to be plausible, not taken from any customer or price list. Put your own in; the arithmetic is the point.

1. The runaway loop

An agent works on a ticket. A tool it relies on starts failing, and the agent does what agents do: it reads the error, thinks, and tries again. Each turn sends the whole context back to the model.

  • Context per turn: 150,000 tokens.
  • Price: $3 per million input tokens, so $0.45 a turn.
  • A normal run: 40 turns, $18.
  • The loop: one turn every 30 seconds from 10 pm until someone looks at 8 am, 1,200 turns, $540. On a model priced five times higher, $2,700.

Nothing in that loop is an error your monitoring would page on. Every call succeeded; the model answered every time.

What buckette does. Give each run its own budget: split a child bucket off the team’s bucket, sized a third above a normal run, and hand the run a key that can spend only that child.

POST /buckets/<team>/split
{"srAllocation": {"tokens": 8000000},
 "srExpiresAt": "2026-11-11T06:00:00Z"}

The run can spend its 8 million tokens and no more: the call that does not fit is refused, so the loop above stops after about 53 turns, around $24, instead of $540. A threshold on the child sends a webhook when a quarter of it is left, so a person hears about the run while it is still running. And if nobody ends the run, the child expires at the time you gave: it is archived and the run’s key stops working with it. A run that ends normally reclaims its child, and what it did not spend goes back to the team.

2. The retry storm

A nightly job enriches 10,000 records through a paid API at $0.02 a call: $200 a night. One night the provider slows down. Your client times out after five seconds and retries, up to five times. The provider finishes the calls your side gave up on, and bills them.

  • Normal night: 10,000 calls, $200.
  • That night: up to 60,000 calls, $1,200, for the same 10,000 answers.

What buckette does. Count calls as a dimension of the job’s bucket, refilled each night with a margin (say 12,000 calls), so the worst night costs $240, not $1,200. When the job runs out, the refusal carries an estimate of when the budget’s refills will cover the call, so the job can choose between waiting a few seconds, queueing the rest for the next refill, and stopping. The key the job uses can also carry its own rate limit in requests a minute, which keeps a client with no backoff from turning a slow provider into a flood: past it, buckette answers 429 before anything is spent.

3. The hold nobody released

A research team shares a pool of rented GPU hours, 100 hours a day at $2.50 an hour. The machines are paid for whether or not anyone uses them. A worker reserves 40 hours for a training job on Friday evening, then crashes before it starts or gives anything back.

  • The 40 hours stay held until someone notices on Monday morning, and every job queued behind them waits.
  • Over two and a half days, that is 100 GPU hours paid for and unused: $250 of idle machines, and a weekend of experiments that did not run.

This one costs money that was never spent on anything. The budget did its job (nobody overspent) and still lost.

What buckette does. A hold can carry an expiry. A worker that runs for a long time refreshes its hold every few minutes, and a worker that dies stops refreshing it, so the hold lapses on the scheduler’s next pass and the hours go back to the pool. Work that was queued on the pool (an Async consume or a queued claim) is served on a following pass, with a webhook or a callback when it goes through.

4. The batch that did not fit

A team shares a monthly allowance of 30 million tokens between a support assistant and internal batch work. On the 3rd, someone starts a backfill over 20,000 documents at about 2,000 tokens each: 40 million tokens, more than the whole month holds.

  • By the 5th the allowance is gone and the backfill stops three quarters of the way through. $90 of tokens bought a backfill nobody can use.
  • The support assistant is refused until the 1st. That costs more than the tokens: three and a half weeks of customers waiting for a human.

What buckette does. Ask before starting. POST /buckets/<allowance>/eta with the batch’s estimate answers with a time, or with “never” and the reason (here, the request is larger than the budget can ever hold), so the batch is resized or split before it spends anything. And so that no batch can starve the assistant, give the two kinds of work classes on the allowance: a floor keeps part of the balance back from the batch class and releases it as the reset approaches, so what the assistant did not need is still spent by the end of the month.

Two ways to lose money

The four stories lose money in two different ways. The loop and the storm spend too much. The forgotten hold and the oversized batch spend too little where it mattered: machines idle, customers refused. A limit alone fixes the first kind and can make the second worse. That is why buckette records both sides, refused work and allowance that expired unspent, as our first post describes.

What the guard costs

Put a budget around a run and the run makes a few more calls: one to split its bucket, one to create its key, one consume (or a hold and a commit) per model call, and one to reclaim what is left. For the agent in the first story, that is about 45 small requests next to 40 model calls of $0.45 each. None of buckette’s plans charges by the call.

So the comparison is not close. A run that costs tens of dollars when it goes well can cost hundreds or thousands when it goes badly, and the guard around it costs a few requests. You do not need to prevent many bad nights for it to pay for itself; for most teams, one is enough.

Where to start

You cannot size a run’s budget without knowing what a good run costs. Start with buckets in reporting mode for a week: they refuse nothing and record every call, and their history is how you find the normal run, and so the runs that are not. The getting started page sets up a first budget in a few minutes, and the guide for agents has every route mentioned here.

If one of these stories is yours, with your own numbers, write to hello@buckette.co. We would like to hear what it cost.