Guardrails, and Where to Go Next 🚦

A cap it cannot exceed

Every horror story about surprise API bills has the same shape: a loop nobody bounded, running unattended. The fix is not vigilance, it is arithmetic.

import json
from datetime import date
from pathlib import Path

PRICES = {
    "claude-haiku-4-5": (1.00, 5.00),
    "claude-sonnet-5": (3.00, 15.00),
    "claude-opus-5": (5.00, 25.00),
}


def cost_of(model, tokens_in, tokens_out):
    """Dollars for one call. Unknown models are priced as the dearest we know."""
    price_in, price_out = PRICES.get(model, (5.00, 25.00))
    return (tokens_in / 1_000_000) * price_in + (tokens_out / 1_000_000) * price_out


class SpendTracker:
    """Track spending per day, and refuse to go over the limit."""

    def __init__(self, limit_usd, path="spend.json", today=None):
        self.limit = limit_usd
        self.path = Path(path)
        self.today = (today or date.today()).isoformat()
        self.spent = self._load()

    def _load(self):
        if not self.path.exists():
            return 0.0
        try:
            data = json.loads(self.path.read_text())
        except json.JSONDecodeError:
            return 0.0
        return float(data.get(self.today, 0.0))

    def would_exceed(self, estimate):
        return self.spent + estimate > self.limit

    def record(self, amount):
        self.spent += amount
        self.path.write_text(json.dumps({self.today: round(self.spent, 6)}))

    def remaining(self):
        return max(0.0, self.limit - self.spent)


t = SpendTracker(limit_usd=0.05, path="demo-spend.json", today=date(2026, 8, 17))
print(f"one call costs ${cost_of("claude-haiku-4-5", 600, 300):.5f}")

for i in range(1, 30):
    c = cost_of("claude-haiku-4-5", 600, 300)
    if t.would_exceed(c):
        print(f"stopped at call {i}: ${t.spent:.4f} spent, limit ${t.limit}")
        break
    t.record(c)

print(f"remaining: ${t.remaining():.4f}")
Path("demo-spend.json").unlink()

one call costs $0.00210
stopped at call 24: $0.0483 spent, limit $0.05
remaining: $0.0017

Read what that actually does. Before each call it estimates the cost, asks whether that would break the limit, and refuses if so. The number lives in a file, so the cap survives restarts. It cannot be talked out of it, because it is not a request to the model, it is your own code declining to make the call.

Wire it in around the API call: check would_exceed before, call record after using the real usage numbers from the response. Estimate pessimistically, record accurately.

Counting tokens properly

To estimate before you spend, you need to know how big your request is. The API will tell you exactly, and it is worth being firm about this:

# Before an expensive call you can ask the API how many tokens your
# messages actually are, rather than guessing. This is the ONLY correct
# way to count tokens for these models: a general-purpose tokenizer from
# another ecosystem will give you a confidently wrong number.
#
#   count = client.messages.count_tokens(
#       model=cfg.model,
#       system=system_prompt,
#       messages=messages,
#   )
#   estimated_input = count.input_tokens
#
# Then estimate the output as your max_tokens ceiling, which is the worst
# case, and check that against the budget before spending anything.

def estimate_worst_case(input_tokens, max_tokens, price_in, price_out):
    return (input_tokens / 1_000_000) * price_in + (max_tokens / 1_000_000) * price_out


print(f"${estimate_worst_case(1200, 1000, 1.00, 5.00):.5f} worst case")
print(f"${estimate_worst_case(1200, 1000, 5.00, 25.00):.5f} on the big model")

$0.00620 worst case
$0.03100 on the big model

Errors that say what to do

Right now, any API problem exits with a stack trace. Every one of these has an obvious human-readable meaning:

import anthropic


def ask_safely(client, **kwargs):
    """Turn every documented failure into a sentence a human can act on.

    The SDK already retries rate limits and server errors for you with
    exponential backoff, so there is no hand-rolled retry loop here.
    Adding one on top usually makes things worse, not better.
    """
    try:
        return client.messages.create(**kwargs), None
    except anthropic.AuthenticationError:
        return None, "Your API key was rejected. Check ANTHROPIC_API_KEY."
    except anthropic.RateLimitError:
        return None, "Rate limited even after retries. Wait a minute and try again."
    except anthropic.BadRequestError as exc:
        return None, f"The request was malformed: {exc}"
    except anthropic.NotFoundError:
        return None, "That model name does not exist. Check your config."
    except anthropic.APIConnectionError:
        return None, "Could not reach the API. Check your internet connection."
    except anthropic.APIStatusError as exc:
        return None, f"The API returned an error ({exc.status_code}). Try again shortly."

Note what is not there: a hand-written retry loop. The SDK already retries rate limits and server errors with exponential backoff. Wrapping your own retries around that gives you retries of retries, which turns a brief hiccup into a long, expensive stall. If you want different behaviour, configure max_retries on the client rather than building a second mechanism.

A tool loop that cannot run away

Chapter 8's loop breaks when the model stops asking for tools. If something unexpected comes back, that is an unbounded loop making paid API calls. Bound it:

def tool_loop_guard(max_rounds=5):
    """A generator that yields round numbers and stops. The point is that
    the loop CANNOT run forever, no matter what comes back."""
    for i in range(1, max_rounds + 1):
        yield i


rounds = list(tool_loop_guard(5))
print("rounds allowed:", rounds)
print("a runaway loop stops after", len(rounds), "API calls, not infinity")

rounds allowed: [1, 2, 3, 4, 5]
a runaway loop stops after 5 API calls, not infinity

Five rounds is generous for a personal assistant; genuine multi-step work rarely needs more. If you hit the limit, tell the user plainly that it gave up rather than pretending the answer is complete.

The habits worth keeping

• Cap before you launch, not after a surprise. Set the limit low enough that hitting it is annoying rather than expensive.
• Estimate pessimistically, record accurately. Assume the full max_tokens before the call; record what really happened after.
• Bound every loop that spends money. Every single one.
• Log the cost where you can see it. A dim line showing today's total after each reply is the cheapest possible defence against drift.

Things worth knowing that this build skipped

• Prompt caching. If your system prompt and notes are stable, the API can cache that prefix and charge a fraction for it. The single biggest cost win available once your assistant is real.
• The tool runner. The SDK can write chapter 8's loop for you from decorated Python functions. Worth adopting now that you know what it is doing.
• Embeddings. The upgrade to chapter 10's word matching, for when "plumber" should find "boiler engineer".
• Structured output. You can require a reply to match a schema, which turns the model into something you can call like a function rather than parse like prose.

Where to take it

• A web interface with FastAPI (Lesson 44), so you can use it from your phone on your own network.
• A daily briefing: run it on a schedule, have it read your calendar and notes, and write you a short summary each morning.
• Tools for the things you actually do. This is the real answer. The assistant becomes useful in proportion to how well its tools fit your life, and nobody else can write those.
• A local model (Lesson 60) if you would rather nothing left your machine at all. Slower and less capable, and the privacy is absolute.

Mark chapter complete +150 XP