A Conversation, Not a Goldfish 🧠

Watch it forget

Two calls in a row, the second referring to the first:

import anthropic

client = anthropic.Anthropic()

first = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=100,
    messages=[{"role": "user", "content": "My name is Ada. Remember it."}],
)
print("1:", first.content[0].text)

second = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=100,
    messages=[{"role": "user", "content": "What is my name?"}],
)
print("2:", second.content[0].text)

It has no idea. Not because it is broken, but because the API has no memory. Each call is a fresh start. It knows nothing except exactly what you put in messages for that one request.

The fix, which is just a list

If the model only knows what is in messages, then remembering is simply: keep the list, and add to it. Every turn, you send the whole conversation so far.

# What you send on turn one
messages = [
    {"role": "user", "content": "My name is Ada."},
]

# What you send on turn two: the whole story so far
messages = [
    {"role": "user", "content": "My name is Ada."},
    {"role": "assistant", "content": "Nice to meet you, Ada."},
    {"role": "user", "content": "What is my name?"},
]

print(f"turn two sends {len(messages)} messages")
print("roles:", [m["role"] for m in messages])

turn two sends 3 messages
roles: ['user', 'assistant', 'user']

That is genuinely the entire trick. Memory is a Python list you keep appending to.

The chat loop

Now a real program. Save it as chat.py:

import anthropic

client = anthropic.Anthropic()
MODEL = "claude-haiku-4-5"

# Created ONCE, outside the loop. This list is the memory.
messages = []

print("Jarvis ready. Ctrl-C to leave.")

try:
    while True:
        user_input = input("\nyou> ").strip()
        if not user_input:
            continue

        # 1. your turn joins the history
        messages.append({"role": "user", "content": user_input})

        # 2. send the WHOLE history, not just the latest line
        response = client.messages.create(
            model=MODEL,
            max_tokens=1000,
            messages=messages,
        )

        reply = "".join(b.text for b in response.content if b.type == "text")
        print(f"jarvis> {reply}")

        # 3. its turn joins the history too
        messages.append({"role": "assistant", "content": reply})

except KeyboardInterrupt:
    print("\n\nBye.")

Run it and have an actual conversation:

$ python3 chat.py
Jarvis ready. Ctrl-C to leave.

you> My name is Ada and I am learning Python.
jarvis> Hello Ada. Nice to meet a fellow Python learner.

you> What is a list, in one sentence?
jarvis> A list is an ordered, changeable collection of items in square brackets.

you> What was my name again?
jarvis> Your name is Ada.

It remembers. You built memory.

The three lines that matter

Everything else is decoration. These are the load-bearing ones:

1. messages.append(...) with your question before the call, so it joins the history.
2. messages=messages in the call, sending the whole history rather than just the latest line.
3. messages.append(...) with the reply after, so its own answers are remembered too.

Miss the third and something wonderfully confusing happens: it remembers your questions but not its own answers, and starts contradicting itself. Worth breaking on purpose once, just to see it.

Why you resend everything, every time

This strikes everyone as wasteful, and it is worth being precise about the trade.

The API is stateless: it stores nothing between calls. That is a deliberate design choice with real benefits. Your conversation is not sitting on somebody else's server. You can edit the history, delete messages, or start again, and there is no hidden state to fight. The cost is that a long conversation resends a lot of tokens.

You can measure exactly how that grows:

def tokens_sent_on_turn(turn, tokens_per_message=120):
    """You resend everything, so turn N sends N messages' worth."""
    return turn * 2 * tokens_per_message // 2   # user + assistant pairs


for turn in (1, 5, 10, 20):
    t = tokens_sent_on_turn(turn)
    cost = (t / 1_000_000) * 1.00
    print(f"turn {turn:>2}: ~{t:>5} tokens in, ${cost:.6f}")

turn  1: ~  120 tokens in, $0.000120
turn  5: ~  600 tokens in, $0.000600
turn 10: ~ 1200 tokens in, $0.001200
turn 20: ~ 2400 tokens in, $0.002400

Turn 20 sends twenty times the tokens of turn 1. The words are cheap; the repetition is what adds up.

Trimming, so it cannot grow forever

The simplest good-enough policy: keep the most recent few exchanges. Here it is as a function you can test without spending a penny.

def trim(messages, keep_pairs=10):
    """Keep only the most recent `keep_pairs` user/assistant exchanges.

    Slicing from the end keeps the newest. Keeping an even number means
    the history always starts on a user turn, which is what the API wants.
    """
    keep = keep_pairs * 2
    if len(messages) <= keep:
        return messages
    return messages[-keep:]


# a pretend history: 14 messages, alternating user and assistant
history = []
for i in range(7):
    history.append({"role": "user", "content": f"question {i}"})
    history.append({"role": "assistant", "content": f"answer {i}"})

print("before:", len(history), "messages, first role", history[0]["role"])
short = trim(history, keep_pairs=3)
print("after :", len(short), "messages, first role", short[0]["role"])
print("kept  :", [m["content"] for m in short])

before: 14 messages, first role user
after : 6 messages, first role user
kept  : ['question 4', 'answer 4', 'question 5', 'answer 5', 'question 6', 'answer 6']

Note it keeps messages in pairs, so the history never begins with an assistant reply, which the API would reject.

If it went wrong

• It answers but forgets immediately You are almost certainly creating messages inside the loop instead of outside it. It must be made once, before while True.
• A BadRequestError about roles The first message must be from user, and content cannot be empty. The if not user_input: continue line guards the second case.
• Ctrl-C prints a red stack trace That is KeyboardInterrupt doing its job. The try wrapper above turns it into a polite goodbye.

Mark chapter complete +150 XP