Words as They Arrive ⚡

The problem with waiting

Ask your chat loop for something long and you get several seconds of nothing, then a wall of text. The program is working perfectly. It just looks like it has crashed.

Streaming fixes the feeling, not the speed. The full answer takes exactly as long either way, but you start reading after a quarter of a second instead of staring at a blank line.

The smallest streaming program

import anthropic

client = anthropic.Anthropic()

with client.messages.stream(
    model="claude-haiku-4-5",
    max_tokens=1000,
    messages=[{"role": "user", "content": "Explain what a variable is, in two sentences."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

print()

Three things changed from chapter 3:

• client.messages.stream(...) instead of .create(...).
• It is used with with, because a stream is a resource that must be closed properly. Lesson 21 covered with for files; identical idea.
• You loop over stream.text_stream, which hands you small pieces of text as they arrive.

Why end="" and flush=True

These two arguments are doing real work, and it is worth understanding rather than copying.

# print() normally waits for a newline before actually showing anything,
# because writing to a terminal is slow and buffering is faster. When you
# print word-by-word with no newlines, that buffering is exactly wrong:
# the text sits in memory and appears all at once anyway.
#
# flush=True says "show it now". end="" says "no newline after this".
for word in ["Streaming ", "means ", "you ", "see ", "it ", "arrive."]:
    print(word, end="", flush=True)
print()

print("...and end='' is what stopped each piece landing on its own line.")

Streaming means you see it arrive.
...and end='' is what stopped each piece landing on its own line.

end="" stops print adding a newline after every fragment, which would otherwise give you one word per line. flush=True forces the text onto the screen immediately instead of letting Python buffer it, which would defeat the entire point by showing everything at the end anyway.

Getting the whole message back

Streaming gives you pieces, but you still need the finished thing: to store in memory, and to read usage numbers off. The stream will assemble it for you.

with client.messages.stream(
    model="claude-haiku-4-5",
    max_tokens=1000,
    messages=messages,
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

    # After the stream finishes, ask for the assembled message.
    # This is how you get usage numbers and stop_reason while streaming.
    final = stream.get_final_message()

print()
print("tokens out:", final.usage.output_tokens)

get_final_message() hands back the same kind of object .create() would have returned, with .content, .usage and .stop_reason all present. So you get the responsive feel and the useful metadata, rather than choosing.

The streaming chat loop

Update chat.py:

import anthropic

client = anthropic.Anthropic()
MODEL = "claude-haiku-4-5"

messages = []

print("Jarvis ready. Ctrl-C to leave.")

try:
    while True:
        user_input = input("\nyou> ").strip()
        if not user_input:
            continue

        messages.append({"role": "user", "content": user_input})

        print("jarvis> ", end="", flush=True)
        with client.messages.stream(
            model=MODEL,
            max_tokens=1000,
            messages=messages,
        ) as stream:
            for text in stream.text_stream:
                print(text, end="", flush=True)
            final = stream.get_final_message()
        print()

        # Rebuild the reply text from the finished message, rather than
        # gluing together the pieces we printed. One source of truth.
        reply = "".join(b.text for b in final.content if b.type == "text")
        messages.append({"role": "assistant", "content": reply})

except KeyboardInterrupt:
    print("\n\nBye.")

Run it and ask for something long, like a recipe or an explanation. It writes to you now, rather than at you.

One subtlety worth noticing

The reply stored in memory is rebuilt from final.content, not from concatenating the fragments you printed. Both would usually work, but there is only one correct source of truth for what the model actually said, and it is the finished message. Building your history out of display side effects is the kind of shortcut that produces a bug six chapters later that nobody can find.

If it went wrong

• Text still appears all at once You dropped flush=True, or your terminal is aggressively buffering. Some IDE consoles do this; try a real terminal.
• Every word on its own line You dropped end="".
• AttributeError on final You called get_final_message() outside the with block. It has to happen while the stream is still open.

Mark chapter complete +150 XP