Teaching Jarvis About You 📚

The problem, and the shape of the answer

The model's knowledge is frozen at its training cutoff and contains nothing private. It has never seen your journal, your company's docs, or last week's meeting notes. You cannot retrain it (that costs millions and is nobody's first move). Instead you do something far simpler: find the relevant piece of your own data and paste it into the prompt.

This is called retrieval-augmented generation, RAG, and despite the intimidating name it is exactly what it says: retrieve the relevant text, augment the prompt with it, then generate. You already have every skill it needs.

The simplest version: just paste it in

If your data is small, do not overthink it. Put the whole thing in the prompt. Modern context windows are huge (hundreds of pages), so "stuff the context" is a genuinely good first answer.

from pathlib import Path


def ask_about_document(document_text, question):
    """Answer a question using a document pasted straight into the prompt."""
    import anthropic                      # imported here so the demo line below runs
    client = anthropic.Anthropic()
    return client.messages.create(
        model="claude-sonnet-5",
        max_tokens=1024,
        system="Answer using ONLY the provided document. If the answer is not "
               "in it, say so plainly. Do not use outside knowledge.",
        messages=[{
            "role": "user",
            "content": f"Document:\n\n{document_text}\n\n---\n\nQuestion: {question}",
        }],
    ).content[0].text


# notes = Path("meeting_notes.txt").read_text(encoding="utf-8")
# print(ask_about_document(notes, "What did we decide about the budget?"))
print("Pattern: read the file (Lesson 21), paste it in, ask the question.")

Pattern: read the file (Lesson 21), paste it in, ask the question.

When your data is too big to paste

A thousand documents will not fit in any context window, and pasting all of them for every question would be slow and ruinously expensive. So you retrieve only the few pieces that are relevant to this question. The question is: how do you find them?

Two approaches, and you should know both:

Keyword search you can already build with what you know. Semantic search is the new idea, and it is worth understanding because it is the engine under most modern RAG.

Embeddings: turning meaning into numbers

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. The crucial property: texts with similar meanings get similar vectors, even if they share no words. "How do I reset my password" and "I forgot my login" land close together; "how to bake bread" lands far away.

You get embeddings from a model (an embedding model, cheaper and smaller than a chat model). Then finding relevant text is just finding the nearest vectors. Here is the core maths, on toy vectors, in pure Python so you can see there is no magic:

import math

def cosine_similarity(a, b):
    """How aligned are two vectors? 1.0 = same direction, 0 = unrelated."""
    dot = sum(x * y for x, y in zip(a, b))
    size_a = math.sqrt(sum(x * x for x in a))
    size_b = math.sqrt(sum(y * y for y in b))
    return dot / (size_a * size_b)


# Pretend these came from an embedding model. In reality they have hundreds
# of dimensions; three is enough to show the idea.
password_reset = [0.9, 0.1, 0.2]
forgot_login   = [0.85, 0.15, 0.25]     # different words, similar meaning
baking_bread   = [0.1, 0.9, 0.3]        # unrelated

print(f"reset vs forgot-login: {cosine_similarity(password_reset, forgot_login):.3f}")
print(f"reset vs baking bread: {cosine_similarity(password_reset, baking_bread):.3f}")

reset vs forgot-login: 0.996
reset vs baking bread: 0.271

"Reset password" and "forgot login" score 0.998 despite sharing no words, because their meanings align. "Baking bread" scores far lower. That single number, cosine similarity, is how semantic search decides what is relevant.

The full RAG pipeline

Putting it together, the shape of a real retrieval system:

1. Chunk your documents into passages (a few paragraphs each), because you want to retrieve relevant sections, not whole books.
2. Embed every chunk once, and store the vectors. For a small project a list works; at scale you use a vector database (Chroma, FAISS, pgvector, and others) that finds nearest neighbours fast.
3. At question time, embed the question, find the few nearest chunks, and paste those into the prompt, exactly like the simple version above.
4. Generate, pinned to those chunks, and ideally cite which chunk each claim came from so the user can check.

# The shape of it (using a hypothetical embed function).
# Real code would use client.embeddings or a library; the logic is this.

def build_index(chunks, embed):
    """Embed every chunk once, up front."""
    return [(chunk, embed(chunk)) for chunk in chunks]


def retrieve(question, index, embed, top_k=3):
    """Find the top_k chunks most similar in meaning to the question."""
    q_vector = embed(question)
    scored = [(cosine_similarity(q_vector, vec), chunk) for chunk, vec in index]
    scored.sort(reverse=True)
    return [chunk for _score, chunk in scored[:top_k]]


def cosine_similarity(a, b):
    import math
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.sqrt(sum(x*x for x in a)) * math.sqrt(sum(y*y for y in b)))


# Then: context = "\n\n".join(retrieve(question, index, embed))
#       ask_about_document(context, question)
print("Chunk, embed, store. At query time: embed question, find nearest, stuff, generate.")

Chunk, embed, store. At query time: embed question, find nearest, stuff, generate.

RAG versus giving it a tool

There is overlap with Lesson 57. A "search my notes" tool (Lesson 57) lets the model decide when to retrieve. RAG-by-stuffing retrieves before the model runs, every time. Modern assistants often combine them: retrieval as a tool the model calls when it judges it needs to. Both are valid; the tool version is more flexible, the stuffing version is simpler and more predictable.

def retrieve_by_keyword(question, chunks, top_k=2):
    """Score each chunk by how many of the question's words it contains."""
    q_words = set(question.lower().split())
    scored = []
    for chunk in chunks:
        chunk_words = set(chunk.lower().split())
        overlap = len(q_words & chunk_words)      # set intersection (Lesson 14)
        scored.append((overlap, chunk))
    scored.sort(reverse=True)
    return [chunk for score, chunk in scored[:top_k] if score > 0]


notes = [
    "The budget meeting is on Friday. We approved 5000 for new laptops.",
    "Remember to water the office plants twice a week.",
    "The laptops should be ordered from the approved supplier by month end.",
]

for chunk in retrieve_by_keyword("what was decided about laptops budget", notes):
    print("-", chunk)

- The laptops should be ordered from the approved supplier by month end.
- The budget meeting is on Friday. We approved 5000 for new laptops.