Class notes · 24 October 2026
Why this topic
The agents we’ve built so far can use tools, both built-in and custom. Two problems remain:
- Memory, or context. How does an agent remember what has happened?
- Approvals. Sometimes a person has to approve or deny what the agent wants to do, or at least be asked first. The earlier Claude-to-Swiggy demo was full of these permission requests.
Models don’t remember
Say turn 1 of a chat is “I’m XYZ and I live in Hyderabad,” followed by a question. In turn 2, the model knows nothing about turn 1 unless turn 1 is sent to it again. Every call has to carry the earlier conversation with it.
The context window
The context window is everything the model reads on each call:
- the system prompt
- the tool definitions
- project instructions, if you’ve loaded them
- the whole conversation so far
- every tool’s input and output
It fills up with every turn. It matters because it’s the agent’s only working memory: this is how the agent “recalls” what it has done.
Context pressure
Every model call reports how many input tokens it read, and that number grows as the context grows. Large tool outputs are the usual culprit. Every model also has a maximum context size, and a long-running conversation keeps moving toward it.
Compaction
When the context nears its limit, agent harnesses compact it. There are two common approaches:
- Summarisation: replace older turns (say, the last 25) with a shorter summary. This is the most popular approach.
- Sliding window: keep only the most recent turns (say, the last 15) and drop everything older.
The right choice depends on what you’re building. Either way, the context gets cleaner and each call is cheaper.
In the Claude Agent SDK: compaction is automatic and summary-based. Near the limit, older history is summarised and the most recent exchanges are kept. You can also trigger it yourself by sending /compact. A summary can lose detail, so instructions that must last the whole run belong in the system prompt, not in an early message.
“But my model has a 1-million-token window”
Two reasons it still matters:
- Cost. Every call pays for everything in the window, so simple tasks end up paying for a long history they don’t need.
- Quality. Anthropic calls this context rot: as the context grows, the model’s ability to accurately recall what’s in it decreases.
Context management vs context engineering
This is a universal problem. It isn’t specific to Claude, to OpenAI’s Agents SDK, or to LangChain and LangGraph. The implementation changes from framework to framework, but the problem statement stays the same.
- Context management is keeping a growing context under control: compaction, summaries, deciding what to drop.
- Context engineering is the broader discipline: deciding what goes into the context at all, including prompts, tools, memory and retrieved information. Context management is one part of it.
Human in the loop
There are two kinds of agents:
- Fully autonomous: the agent decides its own actions.
- Supervised: the agent does the work, but certain actions need a person’s approval.
Example: a business posts 5–10 LinkedIn articles a day. Agents can pick topics from Google Trends and write the articles, but a person reviews each article before it’s posted.
Human in the loop can also be as simple as: the agent wants to delete some files; do you allow it?
Human in the loop is when an agent asks a person for acknowledgement before it acts.
In the Claude Agent SDK: an approval callback (can_use_tool) receives the tool calls that need permission. You can allow, deny with a reason, or change the input before it runs. Plan mode makes the agent propose its changes before it’s allowed to make them.
What context engineering manages
| Part | What it includes |
|---|---|
| Memory | Past conversations, i.e. the history |
| Prompts | The system prompt (the role) and the user’s prompts |
| Tools | Tool definitions, plus the tool calls the model made and their outputs |
The design questions: how do I manage all of this, where do I store it, and what is permanent versus temporary? That design work is context engineering.
Threads, sessions and users
- Single-user system: you can keep everything in one thread, but a thread that runs for a year becomes very expensive. In practice we start a new thread for a new topic and go back to an old thread to continue an old topic.
- Multi-user system: the agent has to remember things per user, and some things across users.
In the Claude Agent SDK: a thread is a session. You continue one by resuming its session ID, and your application keeps track of which session belongs to which user.
Where to store memory: ask two questions first
- Structure. What format is the data? Does it have a schema, is it semi-structured, or is it free text?
- Scope. Does it need to be permanent, or is it transient? If a user comes back to a conversation after six months, do you want to support that?
Examples:
- Schema-based and permanent → a relational database with backups.
- Semi-structured files, kept for only 30 days → blob storage holding JSON files.
Short-term vs long-term memory
| Type | Scope | Example |
|---|---|---|
| Short-term | One thread or session | Reopen an old ChatGPT chat and continue it; the model uses only that thread’s history |
| Long-term | Across threads and sessions | Ask ChatGPT “What do you know about me?”; it answers from memory built across all your chats |
What to store, and where
| You want to store | Typical storage |
|---|---|
| Everything | Any storage that fits; this is the simplest case |
| A summary | Semi-structured files, or a NoSQL database |
| The meaning (to search by similarity, e.g. “have I asked something like this before?”) | A vector database |
A vector database stores embeddings, numeric representations of text, so it can find content that’s similar in meaning even when the words differ. Vector stores are common for semantic recall. Agent tools like Claude Code, though, lean on plain files and search: Claude Code’s memory is a set of markdown files the agent reads and searches as it needs them.
There is no single right answer. Depending on the size of the system, memory might live on the local machine, in the cloud, or in plain files with no database at all. These are design decisions, and the choice depends on need.
Why this matters
These design questions come up in system design interviews, especially for engineers with five or more years of experience.
Further reading
Effective context engineering for AI agents, Anthropic Engineering
Next
So far we’ve built the Ask-Me Tutor (one turn, no tools) and the Downloads Folder Cleaner (an agent that inventories and organises a folder). Next we’ll build a lab exercise with a Claude agent that handles both memory and context.
