Context compaction

What is context compaction?

Context compaction is what an AI assistant does when a long session runs out of room. It summarises the conversation so far and starts a fresh context window from that summary. Anthropic's engineering guidance defines it as taking a conversation nearing the context window limit, summarising its contents, and reinitiating a new context window with the summary. The session carries on. The raw history behind it does not.

Worth saying at the top, because the word is overloaded in our field: this is a different thing from table compaction, which merges many small Parquet files in a lakehouse into fewer large ones. Same verb, different layer. One rewrites files on storage without changing a row, the other rewrites a conversation into a shorter version of itself and does lose detail.

The version most people actually meet: an assistant has been working with you for an hour, and then suddenly forgets a rule you set at the very beginning. That is usually compaction. The rule was in the part of the conversation that got summarised away.

What survives compaction and what quietly does not

The dividing line is simple. Anything re-injected from disk survives, because it is reloaded rather than remembered. Anything that only ever existed as a chat message depends entirely on the summary.

Claude Code documents the split for its own setup. The system prompt is untouched, since it was never part of message history. Project-root CLAUDE.md files and auto memory are re-injected from disk. Rules with paths: frontmatter and nested CLAUDE.md files in subdirectories are lost until a matching file is read again, because they only entered the conversation when a trigger file was read. Skill bodies come back too, but capped at 5,000 tokens per skill and 25,000 tokens in total, with the oldest dropped first.

The summary itself is a judgement call made by a model. Anthropic describes Claude Code's version as preserving architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs or messages. Sensible priorities. They are also priorities about the state of the work, not about the constraints you put around it.

Which is why an instruction from your first message is the classic casualty. "Never write to the production warehouse." "Keep all output in Dutch." "This client is billed per hour, so no speculative refactors." Those are one-line asides with no follow-up thread, and a summariser optimising for where the work stands has little reason to carry them forward. Anthropic's own warning is that compacting too aggressively loses subtle context whose importance only becomes apparent later.

Compaction versus dropping old messages

The blunter alternative is trimming: count the tokens in the history and cut the oldest messages once you approach the limit. LangChain's memory documentation is honest about what that costs, noting that the problem with trimming or removing messages is that you may lose information from the culling of the message queue. Compaction is the answer to exactly that. Summarise instead of delete, so something of the older turns survives in compressed form.

Doing nothing is worse than either. OpenAI's guidance on conversation state warns that a large prompt runs the risk of exceeding the allocated context window for a model, and that tokens generated in excess of the limit may be truncated in API responses. Compaction at least degrades on purpose.

It has also become an API feature rather than only an assistant behaviour. OpenAI's Responses API runs server-side compaction once context crosses a configured token threshold, and exposes a standalone endpoint you can call yourself to get back a compacted context window for the next call. The compaction item it returns is opaque and not intended to be human-interpretable, which is a detail worth holding on to: you cannot read what was kept.

Auto-compaction versus a deliberate handoff

Auto-compaction fires when you hit the ceiling. It compresses whatever you happened to be in the middle of, and it guesses at what mattered. Claude Code compacts automatically as you approach the limit so a full window does not end the session, which is good behaviour, but the timing is decided by your token count rather than by the shape of the work.

A deliberate handoff is you picking the moment and the contents. Claude Code's /compact frees context by summarising the conversation, and passing focus instructions such as /compact focus on the auth bug fix keeps what you choose instead of what the automatic pass guesses is important. /clear goes further and starts a conversation with empty context, which is the better move when the next task is unrelated rather than a continuation.

The habit that follows: compact before you start a long new task, not after the context indicator turns red.

Why a written artifact beats trusting the summary

The durable fix is to stop treating the conversation as storage. Put the decisions in a file in the repository, and the assistant reads them back instead of remembering them. Anthropic calls the general technique structured note-taking, or agentic memory, where the agent regularly writes notes persisted to memory outside of the context window and pulls them back into the context window later.

For a data or BI project that means the things you would otherwise keep repeating in chat get written down: naming conventions for measures, which environment is safe to write to, the calculation that was already fixed and should not be re-derived, the client's reporting calendar. It costs five minutes and it removes a whole class of surprise.

The asymmetry is the point. A file on disk survives compaction, survives /clear, and is still there when someone else picks the work up next week. A summary survives none of that reliably.

What to watch out for with context compaction

• Restate hard constraints after a compaction. If a session has been running long enough to compact, assume the rules from your opening message are no longer in play and say them again, or better, put them in a file the assistant reloads.

• A confident assistant is not a remembering one. After compaction the model treats its own summary as the record. If the summary is thin or wrong on a point, nothing in the session raises a hand about it.

• Compaction is not free. Summarising a nearly full window is itself a model call over that window, so it costs tokens and time at the moment it runs.

• A longer session is not a better one. If the task has genuinely changed, clearing beats compacting. Carrying a summary of irrelevant work forward wastes context and gives the model old assumptions to reason from.