---
title: AI context platforms and the engineering problem underneath them
description: >-
  Context is a finite resource with diminishing returns. What an AI context
  platform has to solve, and the four techniques doing that work today.
author: elroot_labs
date: '2026-11-13'
categories:
  - artificial-intelligence
---
An AI context platform is a label for the layer that decides what goes into an agent's context window, what stays, and what gets removed when the window fills. The label is new and arrived with the usual amount of vendor noise. The problem underneath it is real, measurable, and the main reason long-running agents degrade in ways that look like the model getting worse.

If you are evaluating one of these products, or deciding whether you need one, the useful thing is to understand what the layer has to do. The techniques are documented and the hard part is identifiable.

## Context is a budget, not a container

The intuitive model of a context window is a container that holds a certain number of tokens, where everything inside is equally available. That model is wrong in a way that matters.

Anthropic's engineering write-up on [context engineering](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents) defines the discipline as "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference", and the reason it needs strategies is that models have an "attention budget". Context has to be treated as "a finite resource with diminishing marginal returns". As a window fills, the model's "ability to accurately recall information from that context decreases", a degradation the write-up calls context rot.

The practical consequence for an agent is specific. An agent that runs for fifty steps accumulates tool output, and by step forty the instruction it was given at step one is competing for attention with thirty-nine observations, most of which no longer matter. Nothing failed, no error was raised, and the agent quietly starts ignoring its own brief.

## The four techniques doing the work

The same write-up names the approaches currently in use, and any product in this category is some combination of them:

- Compaction, which summarises a nearly full window and starts a fresh one from that summary.
- Structured note-taking, where the agent writes notes to storage outside the window and reloads them when needed.
- Sub-agent architectures, delegating focused work to agents with clean windows that return condensed summaries.
- Just-in-time retrieval, holding lightweight references and loading the full data at runtime rather than up front.

Each trades something. Compaction is simple and loses detail. Note-taking keeps detail but depends on the agent writing good notes. Sub-agents isolate context well and add coordination cost. Just-in-time retrieval keeps the window small and moves the difficulty into deciding what to fetch.

## The unsolved part is getting the detail back

Vendor material tends to stop at the list above. The open problem sits one level down, and it is worth knowing before you buy something that claims to have solved it.

Compaction throws information away, and the question is whether what was discarded can be recovered when it turns out to matter. The ARC paper, [Addressable Recall Compaction for Long Context-Window Control in AI Agents](https://arxiv.org/abs/2607.25066), states the problem directly: existing methods that discard, summarise or retrieve earlier information "may remove task-critical details or fail to recover them reliably". That covers the whole family, including the retrieval-based approaches often presented as the fix.

ARC's own proposal is to keep tool observations in an append-only, ID-addressable log, swap older observations for compact citations when the window must shrink, and let the agent request stored content by its identifier, "without re-executing the corresponding tools or depending solely on similarity-based retrieval". The design "separates archival storage from active-context presentation". Its reported figures are 99.40% exact-answer accuracy on Needle-in-a-Haystack against a best baseline of 88.12%, and 29.97% on LongBench-v2 Hard against 28.25%, which the authors qualify as holding "under the tested settings" ([ARC, arXiv:2607.25066](https://arxiv.org/abs/2607.25066)).

Note the gap between those two numbers. Finding a planted fact in a long context is close to solved. Hard multi-step reasoning over long context has barely moved. A platform that demonstrates the first and implies the second is overselling.

## Questions worth asking a vendor

If you are evaluating a product in this category, four questions separate the serious ones:

1. When context is compacted, what happens to the discarded material? If the answer is that it is summarised and the original is gone, accept that some detail is unrecoverable and decide whether your task tolerates it.
1. Can the agent ask for a specific earlier observation by identifier, or only search for something resembling it? Similarity search fails exactly when the thing you need is phrased differently from the thing you remember.
1. What is measured when the vendor claims improvement, retrieval of planted facts or completion of hard multi-step tasks? These are very different claims.
1. What does the system log about what was in the window at each step? Without that you cannot debug a run, whatever else the platform does.

That last one connects to the oversight problem. A guardian layer watching for goal drift needs to see what the agent could actually see at the moment it drifted, which is a context-management question before it is a monitoring question. We have written about that side in [guardian agents and runtime oversight](/blog/guardian-agents-runtime-oversight).

## Before you need a platform

Most teams running a single agent over short tasks do not need to buy anything here. Three cheaper steps come first.

Measure what you actually use. Record tokens in the window at each step across real runs, and look at the distribution rather than the mean. Many agents that feel like they need context management are simply loading a large document on every call for no reason.

Log the window itself, or a hash and a manifest of what was in it. This is the single most useful artefact when a run goes wrong, and it is nearly free to capture while you are already logging tool calls.

Then test whether the failures you see are context failures at all. An agent that forgets its instruction at step forty and one that never understood it look identical from outside, and only the first is fixed by anything described here. An [evaluation set built from your own failure cases](/blog/agent-evaluation-test-set) is how you tell them apart.

If you are deciding whether a context layer is worth building for an agent already in production, [tell us about the project](/contact).
