If you’ve ever stared at a usage dashboard and wondered why your token counts look astronomical, a recent self-audit of Claude Code usage offers a genuinely useful reality check — and it connects to two broader engineering problems every team building on LLMs eventually hits: context compaction and file-upload limits.

The 97% cache-read surprise

A developer decided not to trust Claude Code’s UI-reported usage numbers and instead parsed the raw session JSONL files directly — the actual first-party data source — to see what a week of real usage looked like. The topline number was eye-popping: roughly 3.35 billion tokens processed in one week from a single developer’s environment.

But the breakdown tells the real story. Of that 3.35 billion:

  • 96.9% was cache_read (re-reading previously cached context)
  • 2.8% was cache_creation (writing new cache entries)
  • 0.33% was actual model output
  • 0.025% was raw new input

In other words, only a tiny sliver of that eye-watering total represented genuinely new work. The overwhelming majority was Claude re-reading context it had already cached from earlier turns in the same session. This matters for a few reasons. First, it reframes how you should think about “token usage” as a cost or performance metric — a billion-token week sounds alarming until you realize 97% of it is essentially free re-reads rather than fresh computation. Second, it’s a strong practical argument for verifying vendor dashboards against raw logs when the numbers matter for billing, capacity planning, or performance debugging. UI aggregates can obscure exactly this kind of structural detail. Third, it implicitly validates prompt caching as a mechanism — if caching weren’t working, that 97% would instead be full-price input tokens on every single turn, which would make agentic coding tools like Claude Code prohibitively expensive and slow.

The self-audit is a nice reminder that with agentic tools that repeatedly reload large tool definitions, file contents, and conversation history, cache hit rate is arguably a more important metric to watch than raw token count.

The other side of the same coin: compaction for agents that never stop

Caching handles the case where you’re re-sending the same context repeatedly within a bounded session. But what happens when an AI agent is meant to persist for months, sitting in a chat workspace and expected to “remember” a relationship rather than a single conversation? That’s the subject of a separate piece on context compaction design for persistent AI agents.

The core problem is structural: context windows have a hard physical ceiling, and a long-lived agent that naively stuffs the entire conversation history into every prompt will eventually blow past that limit — or, even before hitting the wall, start responding more slowly and expensively as the prompt bloats. Unlike a stateless chatbot that resolves each exchange independently, a resident agent has to make deliberate architectural choices about what to keep, what to summarize, and what to evict entirely.

This is essentially a systems-design problem dressed up as an NLP problem. Teams building these agents need explicit policies for summarization thresholds, tiered memory (hot context vs. archived/retrievable history), and mechanisms to decide what’s safe to compress without losing information the user will later expect the agent to “remember.” The Claude Code cache-read stats and this compaction discussion are really describing two ends of the same spectrum: short-term efficient reuse of context (caching) versus long-term lossy-but-necessary reduction of context (compaction). Any team building agentic products at scale eventually needs both.

Upload limits and the silent switch to RAG

The third piece rounds out the picture by tackling a common point of confusion: file upload limits versus context window limits are separate constraints, and conflating them causes subtle bugs in how people reason about AI behavior. Concretely, the per-file size caps differ sharply across vendors — ChatGPT allows up to 512MB per file, Claude caps at 30MB, and Gemini sits at 100MB. But clearing that per-file check doesn’t mean you’re safe: the context window is an entirely separate, aggregate limit across the whole conversation.

The more important — and less obvious — detail is what happens as you approach that aggregate limit. According to Anthropic’s own documentation, Claude’s Projects feature automatically shifts behavior once uploaded materials approach the context window’s capacity: instead of stuffing the full text of your documents directly into context, it silently switches to a RAG (retrieval-augmented generation) mode, where only relevant fragments are retrieved and passed to the model.

This silent mode-switch has real consequences for anyone relying on “the model has read my entire document.” Once RAG kicks in, answers are no longer grounded in full-document reasoning — they’re grounded in whatever fragments a retrieval step decided were relevant. That’s a qualitatively different failure mode than simply running out of space: the system keeps working and keeps giving confident answers, but the underlying reasoning basis quietly changes from “comprehensive” to “best-guess retrieval.” For anyone building workflows around large reference documents in Claude Projects, ChatGPT, or Gemini, it’s worth explicitly testing whether you’re still in full-context mode or have tipped into retrieval mode — because the UI won’t necessarily tell you.

The throughline

All three of these posts are really about the same underlying tension: context windows are finite and expensive, and every LLM product — from coding agents to persistent chat companions to document-upload features — has to make an implicit or explicit tradeoff between completeness and efficiency. Prompt caching, context compaction, and automatic RAG fallback are three different engineering answers to the identical constraint. If you’re building or evaluating any LLM-powered product, understanding which of these mechanisms is active — and when it silently switches — matters more than the headline context-window number on a spec sheet.