As enterprises rush to deploy AI agents capable of autonomous task execution, a fundamental engineering problem is coming into sharper focus: these systems systematically lose track of earlier instructions, user preferences, and critical context as conversations and workflows grow longer.
Writing for O'Reilly Radar as part of an ongoing series on agentic engineering and AI-driven development, Addy Osmani highlights context degradation — the progressive erosion of an AI agent's working memory — as a structural limitation baked into the architecture of large language models themselves. Unlike human engineers who accumulate understanding over a project's lifetime, AI agents operating within fixed-size context windows face an unavoidable trade-off: as new information enters, older details are compressed or discarded entirely.
Why This Matters Now
The timing is significant. Across the technology industry, organisations are investing heavily in agentic AI — autonomous systems that can plan, reason, and execute multi-step tasks without constant human oversight. Context degradation threatens to undermine this ambition at a foundational level. An agent that forgets half of what it was told mid-project is not merely inconvenient; it is unreliable for the kind of sustained, complex work that enterprises expect from their AI investments.
The problem is not hypothetical. As Osmani describes, developers building agentic systems report that agents routinely lose track of earlier decisions, contradict previously stated requirements, and fail to maintain coherent state across long interactions. For workflows spanning dozens or hundreds of steps, the accumulated data loss can render an agent's output unusable or, worse, subtly incorrect.
Mitigation Strategies Take Shape
Engineering teams are not standing still. Osmani outlines several practical mitigation strategies emerging to address the problem:
Retrieval-Augmented Generation (RAG): Rather than relying solely on what fits within the context window, agents can query external knowledge bases in real time, pulling relevant information on demand. This shifts the burden from memorisation to retrieval, though it introduces its own complexities around query quality and latency.
External memory stores: Persistent databases, vector stores, and structured logs allow agents to offload information that no longer fits in context. When designed well, these systems act as a long-term memory layer that agents can consult as needed.
Hierarchical summarisation: As conversations grow, older segments can be compressed into progressively shorter summaries, preserving the gist while freeing up context space. The challenge lies in deciding what to keep and what to summarise away — a non-trivial problem when critical details may lurk in seemingly minor exchanges.
Tool-use patterns: Some architectures give agents the ability to write and read from structured scratchpads or notebooks, effectively creating an externalised working memory that persists across turns.
The Broader Engineering Picture
What makes context degradation particularly thorny, Osmani argues, is that it is not a bug to be patched — it is a consequence of how transformer-based models process information. Current context windows, even those stretching to hundreds of thousands of tokens, are ultimately finite. Larger windows help, but they merely push the boundary outward rather than eliminating it.
That said, not everyone views the situation as purely pessimistic. Rapid advances in context window capacity — from thousands to hundreds of thousands of tokens in just a few years — suggest the ceiling is rising fast enough that some use cases will simply outgrow the problem. Osmani's counterpoint is that ambition tends to scale alongside capability: as windows expand, agents are tasked with longer and more complex workflows, ensuring the memory constraint reasserts itself.
For IT professionals and developers working with agentic systems, the practical takeaway is clear: designing for memory loss is not optional. Robust agentic architectures must assume that context will degrade and build in safeguards — whether through external memory, structured state management, or careful workflow design that limits the amount of critical context an agent must retain at any given moment.
Osmani's analysis aligns with a growing body of practitioner experience — from engineering blog posts to conference discussions — confirming that context management is now a core design discipline for anyone building on LLMs. The gap between what AI agents promise and what they can reliably deliver, he contends, remains substantial. Bridging it will require not just better models, but better engineering practices around the limitations those models carry with them.
當企業爭相部署能自主執行任務的 AI 代理時，一個根本性的工程問題正愈發凸顯：隨著對話與工作流程變長，這些系統會系統性地失去對早期指令、用戶偏好及關鍵上下文的追蹤。
Addy Osmani 在為 O'Reilly Radar 撰寫的系列文章（專注於代理工程與 AI 驅動開發）中，強調了上下文退化——即 AI 代理工作記憶的逐步侵蝕——是內建於大型語言模型架構中的結構性限制。與人類工程師能在專案生命週期中累積理解不同，運作於固定大小上下文窗口內的 AI 代理面臨一個不可避免的權衡：當新資訊進入時，舊的細節會被壓縮或完全丟棄。
為何此刻至關重要
時機非常關鍵。整個科技產業中，各機構正大力投資代理式 AI——即能自主規劃、推理並執行多步驟任務、無需人類持續監督的系統。上下文退化威脅從基礎層面動搖這一雄心。一個在專案中途就忘記一半所被告知資訊的代理，不僅帶來不便；對於企業期望其 AI 投資所能承擔的持續性、複雜工作而言，它更顯得不可靠。
問題並非假設性的。正如 Osmani 所描述，開發代理式系統的工程師報告指出，代理經常會失去對先前決策的追蹤、與先前陳述的需求相矛盾，並且無法在長時間的互動中維持連貫狀態。對於跨越數十甚至數百步驟的工作流程，累積的數據流失可能使代理的輸出變得無法使用，或更糟的是，產生微妙的錯誤。
緩解策略逐漸成形
工程團隊並非無所作為。 Osmani 概述了數種正在湧現的實用緩解策略，以應對此問題：
檢索增強生成（RAG）： 代理不再僅依賴上下文窗口內的資訊，而是可以即時查詢外部知識庫，按需提取相關資訊。這將負擔從記憶轉移到檢索，儘管這也引入了其自身在查詢質素和延遲方面的複雜性。
外部記憶儲存： 持久化資料庫、向量儲存庫和結構化日誌，允許代理卸載那些不再適用於上下文的資訊。設計得當的話，這些系統可充當長期記憶層，供代理按需查閱。
層次化摘要： 隨著對話增長，較舊的部分可被壓縮成逐步精簡的摘要，保留要點同時釋放上下文空間。挑戰在於決定保留什麼、摘要掉什麼——當關鍵細節可能潛藏於看似微不足道的交流中時，這絕非易事。
工具使用模式： 某些架構賦予代理寫入和讀取結構化草稿本或筆記本的能力，有效地創建了一種跨對話回合持續存在的外部化工作記憶。
更廣泛的工程視角
Osmani 認為，上下文退化之所以特別棘手，在於它並非一個可以修補的缺陷——它是基於 Transformer 模型處理資訊方式的必然結果。當前的上下文窗口，即使擴展到數十萬 token，本質上仍是有限的。更大的窗口有幫助，但它們只是將邊界向外推移，而非消除它。
話雖如此，並非所有人都對此持純然悲觀的看法。上下文窗口容量的快速進步——短短數年內從數千 token 擴展到數十萬 token——表明其上限提升得足夠快，以至於某些應用場景將自然地超越這個問題。 Osmani 的反論點是，雄心往往與能力同步擴展：隨著窗口擴大，代理被賦予更長、更複雜的工作流程，這確保了記憶限制會重新凸顯。
對於從事代理式系統工作的 IT 專業人士和開發者而言，實際的啟示非常明確：為記憶流失而設計並非可選項。穩健的代理架構必須假設上下文將退化，並建立防護措施——無論是通過外部記憶、結構化狀態管理，還是仔細的工作流程設計，以限制代理在任何給定時刻必須保留的關鍵上下文數量。
Osmani 的分析與日益增長的實踐者經驗相符——從工程 blog 文章到研討會討論——證實上下文管理現已成為任何基於 LLM 進行建構者的核心設計學科。他主張，AI 代理所承諾的與它們能可靠交付的之間，差距仍然顯著。彌合這一差距，不僅需要更好的模型，更需要圍繞這些模型自身所攜帶的限制，發展出更好的工程實踐。