Speaker Details Context Engineering Failures and Fixes at AGNTCon

A speaker delivered a talk titled 'Context Fails & Fixes' at AGNTCon, describing it as a 'State of Context Engineering'. The speaker is currently writing a book called 'The Context Engineering Handbook' with O'Reilly. The presentation addressed the 'RAG is dead' moment that occurred at the end of 2024. Model builders had initially hoped that large enough context windows would eliminate the need for selective context building or agentic systems.

Gemini 1.5 Pro was the first model to launch with a million-token context window. Its marketing material teased a 10-million-token context window model that was in the lab but not yet released. Llama 4 Scout was released with a 10-million-token context window before Gemini's teased model. Llama 4 Scout failed to catch on due to inference challenges and never received another version.

The talk covered context poisoning, where irrelevant or incorrect information prevents the model from proceeding. In a Gemini 2.5 benchmark playing Pokémon Red, the model became poisoned by knowledge of FireRed's requirement for tea instead of lemonade. Context distraction occurs when a model focuses on the growing context rather than its base knowledge. Gemini 2.5 began repeating actions from its history instead of synthesizing new plans once the context grew beyond 100,000 tokens. An enterprise team's agents performed terribly because their software was loading 600 skills for every engineer at startup. The speaker's demonstration harness required about 16 lines of changes to implement four fixes and one loop.