Context compaction in autonomous AI sessions has triggered a serious reproducibility problem in automated programming workflows. Open-source developer Simon Willison documented this critical flaw during an extensive hands-on trial with OpenAI's GPT-6 Astra model inside ChatGPT Work. While the system successfully created interactive maps and exportable GPX files over a 27-minute autonomous run, the underlying Python scripts disappeared once the platform compressed the context window. The case highlights an emerging vulnerability in agentic engineering, showing that summarization routines designed to conserve tokens can destroy the auditability of complex tasks.
In his experiment, Willison tasked GPT-6 Astra (Max) with generating custom running routes based on geospatial data retrieved through the OpenStreetMap Overpass API. Over the course of nearly half an hour, the model operated semi-autonomously, querying map data, debugging coordinate mismatches, and calling a dedicated visualize skill to render dynamic D3 maps and downloadable GPX files. Throughout this workflow, Astra dynamically authored and executed Python code inside its sandboxed environment to filter waypoints and compute elevation profiles. For developers observing agent benchmarks, the test initially stood out as an impressive demonstration of sustained multi-step problem solving.
The breakdown occurred when ChatGPT Work initiated its automatic context compaction routine to prevent the session from exceeding memory limits. As the chat history swelled with iterative execution logs and API payloads, the system condensed earlier turns into a generalized text summary. While the generated GPX files and final visual maps survived the compaction phase, the underlying Python code blocks that generated them were permanently deleted. Willison discovered that neither the user interface nor the agent's internal memory retained the raw code, leaving no audit trail to verify how the model had calculated the final routes.
This phenomenon, increasingly described across the developer community as compaction loss, reveals a structural tension between context management and technical reliability. Autonomous coding agents like GPT-6 Astra and Anthropic's Claude Code often run for hours, consuming millions of tokens across hundreds of micro-actions. Compaction algorithms are built to discard verbosity and retain semantic milestones, treating code execution artifacts like disposable chat turns. However, software engineering requires strict determinism, meaning that losing intermediate execution scripts makes auditing edge cases or replicating bugs virtually impossible.
The debate triggered by the experiment has prompted calls for new infrastructure standards in agentic environments. Willison and other practitioners argue that frontier labs must provide dedicated persistence layers for pre-compaction logs instead of treating agent interactions like casual conversations. Tool executions, system invocations, and raw scripts should be archived to external storage or indexed via reproducible artifacts before memory trimming occurs. Without persistent execution logs, enterprise deployment of autonomous agents will remain risky for regulated industries that mandate complete auditability.
The issue comes at a time when the tooling ecosystem around autonomous agents is maturing rapidly to handle agent byproduct data. Independent developers have begun rolling out specialized utilities, such as commit-rewriter to purge intermediate scratchpads from Git histories, and shot-scraper updates designed to capture visual evidence from background workflows. As models take on deeper autonomous responsibilities, the boundary between conversation and software deployment is blurring. Addressing compaction data loss represents a critical test for whether frontier labs can transform creative chat platforms into durable engineering runtimes.