OpenAI Codex Agent Loop Structure: 5-Step Anatomy of the While Loop

By /

Table of Contents

When asking an AI coding agent to “refactor this function,” the internal process doesn’t simply generate code once and call it done. The OpenAI Codex agent loop structure runs a repeating cycle of code generation → tool call → result feedback → next turn input. This article dissects the internal mechanics based on Codex CLI’s agent-loop.ts source code.

One thing worth noting upfront: Codex CLI was originally written in TypeScript (ink-based), but has since been rewritten in Rust. The codebase is 96.2% Rust, and optimizing agent loop efficiency was the core motivation behind this transition. The structural patterns from agent-loop.ts discussed below follow the same design philosophy in the Rust version, though the TypeScript implementation is effectively legacy at this point.

Why AI Coding Agents Can’t Solve Tasks in a Single Call

Having an LLM generate code in one shot leaves no way to know whether the result compiles, passes tests, or conflicts with existing code. The limitations of the single prompt-response pattern are clear.

ApproachCode GenerationExecution VerificationError CorrectionContext Retention
Single API CallOXXX
Manual IterationOO (human)O (human)△
Agent LoopOO (auto)O (auto)O

This is analogous to designing a data pipeline where an ETL job retries on failure at any stage, logs the failure cause, then passes control to the next stage. The agent loop is fundamentally the same pattern — a model call is one “task,” tool execution is a “transformation,” and result feedback becomes the “next stage input” in the pipeline.

Core premise of the agent loop
The agent loop repeats until the LLM determines there are “no more tasks to process.” Since the LLM itself decides the termination condition, a max iteration limit becomes important — yet this isn’t explicitly specified even in the official documentation.
With this understanding of why the structure is necessary, it’s time to look at the actual implementation.

The Core of the OpenAI Codex Agent Loop Structure — While Loop Turn Iteration

Codex CLI’s agent loop operates on a while (turnInput.length > 0) structure. Each turn calls the OpenAI Responses API, executes any tool calls included in the response, then feeds the results as input for the next turn.

while (turnInput.length > 0) {
  // Send request to OpenAI Responses API
  const responseCall = this.oai.responses.create({
    model, instructions, input,
    stream: true,
    parallel_tool_calls: false,
    tools // shell function
  });
  // Handle response and tool calls
  turnInput = []; // Reset for next iteration
}

Three things stand out in this code.

stream: true and Real-Time Response Handling

The oai.responses.create call sets stream: true. Streaming mode delivers response tokens to the client as they’re generated, allowing the user to observe the agent’s “thinking process” in real time. This follows the same principle as monitoring intermediate results via streaming in a batch pipeline.

What parallel_tool_calls: false Means

The explicit disabling of parallel tool calls is notable. In the context of a code generation agent, this is a sensible choice — the result of modifying file A may be a precondition for modifying file B. In data pipeline terms, this is equivalent to executing tasks with sequential dependencies in a DAG serially.

turnInput Initialization and Loop Termination Condition

At the start of each iteration, turnInput = [] resets the array. If the response contains a tool call, the result is fed back as a function_call_output. If there are no tool calls, turnInput remains an empty array and the loop terminates naturally. This is a pattern where control flow is determined by data presence rather than an explicit break statement.

The role of handleFunctionCall
Tool calls (shell functions) in the response are executed by handleFunctionCall(). This function runs shell commands in a sandboxed environment and returns the stdout/stderr as a function_call_output, which becomes the input for the next turn — this is the core mechanism.

TypeScript to Rust — Codex CLI Architecture Transition

Codex CLI was rewritten from TypeScript (ink-based) to Rust. This wasn’t a simple language swap — it was a decision to fundamentally improve agent loop efficiency.

The core motivation for the Rust transition lies in performance optimization of the agent loop — the harness that repeatedly calls the model. Eliminating GC (garbage collector) overhead achieved reduced memory consumption and faster execution. Since the agent loop iterates through tens to hundreds of turns, GC pauses at every turn accumulate into significant latency.

CategoryTypeScript (ink-based)Rust
RuntimeNode.js (V8 GC)Native binary
Memory ManagementGC-dependentOwnership-based
Agent Loop OverheadCumulative GC pausesZero GC
Codebase ShareLegacy96.2%
Installationnpmnpm + Homebrew

The sandbox implementation also moved down to the OS level. macOS uses sandbox-exec with Seatbelt policies, while Linux uses Landlock and seccomp. This applies kernel-level sandboxing one layer below the process-level isolation of the previous TypeScript version.

Caution when referencing legacy TypeScript code
The agent-loop.ts analyzed in this article is from the TypeScript implementation. Since the current Codex CLI main codebase has transitioned to Rust, forking or extending the TypeScript code directly isn’t recommended. That said, the design patterns of the agent loop follow the same structure in the Rust version.

On the extensibility front, Codex CLI currently supports a newline-delimited JSON wire protocol and MCP (Model Context Protocol) servers. This is a design choice to provide a standardized interface when connecting the agent loop to external tool chains.

Context Management Strategy — Server-Side Storage vs. Client-Side Transmission

Context management in the OpenAI Codex agent loop structure operates in two modes. This branching is determined by a single disableResponseStorage flag.

Server-side storage mode (disableResponseStorage: false): Only the previous_response_id and new input delta are sent. Since the server retains previous conversation history, the client doesn’t need to send the full history with every request. This reduces network payload and client memory overhead.

Client-side transmission mode (disableResponseStorage: true): The full conversation history (transcript: Array<ResponseInputItem>) excluding system messages is included in every request. Since no state is stored on the server, this is advantageous for privacy — but payload grows linearly as turns accumulate.

A similar trade-off exists in data pipeline design when choosing a checkpoint strategy. Delegating state to external storage (server) makes each stage lighter but creates an external dependency; carrying state every time is self-contained but increases transmission cost.

[Server-side storage mode]
Turn 1: input(full) → response_id=A
Turn 2: previous_response_id=A + delta → response_id=B
Turn 3: previous_response_id=B + delta → response_id=C

[Client-side transmission mode]
Turn 1: input(full) → response
Turn 2: transcript(turn 1 full) + new_input → response
Turn 3: transcript(turn 1+2 full) + new_input → response
No official documentation on context compaction
Detailed official specs for the compaction endpoint (/responses/compact) — used when conversation history approaches the token limit — haven’t been confirmed. The official Codex guide on platform.openai.com is also inaccessible (403), leaving a gap in API-level official documentation.

Error Handling and Retry — Exponential Backoff Mechanism

Exponential backoff retry visualization

Error handling in the agent loop is the factor that determines pipeline stability. Codex CLI limits retries with MAX_RETRIES = 5 and applies different strategies by error type.

const RATE_LIMIT_RETRY_WAIT_MS = parseInt(
  process.env["OPENAI_RATE_LIMIT_RETRY_WAIT_MS"] || "2500", 10
);
const MAX_RETRIES = 5;

Transient errors (timeout, 5xx server errors, connection failures) are retried with exponential backoff. On a 429 Rate Limit response, the wait time follows whichever is longer: RATE_LIMIT_RETRY_WAIT_MS (default 2500ms) × 2^(attempt-1), or the wait time suggested by the server’s Retry-After header.

Handling Non-Retryable Errors

Non-transient errors like token limit exceeded (context length exceeded) won’t produce different results on retry. In these cases, the error is displayed to the user and the request is aborted. This mirrors the data pipeline pattern of routing structural errors like schema mismatches to a dead letter queue rather than retrying.

Adjusting Retry Parameters via Environment Variables

The OPENAI_RATE_LIMIT_RETRY_WAIT_MS environment variable allows external adjustment of the base wait time. This is a reasonable design since API quotas can vary by operational environment. For example, in environments using a team-level API key, Rate Limits may be hit more frequently — increasing the wait time to 5000ms or more reduces retry failure rates under Rate Limiting.

The wait time calculation for exponential backoff follows this progression:

Attempt 1: 2500ms × 2^0 = 2500ms
Attempt 2: 2500ms × 2^1 = 5000ms
Attempt 3: 2500ms × 2^2 = 10000ms
Attempt 4: 2500ms × 2^3 = 20000ms
Attempt 5: 2500ms × 2^4 = 40000ms (max retry)

If all 5 retries fail, the entire agent loop halts. Total wait time can reach approximately 77.5 seconds.

Controlling Max Iterations Outside the Agent Loop

Since Codex CLI has no explicit max iteration parameter internally, a wrapper pattern around the agent loop is practical. This applies equally whether implementing a custom client that calls the OpenAI Responses API directly or running Codex CLI as a subprocess.

const MAX_AGENT_TURNS = parseInt(
  process.env["MAX_AGENT_TURNS"] || "20", 10
);

async function runWithTurnLimit(
  client: AgentLoop,
  initialInput: string
): Promise<string> {
  let currentInput = initialInput;
  let turnCount = 0;

  while (turnCount < MAX_AGENT_TURNS) {
    const result = await client.step(currentInput);
    turnCount++;
    if (result.done) return result.output;
    currentInput = result.nextInput;
  }
  throw new Error(`Agent loop exceeded ${MAX_AGENT_TURNS} turns`);
}

Exposing MAX_AGENT_TURNS as an environment variable allows adjustment based on task complexity. Simple function refactoring can use 5–8 turns, while multi-file refactoring uses 20–30 turns — this also serves as cost management. In production, logging each turn’s input/output in JSON Lines format makes it possible to aggregate which turns have concentrated tool calls and use that data for prompt improvement.

Agent Loop Design Patterns Through the Lens of Data Pipelines

Viewing the OpenAI Codex agent loop structure through the lens of data pipelines reveals familiar patterns. Here’s a flow diagram mapping each component’s role:

flowchart LR
  A[User Input] --> B[turnInput]
  B --> C{Empty?}
  C -->|No| D[Responses API Call]
  D --> E{Tool call exists?}
  E -->|Yes| F[handleFunctionCall]
  F --> G[function_call_output]
  G --> B
  E -->|No| H[Loop Exit]
  C -->|Yes| H

Event-driven loop: Process when data exists in the turnInput array; terminate when it’s empty. This is structurally similar to a Kafka consumer that processes messages when they exist in a topic and continues polling when they don’t. The difference is that agent loops assume finite termination, while streaming pipelines assume infinite execution.

Linear dependency chain: The parallel_tool_calls: false setting corresponds to a chain structure where all tasks in a DAG have serial dependencies. It’s equivalent to declaring task_a >> task_b >> task_c in Airflow. Given the nature of code modification where each step’s result becomes the next step’s input, this design prioritizes correctness over parallelization.

Checkpoints and state management: The previous_response_id in server-side storage mode is conceptually identical to Spark Streaming checkpoints. It provides the foundation for storing intermediate state externally and resuming from that point on failure. Client-side transmission mode is closer to the pattern of Apache Beam’s stateless DoFn, which receives the full state as input every time.

Retry and backpressure: Exponential backoff retry produces the same effect as applying backpressure to a downstream system in a pipeline. A 429 Rate Limit is the signal “capacity exceeded, slow down” — waiting exponentially and retrying is the standard pattern for absorbing backpressure on the producer side.

Considerations when extending the agent loop into a pipeline
To integrate Codex CLI’s agent loop into a custom pipeline, leveraging the newline-delimited JSON wire protocol is the natural approach. Logging each turn’s input/output in JSON Lines format enables tracking tool call sequences and error points alongside turn numbers.

Current Limitations and Undocumented Areas

Analyzing the agent loop structure has revealed certain limitations and documentation gaps. Documenting these honestly is necessary for completeness of the technical analysis.

No max iteration limit in official documentation. The while (turnInput.length > 0) structure theoretically carries the possibility of an infinite loop. If the model continues generating tool calls every turn, the loop may never terminate. In practice, token limits or Rate Limits act as de facto upper bounds, but an explicit max iteration parameter hasn’t been confirmed in official documentation.

Context compaction spec is unconfirmed. In long-running agent sessions where conversation history approaches the token limit, compaction becomes necessary. The existence of a compaction endpoint (/responses/compact) is known, but detailed official specs are currently unavailable.

The official Codex guide on platform.openai.com is inaccessible (403). With API-level official documentation absent, the analysis in this article is based on the Codex CLI GitHub repository source code and community discussions.

Agent loop internals after the Rust transition. While the TypeScript implementation’s agent-loop.ts is publicly available for analysis, the Rust-rewritten current version hasn’t been analyzed at the same source level. Even if the design philosophy remains the same, implementation details may differ.

Here’s a summary of how these limitations manifest in actual integration projects and what realistic mitigations are currently available:

LimitationActual SymptomRealistic MitigationDependency
No max iteration definedLoop runs indefinitely on complex requestsControl via MAX_TURNS env variable in wrapperClient implementation
Context compaction undocumentedcontext length exceeded on long sessionsManual trimming after token count with tiktokenRequires custom implementation
Official docs inaccessibleLatest API spec unconfirmedTrack GitHub source + DiscussionsCommunity-dependent
Rust version internals opaquePossible divergence from TypeScript analysisObserve per-turn behavior via NDJSON logsWire protocol utilization

The absence of official documentation is the biggest long-term risk. Until the Codex-specific page on platform.openai.com is restored or OpenAI updates its official API reference, the CHANGELOG and Discussions tab in the GitHub openai/codex repository remain the most reliable information sources. With the Rust codebase at 96.2%, verifying actual behavior through NDJSON logs before directly applying TypeScript analysis results to the Rust version is the safe approach.

Beyond the Agent Loop — MCP Protocol and Tool Chain Extension

Codex CLI’s support for MCP (Model Context Protocol) servers signals the extensibility of the agent loop. Currently the agent loop’s tools are limited to shell functions, but MCP adds external database queries, API calls, and file system operations to the agent loop through a standardized interface.

The newline-delimited JSON wire protocol handles the communication layer for this extension. Since each message is delimited as a single line of JSON, it can be immediately parsed with pipeline tools (jq, grep, awk) or loaded directly into a logging system. From a data engineering perspective, this is the foundation for making every turn of the agent loop observable. Collecting turn numbers, tool call types, and execution results in NDJSON format enables querying performance bottlenecks in the agent loop.

There’s also an architectural pattern to consider when extending the agent loop through MCP servers. Given that tool calls execute serially only, independent lookup operations (e.g., document search + code symbol lookup) are more efficient when processed asynchronously inside the MCP server and returned as a single response. This effectively circumvents the parallel_tool_calls: false constraint through internal parallelism within the MCP server.

For deeper understanding of Codex CLI architecture, specific benchmarks from the Rust transition and the Codex sandbox security model make worthwhile next analysis targets. For broader interest in AI coding agent architecture, the OpenAI Responses API streaming protocol and Codex context compaction mechanism are also topics worth exploring as extensions of understanding the OpenAI Codex agent loop structure.

Read next

Scroll to Top