AI Agent State Machine Design: Why Complex Workflows Cannot Rely on Prompts Alone

A reporting agent failed just before sending the email in step 5. Operations reran the task. The agent started again from step 1, generated a new report, and overwrote the previously approved version. The approval state was lost. The approver’s signed record was replaced by the new result, and no log could prove that the first report had been approved.

This was not a database rollback problem, and it was not a message-queue retry problem. The prompt only contained the sentence “continue processing”, so the model inferred the whole flow again. It did not know that steps 1-4 had already produced side effects: an approval API call, a generated report, and a temporary file write. The failure point was step 5, but side effects had started at step 2.

The real problem was not whether the model was smart enough. The task progress was hidden in natural language inside the prompt, with no recoverable state snapshot. The messages carried by a prompt are model context, not execution facts.

Fixing this kind of incident is not a matter of adding one more prompt sentence such as “check progress before continuing”. The sturdier move is to put the current node, completed side effects, next action, and failure compensation into a recoverable state table.

Incident key points

The reporting agent execution flow:

Step	Operation	Side effect	Idempotency
Step 1	Data query	Calls the database and queries user data	Idempotent (read operation)
Step 2	Report generation	Calls the report-generation tool and creates a PDF	Not idempotent (overwrites a file)
Step 3	Approval wait	Sends an approval request and waits for human approval	Idempotent (the API supports it)
Step 4	Approval accepted	Receives an approve event	Idempotent (status query)
Step 5	Send email	Calls the email API and sends the report	Failed (timeout)

Failure cause: the email send in step 5 timed out because of external API rate limiting, and the task was marked FAILED.

Rerun logic: read the “current progress” from the prompt. The prompt only said “approved, continue processing”. Actual execution: start from step 1 again -> regenerate the report in step 2 (overwriting the approved version) -> request approval again in step 3 -> send successfully in step 5.

Business impact: the approved report was replaced, approval records no longer matched the delivered report, the user complained that the report they approved was not the report they received, and the approval flow was wasted because two versions were approved but only one was actually sent.

Anti-pattern checklist

Check whether your agent hits any of these anti-patterns:

Anti-pattern	What it looks like	Hidden risk	Fix
Progress stored in the prompt	A natural-language summary such as “currently at step 3”	Lost after restart, not recoverable	Record the current node in a State field
Trace treated as State	A complete trace is mistaken for task state	Trace does not decide the next step	State records what should happen next
Retry without idempotency checks	On failure, rerun from the beginning	Side effects run twice	Idempotency key + already-executed check
Resume after approval without validation	Continue directly after approval	Does not return to the correct execution point	checkpoint + thread_id

1. State Machine Basics: State, Event, Transition, Guard, Action

A state machine is not required for every agent. A simple customer-support Q&A can work with a messages array. But a complex task with multiple steps, approval, external system calls, and failure recovery must make task progress explicit.

1.1 Core terminology table

The basic terminology comes from the Stately documentation:

Term	Definition	Agent example	Source
State	The mode the machine is in, with one clear semantic intent	INIT, PLAN_READY, TOOL_RUNNING, APPROVAL_PENDING, FAILED, COMPLETED	Stately state machines
Event	An external signal that triggers a state change	timeout, approve, reject, retry, resume, task_received	Stately state machines
Transition	An allowed path between states, expressed as a deterministic mapping	INIT -> PLAN_READY (event: task_received)	Stately state machines
Guard/Condition	A precondition for entering a state	Only enter TOOL_RUNNING when the budget is sufficient	Stately state machines
Action	An operation executed during a transition	Call a tool when entering TOOL_RUNNING	Stately state machines
Checkpoint	A state snapshot used for recovery	A LangGraph checkpointer persists graph state	LangGraph Persistence

Determinism principle: the same State + Event combination should point to exactly one next state, avoiding ambiguity. Finite state set: a state machine is not an infinite flowchart. It is a finite set of reachable states plus explicit transition rules.

1.2 Trace vs State vs Audit comparison

Trace, Audit Log, and State Snapshot solve three different problems:

Concept	Problem it solves	Is it business state?	Does it decide the next step?	Agent example
Trace	Observability and diagnostic skeleton	No	No	OpenAI Agents SDK trace (workflow_name, trace_id)
Audit Log	Compliance record and audit trail	No	No	Permission-model audit fields (actor, traceId, action, result)
State Snapshot	Current state that decides the next step	Yes	Yes	LangGraph checkpoint (current node, completed steps, what should happen next)

The distinction matters: a trace helps you observe what happened, but it is not business state. An audit log records compliance history for accountability. A state snapshot decides what should happen next, and that is the core of recovery. They cannot replace each other: having a trace does not mean you have state, and having an audit log does not mean the task is recoverable.

2. How LangGraph Handles State Persistence

A checkpoint is not a natural-language summary in the prompt. It is a recoverable, inspectable, replayable state snapshot. The LangGraph persistence documentation defines a checkpoint as a graph state snapshot that includes the full state and the next nodes to execute.

2.1 Checkpointer and Thread State

Core mechanisms (from the LangGraph Persistence documentation):

• Checkpointer: saves thread-scoped state snapshots (graph state snapshots)
• Store: saves cross-thread long-term data (application-defined store)
• Thread_id: the unique entry point for recovering a specific thread state
• Four uses: conversation continuity, human-in-the-loop, time travel, fault tolerance

LangGraph persistence puts short-term thread-scoped state in checkpointers and cross-thread long-term data in stores. A checkpoint includes the state snapshot and the application-defined store. Thread_id is the recovery entry point; the same thread_id can continue from the pause point.

A LangGraph checkpoint contains graph state, the list of next nodes to execute, checkpoint_id, timestamp, and version. Sensitive data should not blindly enter a checkpoint: some graph state fields may contain sensitive information and need explicit configuration to avoid persistence.

2.2 Interrupts and recovery

Core mechanisms (from the LangGraph Interrupts documentation):

• interrupt(): dynamically pauses execution inside a graph node, saves graph state, and waits for external input
• Recovery method: use the same thread_id and Command(resume=…)
• Common patterns: approval, review/edit, tool call review, human input validation
• Idempotent side-effect warning: side effects before interrupt must be idempotent because, on resume, the node reruns from the beginning of the node that called interrupt

An approval pause must be a pause state in the state machine, not a hope that the model will “remember to wait for approval”. Recovery needs the same thread cursor.

Recovery uses the same thread_id and Command(resume=…). Idempotent side effects are a precondition for safe recovery. If there is a side effect before approval, such as a call to an external API, it must be idempotent; otherwise the resumed node may call the API again.

3. Engineering Analogy: Temporal Durable Execution

Reliable long-running tasks are not a new problem. Temporal durable execution provides a mature engineering analogy.

3.1 Durable Execution definition

Core concepts (from the Temporal Durable Execution documentation):

• Durable Execution definition: workflow execution preserves state/progress through failures, crashes, or service interruptions
• Event History: records each step’s state so execution can recover from the last recorded event after a failure
• Three properties: Resumable, Recoverable, Reactive

Reliability for long-running tasks comes from event history and recoverable execution, not from a single process’s memory or the prompt context. An agent state machine needs a similar mechanism: checkpoint/event log + business state, not model inference alone.

Temporal’s Event History and LangGraph’s checkpoint are conceptually similar: both record execution history and support recovery from the failure point. The difference is that Temporal is a full workflow engine, while LangGraph is an agent state-management framework. Agent developers can borrow the main lesson from Temporal: durable execution needs structured state history, not process memory or model context.

4. State Table Template: A Reusable Agent State Table

State-machine concepts are abstract. To make them useful, you need a concrete state model. Here are three templates: a state table, an event table, and an incident-driven state table.

4.1 State table template (executable step block)

Template structure:

State	Event	Guard	Required action	Next
INIT	task_received	None	Initialize context and record start time	PLAN_READY
PLAN_READY	plan_generated	plan_valid	Generate an execution plan and record the tool sequence	TOOL_RUNNING
TOOL_RUNNING	tool_completed	budget_sufficient	Call the tool, record the result, and update the budget	APPROVAL_PENDING or COMPLETED
APPROVAL_PENDING	approve	approval_required	Send the approval request and record the approver	COMPLETED
APPROVAL_PENDING	reject	None	Record the rejection reason and notify the user	FAILED
FAILED	retry	retry_count < max	Check idempotency and roll back to the previous checkpoint	TOOL_RUNNING or APPROVAL_PENDING
COMPLETED	None	None	Record the completion time and clean up resources	Terminal

Template notes: the State column defines all reachable states (INIT, PLAN_READY, TOOL_RUNNING, APPROVAL_PENDING, FAILED, COMPLETED). The Event column defines events that trigger transitions (task_received, approve, reject, retry). The Guard column defines preconditions for entering a state (budget_sufficient, retry_count < max). The Action column defines the required operation during the transition (call a tool, record a result, send approval). The Next column defines the next state as a deterministic transition.

4.2 Event table template (state table supplement)

Template structure:

Event name	Trigger condition	Required prior state	Post state	Produces side effects?
task_received	The user submits a task	INIT	PLAN_READY	No
plan_generated	The LLM generates an execution plan	PLAN_READY	TOOL_RUNNING	No
tool_completed	Tool execution completes	TOOL_RUNNING	APPROVAL_PENDING or COMPLETED	Yes (calls an external API)
approve	The approver accepts	APPROVAL_PENDING	COMPLETED	Yes (sends email, deducts budget)
reject	The approver rejects	APPROVAL_PENDING	FAILED	No
retry	A retry request follows a failure	FAILED	TOOL_RUNNING or APPROVAL_PENDING	Requires idempotency check
timeout	Execution times out	TOOL_RUNNING	FAILED	No

Event table notes: the prior-state requirement makes it explicit which states may accept each event. The side-effect column marks which events need idempotency or compensation.

4.3 Incident-driven state table example (derived from the report overwrite incident)

Complete example: reporting agent state table derived from the opening incident

State	Event	Guard	Action	Next	Idempotency/compensation check
INIT	task_received	None	Initialize thread_id and record start time	QUERY_RUNNING	Not needed
QUERY_RUNNING	query_completed	None	Query data and save the result to state	REPORT_GENERATING	Not needed
REPORT_GENERATING	report_generated	None	Generate the report and save the report ID to state	APPROVAL_PENDING	Idempotency check: if the report already exists, skip generation
APPROVAL_PENDING	approve	None	Record the approver and approval time	EMAIL_SENDING	Not needed
APPROVAL_PENDING	reject	None	Record the rejection reason	FAILED	Not needed
EMAIL_SENDING	email_sent	None	Send the email and record the email ID	COMPLETED	Idempotency check: if the email was already sent, skip
EMAIL_SENDING	timeout	retry_count < 3	Record the failure and check idempotency	EMAIL_SENDING (retry) or FAILED	Idempotency key: email_id + thread_id
FAILED	retry	retry_count < max	Check idempotency and recover from the previous checkpoint	QUERY_RUNNING or REPORT_GENERATING or EMAIL_SENDING	Decide the recovery point from the checkpoint
COMPLETED	None	None	Record completion time and clean up resources	Terminal	Not needed

Incident fix: when step 5 fails (EMAIL_SENDING -> timeout), recovery should resume from EMAIL_SENDING, not QUERY_RUNNING. The checkpoint must record the current node (EMAIL_SENDING), completed steps (QUERY, REPORT_GENERATED, APPROVAL_APPROVED), and what should happen next (EMAIL_SENDING). Report generation and email sending need idempotency keys to avoid duplicate side effects.

5. Idempotency and Compensation: Recovery Is More Than Checkpoints

Having a checkpoint does not mean every side effect can be recovered safely. Recovery also needs idempotency, transactions, compensation, and checks against the external system’s current state.

5.1 Idempotency and compensation concepts

Definitions:

• Idempotent: multiple executions produce the same result and do not create duplicated side effects
• Compensation: undo an already-created side effect and restore consistency
• Transaction rollback: an atomic operation rolls back automatically on failure
• External-state check: inspect the external system before recovery to avoid duplicate operations

The three pillars of state consistency: idempotency identity (action_id + schema_hash), state snapshot chain (snapshot + prev_hash + delta), and registered compensation action (undo_op).

5.2 Idempotency and compensation checklist

Use this checklist to decide which operations need idempotency and which need compensation:

Operation type	Needs idempotency?	Needs compensation?	Idempotency key design	Compensation plan
Data query (no side effects)	No	No	-	-
Report generation (overwrites file)	Yes	Yes	report_id + thread_id	Delete the new report and restore the approved version
Email send (external API)	Yes	Hard	email_id + thread_id	Send a correction or cancellation email in some scenarios
Inventory deduction (database)	Yes	Yes	inventory_id + order_id	Add inventory back as compensation
Ticket creation (external system)	Yes	Yes	ticket_id + thread_id	Close the ticket as compensation
Budget deduction (internal state)	Yes	Yes	budget_id + thread_id	Add the budget back as compensation
Approval request send (no lasting side effect)	No	No	-	-

Decision logic: whether an operation creates an external side effect determines whether it needs idempotency. Reversible operations need compensation. Cross-system calls should include an external-system identifier in the idempotency key. Atomic operations can rely on transaction rollback.

Recovery is more than a checkpoint. It also needs idempotency, transactions, compensation, and external-state checks. The claim that a checkpoint alone can safely recover all side effects is inaccurate.

6. Agent Task State Checklist: Recoverable vs Unrecoverable

Not every checkpoint can recover. A terminal state is the end state of a workflow execution: completed, failed, timed out, or cancelled. A terminal state cannot resume; it can only be rerun or compensated.

6.1 State classification table

State type	Recoverable?	Recovery condition	Recovery method	Example
Failed	Yes	retry_count < max	Recover from the previous checkpoint	Tool call timeout
Retry	Yes	Idempotency check passes	Re-execute from the failed node	Email send failed
Compensation	Partially	A compensation plan exists	Execute undo_op	Inventory deduction failed
Approval Pause	Yes	approve/reject event	Command(resume=…)	Waiting for approval
Terminal	No	None	No recovery path	COMPLETED, FAILED (retry_count = max)

State checklist notes: a Failed state can recover through retry if retry_count < max. A Retry state requires an idempotency check and re-executes from the failed node. A Compensation state is partially recoverable if a compensation plan exists. An Approval Pause state recovers through an approve/reject event. A Terminal State is not recoverable, such as COMPLETED or FAILED after the maximum retry count.

7. Further Reading

State-machine design is only the starting point. State modeling has to match the business scenario, and different tasks need different state granularity and recovery strategies.

Series navigation

Article	Relationship	Link
Human-in-the-loop Agent Design: Which Steps Need Human Approval	Approval pause details	/blog/en/posts/ai/20260707-human-in-the-loop-agent-approval-design/
Agent Cost Control: Model Routing, Tool Budgets, and Failure Retries	Budget and retry strategy	/blog/en/posts/ai/20260707-agent-cost-control-model-routing-tool-budget-cache-retry/
LangGraph State Management in Practice: 2026 Agent Architecture Best Practices	LangGraph state management	/blog/en/posts/ai/20260424-langgraph-agent-architecture/
AI Agent Monitoring, Alerting, and Failure Recovery: From Logs to State Machines	Monitoring and recovery	/blog/en/posts/ai/20260527-ai-agent-monitoring-recovery/
LangGraph vs AutoGen State Tracking	Framework comparison	/blog/en/posts/ai/20260526-langgraph-autogen-state-tracking/
Agent Evaluation Datasets and Regression Tests: How to Avoid Breaking the Whole System with One Change	Evaluation and regression testing	Preview, next article in the series

External references

High-confidence sources:

Source	Confidence	Topic	Link
LangGraph Persistence documentation	high	Checkpointer, Store, Thread State, Checkpoint	https://docs.langchain.com/oss/python/langgraph/persistence
LangGraph Interrupts documentation	high	interrupt(), Command(resume=…), thread_id	https://docs.langchain.com/oss/python/langgraph/interrupts
Temporal Durable Execution documentation	high	Event History, Durable Execution, Resumable/Recoverable	https://docs.temporal.io/temporal
OpenAI Agents SDK Tracing documentation	high	Trace, Span, workflow_name, trace_id	https://openai.github.io/openai-agents-python/tracing/
AWS Step Functions State Machines documentation	high	State Machine, Flow State, Task State, StartAt, Next	https://docs.aws.amazon.com/step-functions/latest/dg/concepts-statemachines.html
Stately: State machines and statecharts	medium	State, Event, Transition, Guard, Action, Hierarchy	https://stately.ai/docs/state-machines-and-statecharts

A state machine is not required for every agent, but complex tasks must make progress explicit. The next step is not to add more frameworks. It is to design the right State, Event, Transition, Guard, and Action for your business scenario, and move task progress out of natural-language prompts into structured state.