Agentic Workflows: How to Build AI Workflows That Ship

Agentic workflows are processes where an LLM decides which steps to run, in what order, and under what conditions, instead of a fixed pipeline where Step A always leads to Step B. Every few months a new framework launches promising "autonomous AI agents that handle your entire business process," and teams spend three months building systems that crash on the first real user. After shipping 6+ agentic workflow systems at HinterBuild, the pattern is consistent: frameworks give structure, not substance. Workflow design determines whether you ship.

Key Takeaways:

• Most production "agentic" workflows are 80% deterministic steps and 20% LLM decisions; keep the LLM at the decision points and out of the plumbing.
• Split the system into three layers: an Orchestrator (brain), self-contained Steps (muscle), and Infrastructure (skeleton) that persists state and logs every transition.
• Start with a linear pipeline; add branch-and-merge, parallel fan-out, and approval gates only when a concrete requirement forces you to.
• Persist workflow state outside process memory from day one so a restart or crash resumes the run instead of losing it.
• Give every step a typed result (completed / failed / retry / needs_approval) and a per-step retry policy; blanket retries hide root causes.
• Gate irreversible actions (refunds, deletes, outbound messages) behind human approval and measure approval latency as a first-class metric.

Table of Contents:

• What Are Agentic Workflows?
• Three-Layer Architecture
• Workflow Patterns That Work
• Where Agentic Workflows Break
• State Persistence and Resumability
• Choosing the Orchestrator Model
• Design & Implementation Checklists
• Two-Week Starter Plan
• Frequently Asked Questions

What Are Agentic Workflows?

Short answer: An agentic workflow is a process where an LLM decides which steps to run, in what order, and under what conditions, unlike fixed pipelines where Step A always leads to Step B.

Anthropic's Building Effective Agents guide draws the same line: workflows are systems where LLMs and tools are orchestrated through predefined code paths, while agents are systems where the LLM dynamically directs its own process. Most useful production systems sit in between, and that middle ground is what this guide covers.

Non-Agentic vs Agentic

Be honest about what you build. Many "agentic workflows" predetermine most paths, and that is fine as long as you do not over-engineer agent freedom.

Three-Layer Architecture for Agentic Workflows

Most agentic workflow failures happen when teams skip layers or blur boundaries.

Layer 1: The Orchestrator (The Brain)

The orchestrator (LLM or smaller model) decides: next step, retry, escalate, or complete.

Needs:

• Current workflow state (JSON state machine)
• Available steps with descriptions
• Constraints (what is allowed / forbidden)
• Completion criteria

Does NOT need: Implementation details of each step.

from typing import TypedDict, Literal

class WorkflowState(TypedDict):
    workflow_id: str
    current_step: str
    step_history: list[str]
    context: dict
    errors: list[dict]
    retry_count: int
    status: Literal["running", "completed", "failed", "awaiting_approval"]

async def orchestrator_decide(state: WorkflowState, available_steps: list[dict]) -> str:
    """LLM reads state and picks next step."""
    prompt = f"""
    Current state: {state}
    Available steps: {available_steps}
    Pick the next step. Return step name only.
    Constraints: Never skip validation. Require approval for refunds > $100.
    """
    return await llm.complete(prompt)

In practice you should constrain the orchestrator's output with tool calling or a JSON schema rather than free text, so a hallucinated step name is a validation error rather than a runtime exception. See structured output patterns for the mechanics.

Layer 2: The Steps (The Muscle)

Each step is self-contained: validated inputs, structured outputs, error handling, idempotency.

from pydantic import BaseModel
from typing import Literal

class StepResult(BaseModel):
    status: Literal["completed", "failed", "needs_approval", "retry"]
    output: dict | None = None
    error: str | None = None
    retry_after_seconds: int | None = None

async def lookup_order_step(context: dict) -> StepResult:
    """Step: lookup order with validation."""
    order_id = context.get("order_id")
    if not order_id or not order_id.startswith("ORD-"):
        return StepResult(status="failed", error="INVALID_ORDER_ID")

    try:
        order = await db.fetch_one("SELECT * FROM orders WHERE id = $1", order_id)
        if not order:
            return StepResult(status="failed", error="NOT_FOUND")
        return StepResult(status="completed", output={"order": dict(order)})
    except Exception as e:
        logger.exception(f"Order lookup failed: {e}")
        return StepResult(status="retry", error="DB_ERROR", retry_after_seconds=30)

Notice the split between failed (a deterministic outcome the orchestrator must route around) and retry (a transient outcome the infrastructure handles without consulting the LLM). Collapsing those two into one generic error is the single most common cause of runaway retry loops. Build reliable steps with our backend API engineering patterns.

Layer 3: The Infrastructure (The Skeleton)

Required for production:

• State persistence (survive restarts)
• Structured logging with workflow_id
• Human-in-the-loop approval gates
• Versioned step APIs
• Metrics and alerts

Skip infrastructure and you get time-travel bugs, lost state, and unobserved failures. Deploy on cloud infrastructure with proper staging.

Agentic Workflow Patterns That Work in Production

Pattern 1: Linear Pipeline

Step A → Step B → Step C → Complete

Best for: Order processing, document pipelines, known sequences.

Example: Validate order → check inventory → process payment → send confirmation

Caution: Design steps so new ones can be injected without rewriting the entire pipeline.

Pattern 2: Branch-and-Merge

Step A → (B OR C) → Step D → Complete

Best for: Ticket routing by urgency, conditional processing.

Caution: Track branch_taken explicitly in state. Merge points fail when branch outputs do not match downstream expectations.

Pattern 3: Human-in-the-Loop Approval Gates

Step → Approval? → (Yes: human acts / No: continue) → Complete

Best for: Refunds, financial transactions, sensitive data changes.

See how AI agents fail when approval gates are missing, and approval gate patterns for the UX side.

Pattern 4: Parallel Execution

Multiple steps run concurrently → results converge → next step.

Best for: Research from multiple sources, independent data gathering.

Caution: Design for partial failure. What if 2 of 3 parallel steps fail?

Pattern Comparison

For multi-agent variants of these patterns, see multi-agent orchestration patterns. For the reasoning loop inside a single step, the ReAct paper is still the reference point; we compare it with plan-and-execute in ReAct vs Plan-and-Execute.

Where Agentic Workflows Break

The failure modes below account for most of the incidents we have debugged. None of them are caused by the model being "not smart enough."

Unbounded Loops

The orchestrator picks retry or re-runs a lookup step because the state it sees never changes. Without a hard step budget the workflow runs until the API bill or a timeout stops it. Fix: cap total steps per workflow (30-50 is a sane default for support-style tasks), cap retries per step, and make the orchestrator's history visible in its prompt so it can see it already tried the step. We cover the full safeguard set in preventing agent loops.

Context Drift Across Steps

Each step appends output to context, and by step 12 the orchestrator is reasoning over 40k tokens of stale intermediate data. Decisions get worse and slower. Fix: steps return a compact, typed summary, not raw payloads. Store the full payload in object storage keyed by workflow_id and step index; put only the reference in state.

Non-Idempotent Side Effects

A step charges a card, the process dies before state is persisted, the workflow resumes and charges again. Fix: every step that writes externally takes an idempotency key derived from workflow_id + step_name + attempt, and the external call is made only after the "starting step" state transition is durable. The idempotency guide covers key design.

Silent Merge Failures

Branch B returns {"customer": {...}}, branch C returns {"customer_record": {...}}, and the merge step reads whichever key it was written against. Fix: a shared Pydantic schema for merge inputs, validated at the merge boundary, with a failed result rather than a KeyError in production.

Approval Gates That Never Resume

The workflow pauses for approval, the ticket is closed by a human in another system, and the workflow sits in awaiting_approval forever. Fix: approvals have TTLs, expiry transitions to an explicit expired state that alerts, and the resume path is tested in CI exactly like the happy path.

State Persistence and Resumability

The single infrastructure decision that separates demos from production is where workflow state lives. In-memory state means one pod restart loses every in-flight run.

For most teams a single workflow_runs table with state JSONB, status, updated_at, and an append-only workflow_events table is enough. Every state transition is one transaction: write the event, update the state, commit. The orchestrator reads state fresh at the start of each decision, so a second worker can pick up a run the first one abandoned.

If your workflow is graph-shaped, LangGraph's persistence layer gives you checkpoints, time-travel debugging, and resumable interrupts without writing the table yourself; we cover it in stateful agents with LangGraph checkpoints. If runs span hours or days with many external waits, durable execution engines like Temporal are built precisely for that problem and will save you from reinventing retries, timers, and signals.

Choosing the Orchestrator Model

The orchestrator makes small, frequent decisions over structured state. That is a different workload from generating a long answer, and it changes which model you want.

• Decision-only orchestration (pick the next step from 5-10 options): a small, fast model is usually enough, provided the output is constrained to a schema. Latency and cost per decision matter more than raw capability because the decision runs on every transition.
• Orchestration with synthesis (decide the next step and draft a customer-facing message): use a stronger model, but split the two calls so the decision stays cheap and the synthesis can be cached or streamed.
• Deterministic routing (the next step is a pure function of state): do not call a model at all. Encode it in code. This is the "80% deterministic" part.

Route the expensive model only where it changes outcomes; LLM routing by task walks through the measurement. Whichever model you pick, use native tool calling (see the Anthropic tool use docs) so step selection arrives as validated JSON, not prose you have to parse.

Design & Implementation Checklists

Phase 1: Design (Before Code)

• One-sentence problem statement
• Defined completion criteria (observable, not "feels done")
• One responsibility per step
• Error handling designed per step (retry / skip / escalate)
• Idempotency designed in
• Human-in-the-loop points identified
• State persistence strategy defined
• Observability plan documented

Phase 2: Implementation

• Orchestrator state persisted (not in-memory only)
• Input/output validation at every step boundary
• Explicit error states with codes
• Per-step retry policies (not blanket retry)
• Timeouts on external calls
• Structured logging: workflow_id, step_name, status
• End-to-end tests including failure scenarios

Phase 3: Operations

• Monitor success rates and step latency
• Track human approval latency
• Review retry rates for root causes
• Test disaster recovery on persisted state
• Schedule versioning reviews

Implement observability and monitoring from day one, and trace each decision so you can answer "why did it pick that step?" after the fact; agent observability shows what to record.

Two-Week Starter Plan

Week 1: Minimum Viable Workflow

Week 2: Production Hardening

Goal: A real workflow you can ship, observe, and iterate, not perfection on day 14.

For the full production pattern including tool calling, see Building Production AI Agents.

Frequently Asked Questions

What is an agentic workflow?

An agentic workflow is a process where an LLM orchestrator decides which steps to execute based on current state and constraints, rather than following a fixed predetermined sequence. The steps themselves are ordinary code with typed inputs and outputs; only the routing between them is delegated to the model. This keeps the system debuggable while still adapting to inputs that vary per request.

How is an agentic workflow different from a regular automation?

Regular automation runs fixed steps in a fixed order. Agentic workflows let the LLM choose paths within defined boundaries, which is useful when the correct sequence depends on input that varies per request, such as a support ticket that might need a refund, an escalation, or a knowledge-base answer. If the correct path can be computed from the input with an if statement, regular automation is the better choice.

Do I need LangGraph or CrewAI for agentic workflows?

No. You can build agentic workflows with a JSON state machine, Python functions, and an LLM orchestrator. Frameworks like LangGraph add structure, checkpointing, and interrupt handling, but they do not replace validation, observability, or infrastructure. Our framework comparison covers when the added structure is worth it.

What is the most common agentic workflow failure?

Skipping infrastructure: no state persistence, no structured logging, no idempotent steps. Pattern choice matters less than reliable step design. The second most common failure is an unbounded loop caused by retry and failure being collapsed into one status.

Should every step require LLM decision-making?

No. Most production agentic workflows are 80-90% deterministic step execution with LLM decisions at 10-20% of decision points. Every LLM decision adds latency, cost, and a non-deterministic branch you must test, so use code wherever the next step is a pure function of state.

How do I add human approval to agentic workflows?

Insert approval gates before irreversible steps. The step returns a needs_approval status, the workflow persists its state and pauses, a ticket or notification is created, and the run resumes when a human decides. Give approvals a TTL and an explicit expiry state so a forgotten ticket does not leave the run stuck forever.

Can agentic workflows work with MCP?

Yes. Steps can be MCP tools with the orchestrator deciding which to invoke, and the MCP schema doubles as the step's input validation. See our MCP tutorial for a working server and client.

Conclusion

Agentic workflows succeed when you invest in the skeleton, not just the brain:

• The orchestrator decides within constraints and returns schema-validated step names.
• Steps validate inputs, log with workflow_id, distinguish failed from retry, and are idempotent.
• Infrastructure persists state on every transition so a crash resumes rather than restarts.
• Approval gates protect irreversible actions and have TTLs.
• The best workflow is the simplest one that works: start linear, add complexity only when simplicity fails.

At HinterBuild, we design and ship agentic workflows for production through our AI agent development practice. Contact us to architect your first production workflow.

Resources:

• Anthropic: Building Effective Agents
• ReAct: Synergizing Reasoning and Acting in Language Models
• LangGraph Persistence
• Temporal Documentation
• Anthropic Tool Use Overview