Deterministic vs agentic workflows: when to trust the LLM
Most AI systems fail because they hand the model decisions it should never own. The fix is knowing exactly where the spine ends and the engine begins.
A deterministic workflow runs on predictable, tested code — rules and state machines — and always produces the same output for the same input. An agentic workflow lets a language model decide the next step. The skill isn't choosing one over the other; it's drawing the line: the LLM handles ambiguity, deterministic code guarantees correctness. Get that line wrong and your system is either brittle or unreliable.
Why "let the agent figure it out" fails in production
An LLM is a brilliant interpreter of fuzzy input and a poor guarantor of exact facts. Ask it to summarize a messy ticket — perfect. Ask it to guarantee that a refund respects your policy — and it will confidently get it wrong often enough to matter. Fully agentic systems inherit that unreliability at every step.
Why fully deterministic fails too
Hard-coding every path is robust but rigid. Real users phrase things in a thousand ways, upload messy documents, and ask questions you didn't anticipate. Pure determinism can't absorb that variance — which is exactly what LLMs are good at.
Own the spine, rent the engine
The durable pattern is to make the spine deterministic — the logic that must be right every time — and to call the model only at the fuzzy decision nodes: understanding intent, extracting a field, drafting a reply. Crucially, the model's output is verified before it drives an action.
user input → [LLM: interpret] → deterministic router
→ deterministic flow (critical) ── or ── verified agent step (fuzzy)
→ action
You get the reliability of a classic system and the flexibility of natural language, without betting correctness on a probabilistic model.
How to draw the line in practice
Ask two questions of every step:
- Does this have to be correct every time? If yes, it belongs in the deterministic spine.
- Does this require interpreting ambiguity? If yes, rent the engine — but verify its output.
This is how Cardynal handles ~1,300 support conversations a day with ~4-second responses: an intent router arbitrating between deterministic flows and verified agents, live in three languages.
If you're architecting an AI product and want a second read on where that line should sit, let's talk.