ProjectConcept design
Deterministic Gate: A Validation Layer for Multi-Agent Pipelines
A multi-agent handoff should earn the right to continue.
Documented
Why I made it
I designed this because fluent intermediate outputs can hide the exact failure that later becomes expensive to explain or undo.
A 5-agent pipeline of 90%-accurate agents produces 59% end-to-end accuracy: the math compounds against you, not for you. Most of that loss isn't model capability, it's unchecked error propagation between steps. This is a design for a deterministic gate placed between every agent handoff: validate the output against a schema, reinject the original task state so context doesn't drift, and route anything below a confidence threshold to a human queue instead of letting it flow downstream silently.
Tools: LangGraph · JSON Schema · Python
This is a design, not a shipped system. No code is public yet. I'm including it because the reasoning is concrete enough to evaluate on its own, and because it's the direct architectural answer to a failure mode I wrote about separately: see Agents fail at orchestration, not reasoning.
The problem
Chain five agents together, each individually 90% accurate, and the pipeline's end-to-end accuracy is roughly 0.9⁵ ≈ 59%, not 90%. That math is unforgiving, and it compounds against you by default: every additional agent in a pipeline is another place for a wrong intermediate output to become the next agent's input, treated as ground truth.
The instinct when a multi-agent system fails is to blame the model: swap in a bigger one, rewrite the prompt. Most of the time that's the wrong target. The actual failure is structural: nothing between steps checks an output for well-formedness, internal consistency, or even being on-topic before the next agent consumes it.Gates are not free. Every check is a place the pipeline can stall on something harmless, so the useful ones are cheap, deterministic and specific — not an LLM asked whether the output looks reasonable.
How it would work
The design is a gate, not a guard rail bolted onto one step; it sits at every handoff in the pipeline.
- Schema validation. Each agent's output is checked against a JSON schema before it's allowed downstream. Malformed output doesn't propagate; it triggers a retry with the validation error fed back to the agent that produced it.
- State reinjection. Long pipelines lose the original task in context drift: by step four, the agent is often optimizing for "produce plausible output given step three" rather than the actual user request. The gate reinjects the original task and key constraints at every step, not just the first one.
- Confidence-gated escalation. Steps below a confidence threshold (self-reported, or scored by a separate lightweight judge call) don't fail silently or retry forever — they route to a human review queue with the full step history attached.
| Failure type | Caught by the gate? | Why |
|---|---|---|
| Malformed output (wrong type, missing field) | Yes | Schema validation fails deterministically |
| Off-topic or task-drifted output | Partially | Reinjection reduces drift but doesn't eliminate it |
| Well-formed but factually wrong output | No | Schema validity says nothing about correctness; this needs the same eval/verification patterns as single-agent systems |
| Low-confidence edge case | Yes | Routes to human queue instead of propagating |
Key design decisions
- Deterministic checks first, model-based checks second. Schema validation is cheap and unambiguous; it should run before any LLM-based confidence scoring, not instead of it. Spending a model call to judge output that would have failed a free schema check is wasted latency.
- Reinjection, not just longer context. The fix for context drift isn't a bigger window; it's putting the original task back in front of the agent at every step, the same way a human manager would re-state the goal in a long meeting rather than trusting everyone remembers it.
- Escalation queue, not infinite retry. A confidence-gated step that fails repeatedly should reach a human, not loop. An unbounded retry on a genuinely ambiguous case just delays the same failure.
Status
Design only. The architecture above is fully specified but not implemented: there's no repo, no benchmark, and no claim here beyond "this is a reasoned response to a documented failure mode." If and when this gets built, the thing worth measuring is exactly the number that motivates it: does the gate measurably close the gap between per-step accuracy and end-to-end accuracy on a real multi-agent pipeline.
Limitations
- A schema gate doesn't catch confidently wrong output. It enforces structure, not correctness: a malformed JSON object gets caught; a well-formed JSON object containing a wrong answer does not.
- Reinjection has a cost. Repeating the original task and constraints at every step adds tokens to every call in the pipeline; for very long pipelines this is a real, not hypothetical, tradeoff against the latency/accuracy gain.
- This is unbuilt. Every claim above is a design argument, not a measured result.