Multi-Agent Veto Boundary

2026-05-22#agent-runtime#multi-agent#orchestration#guardrails#reliability

Background

A point from MOSS 中文, roughly: in multi-agent collaboration, every agent should have an explicit "veto" boundary — the conditions under which it may refuse the orchestrator's instruction, and the conditions under which it must execute.

This is not a standard, named principle I could trace anywhere, but it is well worth keeping as an engineering judgment for agent runtime design.

Core judgment

Multi-agent collaboration is not "upstream speaks, downstream does" order-relaying. The orchestrator decomposes tasks and assigns intent, but the receiving agent should judge before executing:

  • Is this within my role / capability?
  • Are the preconditions, permissions and context sufficient?
  • Does the upstream input satisfy the agreed output contract?
  • Does the instruction conflict with locally verified facts?
  • Does it trigger privacy, security, cost or side-effect risk?
  • If refusing, what structured reason and next-step suggestion should be returned?

In one sentence:

An agent is not a function call; delegation needs an acceptance contract.

Without this boundary, a downstream agent treats upstream output as high-confidence fact and keeps propagating it. Once upstream is wrong, every later agent can "look like it executed successfully" while the whole chain has already drifted off course.

Tracing the lineage

In classical agent theory, autonomy already means an agent holds some control over its own actions and internal state, rather than being driven directly by external systems. In Wooldridge & Jennings' description of the weak notion of agency, autonomy, social ability, reactivity and pro-activeness are the core properties.

The older Contract Net / FIPA line of multi-agent research never modeled task allocation as one-way command either. It models allocation as negotiation: the initiator issues a call for proposals, a participant may propose or refuse, and the initiator then accepts or rejects. The seed of "the receiver first judges whether to participate" is already there.

Guardrails, tripwires, tool safeguards and human-in-the-loop interrupts in modern LLM agent engineering are, at heart, handling the same thing: don't only declare rules in the orchestrator prompt — make executable judgments at handoff and action boundaries.

Recent LLM-MAS failure studies supply the risk-side evidence. MAST (Multi-Agent System Failure Taxonomy) by Cemri et al. sorts multi-agent failures into system design, inter-agent misalignment, task verification and other categories; the papers on error cascades, online auditing and constraint drift all show the same mechanism: a single early error, once accepted downstream, becomes a trajectory-level failure over a long chain.

Design pitfalls

A veto boundary cannot be written as just "you may refuse". A genuinely usable boundary returns at least:

  • Refusal type: scope / precondition / policy / evidence / budget / schema
  • Triggering evidence: which input, which constraint, which tool risk
  • Recovery suggestion: what context to supply, which tool to degrade to, whether human confirmation is needed
  • Termination semantics: is this refusal a hard stop (FATAL), retryable (RETRYABLE), or handed to the orchestrator to arbitrate (ESCALATION)

Otherwise "veto power" becomes another kind of unreliability: agents passing the buck, endless clarification, retry loops.

Article angles worth developing

This could grow into an article on collaboration boundaries between agents:

  • Title direction: An agent is not a function call: why multi-agent collaboration needs veto boundaries
  • Central question: how do you divide the orchestrator's command authority from the sub-agent's autonomy?
  • Key contrast: role declarations in the prompt vs an acceptance contract in the runtime
  • Connects to existing threads: Trace Log, tool result role design, agent iteration harness
  • Closing point: a good agent runtime doesn't make every agent more obedient — it makes every handoff more auditable, more refusable, more recoverable

References