Agent Reference Architectures

Choose the least complex shape that satisfies the workflow

• Start with a direct model call or one agent; add coordination only for a named requirement.
• Keep identity, state, tools, approvals, telemetry, and evidence as explicit boundaries in every shape.
• Select from client constraints and tested behavior, not from a diagram or vendor feature list.

This guide helps a Forward Deployed Engineer, solution architect, technical owner, operations owner, or assurance reviewer choose a production architecture shape for one approved workflow. It covers single-agent, multi-agent/orchestrated, human-in-the-loop, and retrieval-grounded patterns.

Microsoft's architecture guidance places direct model calls, single agents with tools, and multi-agent orchestration on a complexity spectrum. It recommends the lowest level that reliably meets the requirement because coordination adds cost, latency, and failure modes [R302]. This guide uses that principle as its default:

1. Use a direct model call when one bounded request can produce the accepted result.
2. Use one agent when the workflow needs dynamic tool use or an iterative loop inside one domain.
3. Add retrieval when the answer must be grounded in governed knowledge.
4. Add a durable human decision point when execution must stop for an authorized decision.
5. Add multiple agents only when specialization, parallel work, distinct boundaries, or dynamic delegation provides measurable value that one agent cannot provide [R302].

These are reference shapes, not deployable products. Each must be instantiated with client-specific identity, network, data, recovery, assurance, and operating decisions before build. NIST's AI RMF places context, human roles, measurement, and ongoing risk response inside one iterative Govern-Map-Measure-Manage lifecycle; the architecture must expose the boundaries and evidence those activities depend on rather than treating assurance as a later overlay [R301].

Read evidence as architecture input, not product proof

• Open specifications define portable interfaces; they do not prove an implementation conforms.
• Vendor architectures show supported product patterns; they do not establish client acceptance.
• Unknowns and preview features remain proof-of-value work.

The evidence language follows the shared glossary. Vendor-documented means the supplier documents a capability. Independently anchored means an open specification defines the interface, not that a supplier implementation is complete or conformant. Unknown means the reviewed sources do not establish the claim.

This guide cites the Model Context Protocol specification dated 2026-07-28, which the official documentation index identified as the current specification when retrieved on 2026-08-21 [R304]. It does not silently inherit R1's superseded 2025-11-25 citation. The A2A specification page identifies 1.0.0 as its latest released version, so this guide uses that exact version rather than inferring a version from the unversioned landing page [R305].

OpenTelemetry's generative-AI agent span conventions are marked Development. They can align field names and trace structure, but they are not a stable production contract; pin the schema version used by an implementation and test export compatibility [R306].

Build every shape on one production spine

• The spine makes trust and operating boundaries visible before application-specific components.
• Data crosses only named interfaces with an owner, identity, contract, timeout, and evidence event.
• A runtime product can host several boxes, but it must not erase their responsibility boundaries.

User or system
      |
      v
[Entry boundary] -> [Identity and authorization] -> [Orchestrator or agent]
                                                     |       |       |
                                                     v       v       v
                                                  [State] [Tools] [Knowledge]
                                                     \       |       /
                                                      v      v      v
                                                   [Telemetry stream]
                                                          |
                                                          v
                                                   [Evidence pointers]

The Azure baseline Foundry chat architecture is one vendor example of this separation: managed identity and private endpoints form access boundaries; Cosmos DB holds conversation state; Search provides grounding; Firewall inspects external tool egress; and Application Insights receives telemetry [R307]. Treat the pattern as vendor-documented and re-map every component when using another runtime.

MCP 2026-07-28 independently anchors a host-client-server tool boundary. The host coordinates clients and enforces consent and security policy; each client communicates with one server; servers remain focused and cannot see other servers or the full conversation [R304]. MCP does not supply the client's authorization policy, tool allow-list, recovery behavior, or acceptance evidence.

Single-agent reference architecture

• One agent owns one bounded loop, one task state, and a small tool set.
• The host enforces iteration, time, spend, tool, and data boundaries outside the model.
• Escalation is an explicit outcome, not an infinite retry strategy.

[Request]
    |
    v
[Input validation] -> [Single agent loop] <-> [Task/conversation state]
                           |    |
                           |    +-> [Knowledge retrieval, optional]
                           v
                     [Tool gateway]
                           |
                           v
                 [Approved destinations]

Every step -> [Trace + evidence pointers]
Terminal outcomes -> complete | fail | cancel | escalate

Use this shape when one domain owner can define the goal, the tool set is small enough to review, and one agent can keep the necessary context without overloaded instructions. Microsoft's guidance calls a single agent with tools the usual enterprise default and specifically recommends iteration limits to guard against infinite tool-call loops [R302].

Execution contract

Foundry's runtime model separates a versioned agent definition, a persisted conversation, and one response as the unit of execution [R308]. That separation is useful beyond Foundry: bind every run to an immutable agent configuration, keep state ownership explicit, and record each execution attempt independently.

Acceptance evidence

• A loop-limit test proves the agent terminates when a tool or model keeps requesting another turn.
• Tool tests prove allowed destinations succeed and blocked destinations fail before execution.
• Replay tests prove the same request and checkpoint do not duplicate an irreversible action.
• Failure tests prove timeout, cancellation, and escalation produce distinct terminal records.
• Trace tests correlate the request, model calls, retrieval, tools, and final disposition.

The detailed implementation of memory, state, tool design, and failure handling belongs to R4. R3 specifies where those boundaries sit, what crosses them, and what evidence the architecture must expose.

Multi-agent and orchestrated reference architecture

• Add an orchestrator only when specialization or coordination is measurable and necessary.
• Give every agent a bounded identity, input/output contract, tool scope, and state ownership rule.
• Make aggregation and partial failure deterministic before allowing agents to call one another.

                         +-> [Agent A] -> [Result A] -+
[Request] -> [Coordinator] -> [Agent B] -> [Result B] ---> [Aggregator] -> [Outcome]
             |           +-> [Agent C] -> [Result C] -+
             |
             +-> [Shared task ledger, not shared hidden memory]

Each edge: caller identity | task ID | schema | timeout | retry | evidence pointer

Microsoft documents five orchestration patterns: sequential, concurrent, group chat, handoff, and magentic. Sequential execution is a predefined pipeline. Concurrent execution fans out independent work and may aggregate results. Group chat uses a manager to choose speakers. Handoff transfers control dynamically. Magentic uses a manager-led plan and progress ledger [R302]. These are vendor-documented design patterns, not proof that any runtime implements them safely.

A2A 1.0.0 provides a portable boundary for opaque remote agents: Agent Cards declare capabilities and authentication requirements; messages can create stateful tasks; task status, streaming, polling, cancellation, and push notifications support long-running work [R305]. It does not define the coordinator's decomposition, shared-state conflict policy, aggregation rule, or compensation logic. Keep those decisions in the architecture.

Shared state and partial failure

• Prefer immutable task inputs and append-only result records over multiple agents mutating one conversational object.
• Give each work item a parent task ID, child task ID, attempt, deadline, and idempotency key.
• Define whether one child failure fails the whole task, yields a partial result, triggers a substitute, or starts compensation.
• Aggregate only schema-valid results. Record missing, late, rejected, and conflicting results rather than asking a model to hide them.
• Propagate cancellation from parent to children and record children that could not be cancelled.

A2A agents collaborate without sharing internal state, memory, or tools; the protocol exchanges declared messages, tasks, status, and artifacts across the boundary [R305]. Use that opacity as the default even inside one platform. If agents share a store, expose it as a separately owned state service with concurrency and authorization rules.

Acceptance evidence

• Pattern comparison proves the selected coordination shape beats the single-agent baseline on the named requirement.
• Contract tests reject incompatible Agent Cards, message schemas, and unsupported capabilities.
• Fault injection covers one child timeout, one malformed result, one duplicate completion, and one unavailable coordinator.
• Aggregation tests prove deterministic handling of missing and conflicting results.
• Trace tests preserve parent-child correlation across every remote-agent boundary.

Human-in-the-loop reference architecture

• Pause durably before the protected transition; do not hold a process, socket, or model call open.
• Bind the decision to the exact request and configuration version the approver reviewed.
• Treat approve, deny, timeout, cancel, supersede, and resume as separate recorded paths.

[Agent proposes action]
          |
          v
[Persist request + version] -> [Approval queue] -> [Authorized decision]
          |                         |                       |
          |                         +-> (a)                 +-> (b)
          |                                                 |
          +---------------- approved same version ----------+
                                    |
                                    v
                              [Resume execution]

(a) timeout/escalate
(b) deny/cancel/supersede

Temporal's vendor-documented approval pattern blocks a durable workflow until a signal carries the decision, approver, comments, and timestamp, or a durable timeout follows the rejection or escalation path [R303]. A2A 1.0.0 independently defines interrupted INPUT_REQUIRED and AUTH_REQUIRED task states and continuation on the same task, but it does not define approval authority, approval expiry, or version binding [R305].

Technical approval contract

The approval queue is a technical position, not a decision-rights model. R3 owns where the boundary sits, how pause/resume and timeout/denial paths behave, and the event contract emitted there. The AI and Agent Governance Framework (GOV1) owns who may decide, when an approval is required by risk tier or lifecycle gate, and restart authority. R3 references that authority and does not restate it.

R3 names the technical records and stable pointers the architecture must produce. The AI Agent Control Checklists (GOV3) own control outcomes and evidence sufficiency. GOV6 owns audit-pack structure, assembly, retention ownership, and assurance presentation. This guide defines neither a parallel control catalog nor an evidence-pack schema.

Acceptance evidence

• Restart the worker while approval is pending and prove the request remains pending once.
• Submit an approval for an obsolete version and prove resume is rejected.
• Exercise approve, deny, timeout, cancel, duplicate decision, and late decision paths.
• Prove the deciding identity has the GOV1 authority reference required for the request.
• Correlate the decision event to the exact resumed action and its final disposition.

Retrieval-grounded reference architecture

• Separate ingestion-time trust from retrieval-time authorization.
• Bind every answer to an index version, query identity, retrieved items, and source pointers.
• Evaluate retrieval and answer behavior separately; a fluent answer can still retrieve the wrong evidence.

Ingestion plane
[Sources] -> [Classify + authorize] -> [Parse/chunk] -> [Embed/index]
                  |                                        |
                  +---------- [Lineage + index version] ----+

Query plane
[User identity] -> [Query policy] -> [Authorized retrieval] -> [Rerank]
                                                               |
                                                               v
                                                    [Grounded generation]
                                                               |
                                                               v
                                            [Answer + citations + evidence]

Azure AI Search documents full-text, vector, hybrid, and agentic retrieval over indexed and remote content, with index-time enrichment and document-level access-control options [R309]. Its security filter pattern also makes an important limitation explicit: a caller-supplied principal string in an OData filter is not authentication or authorization. The application must establish identity, and production designs should prefer supported ACL enforcement where available [R310].

Answer synthesis can return natural-language answers with citations to retrieved sources, but the feature and its 2026-05-01-preview API are Preview and not recommended for production workloads [R311]. Therefore citations are an architecture requirement to implement and test, not a reason to assume one vendor feature is production-ready.

Ingestion contract

• Register source owner, classification, allowed audiences, extraction method, and retention rule.
• Preserve source URI, source version or integrity value, chunk lineage, and index version.
• Reject untrusted executable content and record parsing failures rather than silently omitting it.
• Re-index through an explicit publish step so query traffic never mixes an unapproved partial index.
• Define deletion propagation from source to chunks, embeddings, caches, and evidence pointers.

Query contract

• Authenticate the caller before forming retrieval filters.
• Apply authorization before ranking and before returning snippets or metadata.
• Record query, rewritten subqueries, filters, corpus/index version, retrieved item IDs, scores, and reranker version.
• Give generation only the retrieved fields needed for the answer.
• Return source pointers that resolve to the exact retrieved versions; mark unsupported statements and empty retrieval explicitly.

Acceptance evidence

• Retrieval tests measure whether required evidence appears in the retrieved set.
• Authorization tests prove permitted documents are returned and forbidden documents are absent from results, metadata, caches, and traces.
• Grounding tests distinguish unsupported statements from incorrect but cited statements.
• Freshness and deletion tests prove index changes reach query results within the accepted window.
• Failure tests cover no result, low relevance, conflicting sources, stale index, unavailable embedding/model service, and citation-resolution failure.

R3 specifies the evidence each retrieval shape must produce. R5 owns dataset construction, scoring methods, thresholds, and regression practice.

Compose patterns without hiding boundaries

• Human approval and retrieval are cross-cutting components, not alternative runtime categories.
• Compose by preserving each boundary's contract and correlation IDs.
• Reject a composition whose combined failure behavior cannot be explained before build.

A single agent can use retrieval and pause for approval. A sequential workflow can retrieve before each stage and pause before one protected transition. A concurrent system can give each specialist a separately authorized retrieval scope. Composition is valid only when the combined architecture still has one accountable task owner and explicit terminal states.

MCP and A2A are complementary boundaries rather than competing orchestration strategies: MCP connects a host to focused context and tool servers, while A2A connects opaque agents through declared capabilities and task operations [R304][R305]. Neither protocol chooses the system's approval authority, retrieval policy, aggregation rule, or recovery objective.

Use a pattern decision record

• Record the simplest viable baseline and the requirement that forces each added component.
• Decide from failure behavior and acceptance evidence, not from nominal feature coverage.
• Re-open the choice when the workflow, boundary, or evidence requirement materially changes.

Record one outcome:

• Adopt - the shape passes mandatory boundaries and acceptance evidence.
• Revise - the shape is plausible but one or more boundaries or tests remain incomplete.
• Reject - the shape adds unjustified complexity or fails a mandatory boundary.

No weighted score can offset a missing identity boundary, unauthorized retrieval, unbounded loop, unrecoverable approval state, or undefined partial-failure outcome.

Keep reusable architecture and test records

• Architecture records make responsibility, versions, and unknowns reviewable.
• Acceptance records bind a measured result to an environment and artifact.
• Stable pointers connect technical evidence to the GOV3 sufficiency assessment and later GOV6 pack.

Architecture decision record

Acceptance-test record

Respect adjacent deliverable boundaries

• R3 owns component position, responsibility boundaries, information flow, and required evidence.
• Implementation methods, evaluation methods, monitoring programs, and governance authority remain with their assigned deliverables.
• A reference architecture points to those owners; it does not copy their frameworks.

The division of responsibilities among the runtime frameworks is:

• R3 vs. R4: R3 says that a state, tool, and failure boundary exists, where it sits, what crosses it, and which architecture-level outcomes must be visible. R4 says how to implement task decomposition, tools, memory, state, retries, compensation, and failure handling.
• R3 vs. R5: R3 says which acceptance evidence each pattern must produce. R5 defines datasets, evaluators, rubrics, thresholds, regression suites, and online quality measurement.
• R3 vs. R6: R3 says which components emit telemetry, where correlation must cross boundaries, and which events must be available. R6 defines instrumentation detail, metrics, alerting, dashboards, retention, and incident response.
• R3 vs. R7: R3 records bounded execution and architecture-specific capacity or failure choices. R7 defines runtime cost optimization, reliability targets, routing, and operational control.

The cross-track split is:

• GOV1 owns decision rights, risk-tier and lifecycle-gate requirements, approval authority, and restart authority.
• GOV3 owns control outcomes and the test for sufficient control evidence.
• GOV6 owns audit-pack structure, assembly, retention ownership, and assurance presentation.
• R3 owns only the technical placement and behavior of the approval boundary and the records and stable pointers the architecture emits.

This guide deliberately excludes runtime ranking or vendor selection; detailed agent, tool, memory, and failure implementation (R4); evaluation methodology (R5); monitoring operations (R6); cost and reliability control (R7); governance or security control catalogs (GOV1-GOV3); decision rights (GOV1); audit evidence-pack design (GOV6); and regulatory conclusions.

Sources

The references below identify the external sources cited in this framework. Responsibility boundaries with adjacent frameworks are described above.