How to evaluate an agent beyond its best demo
Define the final outcome, preserve failed attempts and measure correction effort. A synthetic invoice example shows how to turn that advice into a usable evaluation.
Read analysisRead the evidence.
Put the work in context.
Source analysis Agents
A tool operation and an independent agent collaboration expose different responsibilities. Map the service first, then check the protocol version and implementation.
AI-assisted editorial review · Sources checked · Recheck due
The takeaways
The A2A project distinguishes structured tool and resource access through MCP from collaboration between independent, often opaque agents through A2A. An agent reached through A2A can use MCP to access its own tools.
Source: A2A and MCP: Detailed Comparison
The 28 July 2026 MCP release introduces a stateless core, retires the initialize/initialized exchange and Mcp-Session-Id header, and places Tasks in an extension. Applications can still carry state explicitly. A statement that MCP cannot support tasks misses this distinction.
Editorial interpretation
Our first illustrative service exposes a customer lookup. The caller provides an identifier and receives a structured record. This is a straightforward candidate for a tool interface. The service owner still needs to decide which records the caller may access and what happens when the identifier is missing.
Our second example delegates an investigation to a separate supplier-support agent. The caller requests an explanation of a delayed shipment; the recipient chooses its own sources, asks follow-up questions and returns a report. An independent-agent interface is worth considering because the recipient owns part of the work and interaction.
The combined example uses both boundaries. A coordinating agent delegates the investigation, while the recipient uses its own customer and shipment tools. These are proposed architecture examples, not tested integrations. The useful design artifact is a short list of what each service receives, decides and returns.
Editorial implications, not an independently measured result.
Editorial interpretation
For the shipment example, define the requested outcome as a report with shipment identity, evidence links, unresolved questions and a named next action. Decide whether the recipient can only investigate or may also request a replacement. That difference changes the authorization needed even if the communication protocol stays the same.
Then walk through an interruption. The caller loses its connection after handing over the request. How will it learn whether the recipient accepted the work? Which identifier lets it reconnect without accidentally starting a second investigation? What does cancellation mean if a downstream action has already happened?
These questions should become integration checks with observable outcomes. Include refusal, partial completion and timeout alongside the successful response. A protocol choice helps structure communication, while the application contract supplies the meaning of completion.
For a small function entirely inside one application, first ask whether adding a network protocol solves a real interoperability need. A clear internal interface may be enough. The purpose of this decision exercise is to expose responsibility and compatibility requirements before increasing the number of moving parts.
Editorial implications, not an independently measured result.
The table scrolls horizontally on smaller screens.
| Proposed service | Interface to investigate | Outcome to define |
|---|---|---|
| Customer lookup | MCP tool | Permitted record or explicit no-match |
| Supplier investigation | A2A agent | Report and unresolved questions |
| Investigation plus local lookups | Both boundaries | Delegation outcome and tool access scope |
Read alongside A2A and MCP: Detailed Comparison / MCP: The 2026-07-28 Specification.
Keep the question open
Does your service expose a defined operation, accept responsibility for an investigation, or need both kinds of interface?
The evidence record
Source publication dates are reported as stated in the source record. An unstated publication date is not our check date.
Capabilities described by a vendor or paper do not establish performance in your environment. Our editorial standards explain the distinction.