Every AI workflow has implicit expectations. The RAG system should answer from the knowledge base, not invent facts. The agent should escalate when it is uncertain, not guess. The classification pipeline should be consistent, not produce different outputs for semantically identical inputs.
These expectations usually live in people's heads, in scattered chat messages, or in a vague bullet point on a project brief. They are not written down in a way that is testable, and they are not shared between the team that builds the system and the team that evaluates it. When something goes wrong in production, everyone has a slightly different idea of what the system was supposed to do in the first place.
A behaviour contract changes this. It is a structured document that specifies what the workflow must do, what it must not do, and how it should behave in edge cases and error conditions. Every evaluation metric and test case traces back to a clause in this contract. It turns a collection of unwritten assumptions into a shared, testable specification that builders and evaluators can both point to.
Think of it as the bridge between intent and evidence. Intent lives in stakeholders' heads; evidence lives in test results. Without a contract, there is nothing connecting the two, and every disagreement about whether the system is 'good enough' becomes a matter of opinion rather than a matter of record.
What a Behaviour Contract Contains
Scope and Authority
What is the workflow authorised to do? A customer-service agent might be authorised to answer product questions, check order status and process returns — but not to offer discounts, modify billing information or make commitments about future features.
Define the scope explicitly. The most dangerous failures happen when a workflow acts outside its authority — not because of a technical bug, but because nobody specified the boundary. An agent that helpfully offers a refund it was never authorised to grant has not malfunctioned in a technical sense; it has done exactly what an under-specified system will eventually do.
Expected Behaviours
For each capability within scope, specify the expected behaviour with enough detail to write a test. 'The agent should answer product questions accurately' is not testable. 'The agent should answer product questions using only information from the current product catalogue; if the question refers to a product not in the catalogue, the agent should say it does not have information about that product' is testable.
The discipline here is to write every expectation as if a stranger will have to verify it without asking you what you meant. If a clause can be read two ways, it will be.
Prohibited Behaviours
Specify what the workflow must never do, regardless of the input. Common prohibitions include making commitments the organisation cannot fulfil, sharing confidential information, generating content that violates brand guidelines, and taking actions outside its defined scope.
Prohibited behaviours are often more important than expected behaviours. A workflow that occasionally fails to answer a question is an inconvenience. A workflow that occasionally promises something the organisation cannot deliver is a liability.
Edge Case Handling
Define how the workflow should handle inputs that fall outside normal parameters: ambiguous queries, contradictory information, requests in unsupported languages, unusual or malformed inputs, and requests that combine in-scope and out-of-scope elements.
The most common edge case specification is: 'When uncertain, escalate to a human.' But this needs to be more specific. What constitutes uncertainty? What does escalation look like — a message to a queue, a notification to a team, a live handoff? What information should the escalation include?
Error Conditions
Define how the workflow should behave when components fail: the knowledge base is unavailable, the downstream API returns an error, the model exceeds its context window, the response takes too long. Error handling for AI workflows is often neglected because the model itself does not throw exceptions — it just generates a worse response, and a worse response is much harder to detect than a stack trace.
A Worked Example
Consider an internal HR assistant that answers employee questions about leave policy. A minimal behaviour contract might read: scope — answer questions about holiday, sick leave and parental leave using the current policy documents only. Expected behaviour — quote the relevant policy section and link to the source document. Prohibited — never estimate an individual's remaining leave balance (that lives in a separate system), never speculate about exceptions, never give legal advice. Edge case — if a question mixes policy with a personal-data request ('how many days do I have left?'), answer the policy part and direct the balance question to the HR portal. Error condition — if the policy documents cannot be retrieved, say so and escalate, rather than answering from memory.
This illustrative contract is only a paragraph long, yet it already generates a dozen concrete test cases: a straightforward policy question, an out-of-scope balance question, a mixed question, a question about a policy that does not exist, a retrieval failure, and so on. That is the point — a short, precise contract produces a rich, testable evaluation suite almost for free.
From Contract to Evaluation
Deriving Test Cases
Each clause in the behaviour contract generates one or more test cases. Scope and authority clauses generate boundary tests. Expected behaviour clauses generate positive test cases. Prohibited behaviour clauses generate negative test cases. Edge case clauses generate robustness tests. Error condition clauses generate failure-mode tests.
Prioritising by Impact
Not all contract clauses carry equal weight. A violated prohibition (the agent promised something it should not have) is more severe than a missed expectation (the agent did not answer a question it could have). Prioritise test cases by the business impact of the failure, not by the ease of testing.
Living Documentation
A behaviour contract is not a launch document that gets written once and filed away. It evolves as the workflow changes, as new edge cases are discovered in production, and as business requirements shift. After every production incident, ask: 'Is there a clause in the behaviour contract that should have caught this? If not, add one.'
Building the Contract Incrementally
You do not need a complete contract before you start. The most reliable approach is to begin with the highest-impact clauses — the prohibitions that would cause real damage and the two or three core capabilities the workflow exists to deliver — and expand from there. A first draft written in an afternoon is worth far more than a perfect document that never gets written.
Each production incident and each surprising evaluation result becomes a candidate clause. Over a few weeks, the contract grows from a rough sketch into a precise specification that reflects how the system actually behaves and how you want it to behave. The gap between those two is exactly the work.
The Practical Value
Teams that write behaviour contracts before building evaluation suites consistently report that the process of writing the contract — the conversations it forces about scope, authority, edge cases and error handling — is as valuable as the document itself. These conversations surface assumptions, resolve disagreements, and create shared understanding between builders and evaluators.
A behaviour contract does not make evaluation easy. But it makes evaluation possible. Without one, you are testing against unwritten, unshared expectations — and discovering the misalignment in production.