AI agents in customer-facing roles have crossed a critical threshold. What began as simple FAQ bots — matching keywords to pre-written answers — has evolved into sophisticated language-model-powered workflows capable of understanding nuance, generating human-like responses, processing multi-step requests and triggering downstream actions in CRM, ticketing and order-management systems.
The commercial appeal is obvious. AI agents can operate around the clock, handle multiple conversations simultaneously, respond in multiple languages and scale instantly to meet demand spikes. When the workflow is reliable, they reduce wait times, resolve issues faster and free human agents for cases that genuinely require human judgement.
But there is a fundamental reliability problem. The language model at the core of these workflows is a probabilistic text generator. It predicts the next most likely token given its context, but it has no concept of truth. It cannot distinguish between a correct policy statement and a plausible-sounding fabrication. It does not know the boundaries of its knowledge. And it can generate responses with absolute confidence that are completely wrong.
When an AI agent hallucinates in a customer-facing workflow, the consequences are not academic. The agent may promise a refund the company does not offer. It may quote a policy that does not exist. It may provide safety information that is inaccurate. And critically, courts and regulators have made clear that the company is liable for these outputs — the agent speaks on behalf of the organisation.
Liability Is Settled: You Own What Your AI Agent Says
The most significant legal precedent was established in 2024 when a Canadian tribunal ruled against Air Canada after its customer-service chatbot incorrectly told a passenger he could book a full-fare flight and then apply for a bereavement discount retroactively. The airline argued the chatbot was a separate entity. The tribunal rejected this, ruling that Air Canada was responsible for all information its AI provided.
This principle has been echoed by regulators globally. The EU AI Act requires that AI systems interacting with people must disclose they are AI. The FTC has signalled that hallucinated information can constitute deceptive practices. For IT teams deploying customer-facing AI workflows, the message is clear: evaluation is not optional.
Where Customer-Facing AI Workflows Fail
Understanding the failure modes is the starting point for any evaluation.
Hallucination in Grounded Systems
Even with retrieval-augmented generation (RAG) — which grounds the model in a knowledge base rather than relying on training data alone — hallucinations persist. Stale or contradictory knowledge-base content, poor document chunking, information overload from too much retrieved context and conflicts between the knowledge base and the model's training data all contribute.
A RAG-powered agent with a poorly maintained knowledge base hallucinates just as confidently as an ungrounded model — the only difference is that the hallucination may be based on outdated information rather than pure fabrication.
Escalation Failures
The most important design principle for customer-facing AI is: when in doubt, escalate to a human. But this runs directly counter to the economic incentive driving deployment — resolving as many interactions as possible without human involvement.
Effective escalation requires confidence scoring combined with intent classification that identifies high-risk scenarios (billing disputes, refunds, safety questions, complaints) and routes them to human agents regardless of model confidence. Without systematic evaluation of escalation behaviour, teams cannot know whether their agent is escalating too rarely (risky) or too often (defeating the purpose).
Downstream Action Failures
Modern customer-facing agents do not just talk — they act. They process refunds, update account settings, create tickets, modify orders. When the language model misinterprets a request and triggers the wrong downstream action, the damage goes beyond a bad answer: it changes state in production systems.
Evaluating these workflows requires testing the full chain from user intent through model interpretation to API call and state change — not just the text the agent generates.
Prompt Manipulation
Adversarial inputs — prompt injection, jailbreaking, role-playing exploits — can cause agents to ignore their instructions, reveal system prompts, offer unauthorised discounts or bypass verification procedures. In a customer-facing context with downstream actions, prompt manipulation does not just produce bad text — it can trigger real business operations.
Building an Evaluation Architecture
Effective evaluation for customer-facing AI workflows requires multiple layers, not a single test suite.
Behaviour Contracts
Define explicitly what the agent should and should not do. A behaviour contract specifies: which actions the agent can take, which topics it must escalate, which promises it is authorised to make, and what it should say when it does not know. Every evaluation metric traces back to a clause in this contract.
Retrieval and Generation Evaluation
Evaluate retrieval and generation separately. Retrieval evaluation measures whether the right documents are found (recall) and whether irrelevant documents are excluded (precision). Generation evaluation measures whether the answer is faithful to the retrieved context (groundedness) and whether it actually answers the user's question (relevance).
Escalation and Abstention Testing
Systematically test whether the agent escalates when it should. Feed it queries that fall outside its knowledge, queries that are ambiguous, queries that involve high-risk operations. Measure escalation rate, false-escalation rate and the most dangerous metric: missed escalations where the agent should have deferred but did not.
End-to-End Workflow Testing
For agents with downstream actions, test the complete workflow: user message → intent classification → model response → API call → state change. Verify that the right action was taken, the right parameters were passed and the resulting state matches expectations. This is where traditional software testing and AI evaluation meet.
Regression Testing After Changes
Knowledge-base updates, prompt changes and model upgrades can all change agent behaviour in unpredictable ways. A response that was correct last week may hallucinate after a knowledge-base refresh. Run your evaluation suite after every change, not just at initial deployment.
Production Monitoring
Pre-deployment evaluation is necessary but not sufficient. Monitor escalation rates, customer satisfaction, conversation abandonment, response accuracy through periodic sampling and sentiment patterns. Track for drift — changes in behaviour over time due to knowledge-base updates, model changes or shifts in query patterns.
The Evaluation Standard for Customer-Facing AI
The organisations that will deploy customer-facing AI successfully are those that treat these agents not as cost-reduction tools but as representatives that require systematic evaluation: behaviour contracts, retrieval evaluation, escalation testing, end-to-end workflow verification and continuous production monitoring.
For the IT teams building and maintaining these workflows, evaluation is not a one-off gate before launch. It is an ongoing discipline — the same discipline you apply to any production system that acts on behalf of your organisation and your clients.