A sales demo shows you a voice agent at its best. The real test is how it behaves when a conversation goes somewhere the demo didn't rehearse. Here's what's actually worth testing before you commit to a vendor.
Malik Kolade
•
A sales demo shows you a voice agent at its best. The real test is how it behaves when a conversation goes somewhere the demo didn't rehearse. Here's what's actually worth testing before you commit to a vendor.
Why the demo isn't the test
A demo is a controlled environment. The questions are usually ones the agent has been tuned to answer well, asked in a way that doesn't stress the system. That tells you almost nothing about how it'll cope with a real customer who interrupts, mumbles, changes their mind halfway through a sentence, or asks something nobody thought to prepare for. The evaluation that matters happens after the demo, in a trial built around realistic, slightly messy scenarios.
What to actually test
Interruptions. Talk over the agent mid-sentence and see what happens. It should stop naturally and listen, the way a person would, rather than talking over you or losing track of where the conversation was.
Accents, pace, and background noise. Test with different accents, faster speech, pauses, and incomplete sentences, not just clear, well-paced test questions. A voice agent that only performs well with one accent or speaking style will frustrate a real customer base fast.
Unclear or off-script input. Give it an ambiguous answer, or ask something completely outside the expected flow, and see whether it asks a sensible follow-up rather than guessing or stalling.
Real business knowledge, not generic questions. Upload your own source material, your policies, your product information, your actual FAQs, and test whether the agent uses that specific knowledge accurately in a live conversation. A vendor's generic demo script tells you nothing about how it'll handle your business's actual detail.
What happens after the conversation ends. Does the platform capture the customer's information? Can it create or update a support ticket? Can it route the conversation to a person? Is the conversation recorded and available to review afterwards? Can you identify which interactions failed and use that to improve the agent?
What bad behaviour actually looks like
It's easier to evaluate a voice agent once you know what a failure looks like, rather than only what success looks like.
Talking over the customer instead of yielding when interrupted
Repeating the same question after it's already been answered
Misunderstanding an accent and confidently giving the wrong answer anyway, rather than checking
Losing the thread of the conversation partway through
Getting stuck in a loop, unable to move the conversation forward
Not recognising that it has failed, and continuing rather than escalating to a person
That last one is the most important. An agent that knows when it's out of its depth and hands over cleanly is doing its job properly, even in the conversations it can't fully resolve. An agent that pushes on regardless is the one that damages trust.
The checklist
Area | What to test | What good looks like |
Interruption handling | Talk over the agent mid-sentence | Stops and listens, doesn't lose the thread |
Accent and pace coverage | Different accents, faster speech, pauses | Consistent accuracy, not just with one voice profile |
Ambiguous input | Give an unclear or incomplete answer | Asks a sensible follow-up rather than guessing |
Knowledge accuracy | Test with your own real source material | Uses it correctly, not a generic script |
Failure recognition | Ask something genuinely outside its knowledge | Recognises the gap and hands over, rather than bluffing |
Handover | Force an escalation scenario | Hands to a person cleanly, with context intact |
Post-conversation capture | Check what happens after the call ends | Captures details, creates or updates a ticket, logs the interaction |
Review and improvement | Ask to see a failed interaction afterwards | Failed interactions are visible and usable to improve the agent |
How to run this as an actual trial, not just a read-through
Don't evaluate a voice agent purely on a sales call. Ask for a proper trial period, load in a sample of your own real knowledge, and run it through the scenarios above yourself, including the awkward ones. A vendor confident in what they've built won't mind you trying to break it a little. One that only wants to show you a scripted demo is telling you something too.
FAQs
What's the single most important thing to test in an AI voice agent trial?
Whether it recognises when it's out of its depth and hands over cleanly, rather than bluffing through an answer it doesn't actually have. Everything else being equal, that behaviour is what separates a production-ready agent from a demo that falls over with real customers.
Should I test the agent with my own business's information, or is a generic demo enough?
Test with your own material. A generic demo shows you the underlying technology works in principle; it tells you nothing about whether the agent will use your specific policies and product details accurately, which is the part that actually matters.
How do I know if a voice agent handles interruptions properly?
Talk over it mid-sentence during a test call. It should stop and listen naturally, the way a person would, rather than continuing to talk over you or losing track of the conversation.
What should happen after a voice conversation ends?
At minimum, the platform should capture the customer's details, create or update a support ticket where relevant, and make the conversation available to review. If you can't see which interactions failed, you can't improve the agent, which makes this a genuine evaluation criterion, not a nice-to-have.
