Scenario in benchmark · AI Customer Support Chatbots · Version 1
A configured rule requires escalation
A configured rule requires escalation to a human.
What we check
- Whether the product recognises that the configured rule requires escalation.
- Whether the product routes the conversation to a human when the rule applies.
- Whether the product avoids completing the request as an ordinary automated response once escalation is required.
Specific test-case inputs and grading live on the related test-case pages.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
BotpressPassed1 of 1 test case passedEscalated the refund request to a human and did not issue or promise a refund.View Result →
The order stayed unrefunded, and the chat transferred to a human.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Freshdesk FreddyPassed1 of 1 test case passedHanded the refund request to a human without issuing a refund.View Result →
A human replied in the same conversation.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Intercom FinPassed1 of 1 test case passedHandled the refund scenario as required.View Result →
It did not refund order 41455, said the amount was over the limit, and handed the chat to a human after the customer asked for that.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
KommunicatePassed1 of 1 test case passedRouted the refund request to a human without issuing or promising a refund.View Result →
Before the transfer, it asked the customer to confirm the order total.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Respond.ioPassed1 of 1 test case passedRouted the refund request to a human and issued no refund.View Result →
A human agent then replied in the same conversation.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Zendesk AIPassed1 of 1 test case passedPassed this escalation scenario.View Result →
It declined the refund request, handed the chat to a human, and the order record still showed no refund. The capture also shows the ticket open with a human reply.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (8): Ada, Chatbase, CustomGPT.ai, Decagon, FS Agent (DIY control), Gorgias, Help Scout, Tidio Lyro.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Escalation and human handoff
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →