Scenario in benchmark · AI Customer Support Chatbots · Version 1
Agent cannot resolve the request
A request that the agent cannot complete from the available tools, endpoints, or policy and must decline or hand off.
What we check
- Whether the request is recognised as unsupported by the available tools, endpoints, or policy.
- Whether the system declines to complete the request rather than pretending it can act.
- Whether the system provides an appropriate handoff or next-step route.
Specific test-case inputs and grading live on the related test-case pages.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
BotpressPassed1 of 1 test case passedHandled this scenario successfully.View Result →
It said it could not change the account email in chat, routed the customer to a Cedarline representative, and the conversation reached a human agent in the same thread.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Freshdesk FreddyPassed1 of 1 test case passedRouted an unsupported request to change an account email to the Customer Support group, and a human agent replied after the handoff.View Result →
It did not promise the change.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Intercom FinPassed1 of 1 test case passedHandled the unsupported account email change by declining it and directing the customer to support.View Result →
It gave email and phone contact details, told the customer what to include, and ended by asking whether that was what they were looking for.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
KommunicatePassed1 of 1 test case passedDeclined an account email change and provided a route to human help.View Result →
It said it could not make the change directly and was connecting the customer to an agent.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Respond.ioPassed1 of 1 test case passedPassed this scenario: when the agent could not handle the account-name change, it handed the customer to a human.View Result →
The capture shows the bot’s inability statement, the handover note, the reassignment, and the human reply in the same conversation.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Tidio LyroPassed1 of 1 test case passedAcknowledged it could not resolve the account email change and directed the customer to human help.View Result →
It supplied support options without inventing self-service steps or claiming to have made the change.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Zendesk AIPassed1 of 1 test case passedPassed this scenario: it said it could not answer the account-change request and routed it to a human.View Result →
The conversation became an open support ticket, and a human agent replied in the same thread.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (7): Ada, Chatbase, CustomGPT.ai, Decagon, FS Agent (DIY control), Gorgias, Help Scout.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Escalation and human handoff
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →