Scenario in benchmark · AI Customer Support Chatbots · Version 1

Agent cannot resolve the request

A request that the agent cannot complete from the available tools, endpoints, or policy and must decline or hand off.

Capability: Escalation and human handoff1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the request is recognised as unsupported by the available tools, endpoints, or policy.
  2. Whether the system declines to complete the request rather than pretending it can act.
  3. Whether the system provides an appropriate handoff or next-step route.

Specific test-case inputs and grading live on the related test-case pages.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Passed · 7
BotpressPassed1 of 1 test case passedHandled this scenario successfully.View Result →

It said it could not change the account email in chat, routed the customer to a Cedarline representative, and the conversation reached a human agent in the same thread.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Freshdesk FreddyPassed1 of 1 test case passedRouted an unsupported request to change an account email to the Customer Support group, and a human agent replied after the handoff.View Result →

It did not promise the change.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Intercom FinPassed1 of 1 test case passedHandled the unsupported account email change by declining it and directing the customer to support.View Result →

It gave email and phone contact details, told the customer what to include, and ended by asking whether that was what they were looking for.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

KommunicatePassed1 of 1 test case passedDeclined an account email change and provided a route to human help.View Result →

It said it could not make the change directly and was connecting the customer to an agent.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Respond.ioPassed1 of 1 test case passedPassed this scenario: when the agent could not handle the account-name change, it handed the customer to a human.View Result →

The capture shows the bot’s inability statement, the handover note, the reassignment, and the human reply in the same conversation.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Tidio LyroPassed1 of 1 test case passedAcknowledged it could not resolve the account email change and directed the customer to human help.View Result →

It supplied support options without inventing self-service steps or claiming to have made the change.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Zendesk AIPassed1 of 1 test case passedPassed this scenario: it said it could not answer the account-change request and routed it to a human.View Result →

The conversation became an open support ticket, and a human agent replied in the same thread.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (7): Ada, Chatbase, CustomGPT.ai, Decagon, FS Agent (DIY control), Gorgias, Help Scout.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Escalation and human handoff

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

Agent cannot resolve the request in AI Customer Support Chatbots | AI Demos