Scenario in benchmark · AI Customer Support Chatbots · Version 1

User explicitly asks for a human

An explicit request for a human handoff is made.

Capability: Escalation and human handoff1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the agent hands the conversation off to a human after an explicit request.
  2. Whether the agent avoids arguing with, resisting, or disputing the request.

Specific test-case inputs and grading live on the related test-case pages.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Passed · 7
BotpressPassed1 of 1 test case passedHandled the human request successfully in this scenario.View Result →

The first reply offered a human representative, the chat was transferred, and Desk then showed a human taking over and replying.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Freshdesk FreddyPassed1 of 1 test case passedHanded the user to a human without arguing.View Result →

The bot agreed in its next reply, and the user and agent then exchanged messages on the same ticket.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Intercom FinPassed1 of 1 test case passedHandled a direct request for a human.View Result →

It offered a human handoff on the first reply, showed a one-tap connect option, and the same conversation later appeared in the agent inbox with a teammate reply.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

KommunicatePassed1 of 1 test case passedHandled the request for a human by transferring the chat to a human without refusal or deflection.View Result →

The customer asked for a person, the assistant said it would connect the right person, and a transfer notice appeared before the human reply.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Respond.ioPassed1 of 1 test case passedHanded the chat to a human when the customer explicitly asked.View Result →

The agent agreed without asking for a reason or trying to answer instead.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Tidio LyroPassed1 of 1 test case passedPassed the human-handoff scenario.View Result →

The customer asked for a human, Lyro answered once with a direct transfer, and the same conversation then reached a human agent in Tidio's inbox.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Zendesk AIPassed1 of 1 test case passedHandled the request to speak to a human and passed it to a live agent.View Result →

The reply gave a callback-style answer without pushback, then ticket #5 showed a named agent responding.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (7): Ada, Chatbase, CustomGPT.ai, Decagon, FS Agent (DIY control), Gorgias, Help Scout.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Escalation and human handoff

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

User explicitly asks for a human in AI Customer Support Chatbots | AI Demos