Scenario in benchmark · AI Customer Support Chatbots · Version 1
Customer communicates in another supported language
A support question is asked in a supported non-English language.
What we check
- Whether the agent understands the request when it is written in a supported non-English language.
- Whether the agent answers in the same supported language.
- Whether the response stays grounded in the benchmark’s approved source of truth.
- Whether the agent preserves the support meaning and does not force a language switch before responding.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
BotpressPassed1 of 1 test case passedAnswered the French return-policy question in French and returned the Cedarline policy answer.View Result →
It gave the 45-day member window, the 21-day standard, the restocking and refund details, and the reply showed retrieval through a search trace, two source chips, and a source count.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Intercom FinPassed1 of 1 test case passedAnswered the French support question in French.View Result →
The reply matched the English KB on the return window, eligible condition, restocking fee, non-returnable items, refund timing, and damaged-item path, and it stayed French through the closing line.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Respond.ioPassed1 of 1 test case passedHandled the French question in French and returned the return-policy details from the knowledge base.View Result →
The reply matched the expected article facts and added nothing extra.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Zendesk AIPassed1 of 1 test case passedPassed this multilingual support case: it answered the customer's French question in French with the Help Centre's return, damage, and refund details.View Result →
Its answer matched the English articles on return timing, damaged items, and refund timing.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (10): Ada, Chatbase, CustomGPT.ai, Decagon, Freshdesk Freddy, FS Agent (DIY control), Gorgias, Help Scout, Kommunicate, Tidio Lyro.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Multilingual support
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →