Scenario in benchmark · AI Customer Support Chatbots · Version 1

Answer is not available in the knowledge base

The requested information is absent from the source set.

Capability: Knowledge-grounded answering3 pinned test cases per toolNeutral global definition →

What we check

  1. Whether it indicates that the information cannot be found in the source set.
  2. Whether it avoids inventing an answer when the source set is missing the fact.
  3. Whether it responds with an honest refusal or equivalent no-answer behaviour.

Specific test-case inputs and grading live on the related test-case pages.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Different test set · 7
BotpressDifferent test setAnswered one no-answer case correctly, failed one by mixing in domestic return terms, and passed one by escalating without inventing service details.View Result →

It said there was no free-shipping threshold for Mexico, gave no Mexico number, and for Ireland returns said the policy did not specify who pays but also stated the 45-day versus 21-day window. For furniture assembly or installation, it handed the question to a human.

Published on a different test set, so its counts are not compared here.

Freshdesk FreddyDifferent test setPassed all three knowledge-base checks.View Result →

For Mexico, it said no free-shipping threshold was available and gave no Mexico figure. For Ireland, it said it had no Ireland-specific return shipping option and did not offer the US and Canada return terms as the answer. For delivery, it said furniture assembly or installation was not offered.

Published on a different test set, so its counts are not compared here.

Intercom FinDifferent test setPassed all three checks.View Result →

It said the answer was unavailable in each case and did not invent one: Mexico had no free-shipping figure, Ireland had no return-shipping terms, and assembly or installation was not included with delivery.

Published on a different test set, so its counts are not compared here.

KommunicateDifferent test setHandled the Mexico and Ireland questions without inventing policy details, but it did not state that it lacked assembly or installation information.View Result →

Mexico was marked outside shipping coverage with no threshold figure, and Ireland got no international return terms or US/Canada returns answer.

Published on a different test set, so its counts are not compared here.

Respond.ioDifferent test setStated when it could not answer questions outside its knowledge base.View Result →

It gave no invented shipping threshold for Mexico, international return terms for Ireland, or assembly or installation terms.

Published on a different test set, so its counts are not compared here.

Tidio LyroDifferent test setAcknowledged it could not answer an Ireland returns question or a furniture assembly question from its information, but for a Mexico shipping question it declared Mexico unsupported instead of saying it had no Mexico information.View Result →

It gave no Mexico shipping figure, no international return terms and no assembly terms.

Published on a different test set, so its counts are not compared here.

Zendesk AIDifferent test setHandled all three tested questions by saying the knowledge base lacked the requested information and by not inventing answers.View Result →

It said Mexico shipping information was missing, could not confirm Ireland return-shipping policy, and said assembly or installation was not indicated.

Published on a different test set, so its counts are not compared here.

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (7): Ada, Chatbase, CustomGPT.ai, Decagon, FS Agent (DIY control), Gorgias, Help Scout.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
3
Disclosure
0 public · 3 withheld
Capabilities exercised here
Knowledge-grounded answering

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 pins a scenario rubric for this scenario. Benchmark methodology →

Answer is not available in the knowledge base in AI Customer Support Chatbots | AI Demos