AI Demos Research — the structured-intelligence platform. Every verdict on these pages opens to the execution behind it.
Graded 17 November 2026

Does Zendesk AI answer questions from the knowledge base?

Zendesk AI answered the shipping, returns and damaged-item questions in line with the knowledge base. It gave Canada's free-shipping threshold as over $120 and return windows of 21 days from delivery, extended to 45 days for Plus members. For a damaged item, it offered an in-stock replacement or a full refund including original shipping, with photos required within 7 days.

3 of 3 test cases passed

Every test case in this scenario has a result.

Pass rate100%3 of 3 gradable
Coverage3 of 3test cases with a result
3PassDid everything it was expected to do.
0FailNo test case failed.
0Not gradableEvery result here could be graded.
0UntestedEvery test case in this scenario has a result.
The pass rate is a summary. The evidence is the 3 test cases below: what we sent, what we checked, what the tool returned, and the proof.

The 3 test cases

Each test case is judged on its own: Pass, Fail, Not gradable, or Untested. The scenario result above is based on these test cases.

Free-shipping threshold asked for CanadaPassEvidence
What we sent

“I'm in Canada — how much do I need to spend to get free shipping?”

What the tool returned

Progress update Thank you for your patience while I locate that information. Answer In Canada, standard shipping is free on orders over $120. Cedarline Plus members receive free standard shipping on every order, regardless of order size.

Transcribed from Proof 1 · Screenshot

Expected vs. Found
✓Expected: State that Canadian orders qualify for free shipping when spending is over $120.Found: Both the answer and the Canadian shipping requirement put free standard shipping above $120.
✓Must not appear: Do not offer the US threshold of $75 or another incorrect figure as Canada's free-shipping threshold.Found: It did not substitute the US figure of $75 or offer an alternative amount as Canada's threshold.
Supporting proof
Proof 1Screenshot
Chat screenshot: Canada's free-shipping answer and Plus-member exemption
Open original ↗
Why this result

The answer uses the Canada-specific threshold and preserves the Plus-member exemption without substituting another region's figure, meeting the shipping policy's requirements.

Tested by Ajay Vekhande · evidence dated 23 September 2026
Return window questionPassEvidence
What we sent

“What is your return window?”

What the tool returned

The standard return window is 21 days from delivery. Cedarline Plus members receive 45 days. Items must be resalable and returned in their original packaging where possible.

Transcribed from Proof 1 · Screenshot

Every message the tool sent, in order (2)
  1. Progress update

    Thank you for your patience while I locate that information.

  2. Answer

    The final response, printed above.

Expected vs. Found
✓Expected: State a standard return window of 21 days from delivery.Found: Both the answer and return policy give 21 days from delivery.
✓Expected: State that Cedarline Plus members have 45 days from delivery to return items.Found: The answer gives Plus members 45 days from delivery, the extended window specified by the return policy.
✓Must not appear: Give no unsupported return terms; any item-condition clause must retain the policy's meaning.Found: It required resalable items and original packaging where possible, without extra or softened conditions or an order-date deadline.
Supporting proof
Proof 1Screenshot
Chat screenshot: standard and Plus-member return windows with item conditions
Open original ↗
Why this result

The answer supplies both policy-defined return windows and retains the delivery-date anchor and item-condition clause, with no unsupported additions.

Tested by Ajay Vekhande · evidence dated 23 September 2026
Damaged-on-arrival options with their conditionPassEvidence
What we sent

“My rug arrived damaged. What are my options?”

What the tool returned

I'm sorry your rug arrived damaged. Please report it within 7 days, including photos of the item and packaging. The normal return window and restocking fee do not apply. - Replacement, if in stock - Full refund, including original shipping. Refunds follow warehouse inspection and are typically issued within 5-7 business days of receipt.

Transcribed from Proof 1 · Screenshot

Every message the tool sent, in order (2)
  1. Progress update

    Thank you for your patience while I review the available options.

  2. Answer

    The final response, printed above.

Expected vs. Found
✓Expected: Offer an in-stock replacement or a full refund that includes original shipping.Found: It offered replacement subject to stock availability, or a full refund covering original shipping, as the policy provides.
✓Expected: Require photos of the damage within 7 days.Found: It asked for a report within 7 days with photos of the item and packaging.
✓Must not appear: Give no damage-policy condition, deadline or remedy beyond those in the knowledge base.Found: The policy supports the exemptions from normal return limits and restocking fees, plus refunds after warehouse inspection, typically within 5-7 business days of receipt.
Supporting proof
Proof 1Screenshot
Chat screenshot: damaged-item remedies, reporting deadline and refund conditions
Open original ↗
Why this result

The applicable knowledge-base policy supports the remedies and conditions it gave, so the answer meets the damaged-item requirements rather than introducing unsupported terms.

Tested by Ajay Vekhande · evidence dated 23 September 2026

Configuration and setup

How this tool was set up for the run and what the test needed in place. Each row is a fact from the run's records; a fact the records do not hold is left out, not guessed.

Software that produced the output
Zendesk AI
Build or version
Not exposed by the hosted product
Plan or tier
Suite – Professional (Free trial)
Surface
Web app
Tested
23 September 2026 · Ajay Vekhande
Connected to
Knowledge BaseCedarline knowledge base · version 2Test cases: Free-shipping threshold asked for Canada, Damaged-on-arrival options with their condition
Knowledge BaseCedarline knowledge base · version 1Test cases: Return window question

How this scenario is graded

How we decide Pass, Fail and Not gradable. The same rules apply to every tool tested on this scenario.

How results are decided

Each test case gets one result per tool: Pass, Fail or Not gradable. A test case we haven't run yet shows Untested. There are no partial results.

The rules
  • Pass: the tool did everything the test expected, and nothing it said contradicts the correct answer.
  • Fail: at least one expected behaviour clearly didn't happen; the test case's result says which one and quotes the tool.
  • Not gradable: our evidence couldn't settle the outcome (for example a record we needed is missing). It is never counted as a fail, and the test case's result says what's missing.

Where this sits in the benchmark

This page is one cell of a larger study: one tool, one scenario. Only this benchmark's frame appears here.

History of this result

What has happened to this result since it was first published. Runs and grades are never overwritten: a retest or a re-grade publishes a new result and keeps the earlier one readable.

from the publication record
18 November 2026First publishedAI Customer Support Chatbots v1

Act on this result

Nothing filed here edits the run or the grade. A challenge opens a review, and a review can produce a new run or a re-grade — which becomes the current result and leaves this one in the history.

This matches what I see

You run the same kind of test against your own setup and get the same behaviour.

Agree →
This does not match

Yours behaves differently. Tell us what you got, with a screenshot if you have one.

Disagree →
Point out an issue

Something here is wrong — a reference value, a transcription, a grade.

Report an issue →
Request a retest

On a newer build, a larger dataset, or your own setup.

Request a retest →
We have fixed this

Tell us what changed and we schedule a rerun of the failing test case. The old result stays as history.

Vendor notice →
Filed against this evidenceNothing yet. Challenges, counter-evidence and fix notices appear here with their outcome, and stay on the page after they are resolved.
Cite this result
aidemos.com/benchmarks/ai-customer-support-chatbots/results/zendesk/answer-is-available-in-the-knowledge-base · 3 pass · 0 fail · coverage 3/3 · graded 2026-11-17

The same record is available as structured data through the AI Demos MCP server, with the counts, the coverage and every per-test-case reason carried as fields.

Verify the proof files

These files support this result. Open a file to inspect the original evidence.

File fingerprints (SHA-256)

A fingerprint identifies the exact file used for this result.

Free-shipping threshold asked for Canada · Proof 1 · Screenshote9e0efec67eed69f942ffd1b2276e08fef7c63655520878d79af0624c1540649
Return window question · Proof 1 · Screenshotacea320d18b3755c1bf73a068203468ae4485162b608d63629d3182952d5610a
Damaged-on-arrival options with their condition · Proof 1 · Screenshot9c49a2b906a43578d4ef1506abcffd1d17edb76fde80c6273b8fcc9fabe434ec