Extend AI in Structured Document Extraction
Scenario-level performance from current published Results.
17 of 20 scenarios published
Where Extend AI failed
3 of 17 graded scenarios. Open a Result to see the conversation and the evidence.
Aggregate across documents
Extend AI returned £7,838.04 instead of the required £9,592.44 when asked to total three invoices. The shortfall equals one invoice’s £1,754.40 contribution.
View Result →Filter records across documents
Extend AI offered no way to filter records across the three extracted receipts. It returned separate document extractions or one merged extraction, not a filtered set.
View Result →The answer was never in the schema
Extend AI offered no way to ask about a document fact outside the extraction schema. The extraction returned the invoice reference and total due, without an answer about the printed due date.
View Result →How Extend AI performed
Open a capability to explore its scenarios. Each row reports the test set in its published Result.
Field Extraction5 pass
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The field is absent | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI passes this absent-field case: it returned null for the added field and no grounding to an invented value. The receipt contains no loyalty or membership data, and the response carries an empty citations array. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The same field across layouts | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI returned the correct value for the same field across documents laid out differently. The shown runs returned $90.10, $227.33, and USD 157.50, each matching its source document. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The value is directly available | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI passed the directly available value scenario. It returned the receipt number once under its label, and the JSON result matched that value as a string. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The value must be derived | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI derived a fuel cost that was not stated in the invoice. It returned £81.00, the required result of multiplying 450 miles by £0.18 per mile. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The value needs a supplied definition | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI selected the consignment reference required by the supplied definition, not the more prominent invoice number. It cited the consignment line on page 1. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Nested & Repeated Fields3 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A repeated set of records | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI returned the three line items on the invoice page as three records, each with its own description, item code, quantity, unit price and amount, with nothing dropped, duplicated or placed on the wrong record. The subtotal, VAT and total due lines below the table were not returned as records. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The set contains a non-record | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI extracted the genuine invoice items without including the total row. It returned five items from five genuine source rows, with none missing or substituted. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The set is legitimately empty | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI returned an empty set for the legitimately empty document. The extracted set of damage or exception entries is empty, and the source references are empty too. Page 3 says no damage or exceptions were reported. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The set continues across a page break | No published result | ||
OCR1 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A document mixing digital and scanned pages | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI returned the requested values from a document mixing digital and scanned pages. The decision-relevant finding is that the image-only page was processed rather than skipped: the scanned-page values and the digital-page values both came back correct in one extraction call. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A cleanly scanned document | No published result | ||
Querying3 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Aggregate across documents | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedExtend AI returned £7,838.04 instead of the required £9,592.44 when asked to total three invoices. The shortfall equals one invoice’s £1,754.40 contribution. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Filter records across documents | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedExtend AI offered no way to filter records across the three extracted receipts. It returned separate document extractions or one merged extraction, not a filtered set. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The answer was never in the schema | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedExtend AI offered no way to ask about a document fact outside the extraction schema. The extraction returned the invoice reference and total due, without an answer about the printed due date. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Review and Correction3 pass
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Correct a field value | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI saved a corrected receipt number to the stored record: after RHS-70234 was changed to RHS-99999 on its review screen, saved and the run reloaded, the stored record read RHS-99999. The subtotal, tax and total due stayed at 85, 5.1 and 90.1. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Corrected data goes downstream | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI delivered the corrected receipt number through its API. The subtotal, tax and total due stayed at $85.00, $5.10 and $90.10. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Repair a record | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI let a user add and remove invoice items through Human Review. After approval, the traffic cones were present, the site fencing was absent and the telehandler and generator values were unchanged. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Source Grounding2 pass
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The field is absent | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI passes this absent-field case: it returned null for the added field and no grounding to an invented value. The receipt contains no loyalty or membership data, and the response carries an empty citations array. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Trace a value to its location | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedExtend AI traced the returned value to the correct document location. It returned 6103 with a citation to page 2 on "Subtotal" beside £6,103.00. The PDF shows "Subtotal" once on page 2 and not on page 1. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Training1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A field the tool can only get right from examples | No published result | ||
Reading these Results
Published evidence and test coverage answer different questions.
Which scenarios have a Result?
A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.
What does each Result cover?
Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.
Out of 19 scenarios
The summary counts each of the 19 scenarios once. A scenario fails when any of its published test cases failed. “No published Result yet” means nothing is public for that scenario; it does not say whether Extend AI was tested on it.