Tool in benchmark · Version 1

Datalab in Structured Document Extraction

Scenario-level performance from current published Results.

6Passed
0Failed
14No published Result yet

6 of 20 scenarios published

CapabilityScenariosResult
Field Extraction2 pass · 0 fail · 2 of 5 published
Nested & Repeated Fields2 pass · 0 fail · 2 of 4 published
OCR2 pass · 0 fail · 2 of 2 published
Querying0 of 3 published
Review and Correction0 of 3 published
Source Grounding0 of 2 published
Training0 of 1 published
PassedFailedPublished, different test setNo published Result yet

Datalab passed every graded scenario it has a published Result for.

How Datalab performed

Open a capability to explore its scenarios. Each row reports the test set in its published Result.

Field Extraction2 pass3 no Result yet
ScenarioPublished outcomesTest coverageResult
The same field across layouts
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

Datalab returned the correct identifier, subtotal, tax and total due from all three differently laid-out receipts, with the same four-field schema. That included a receipt whose number was relabelled as a work order number and moved to a top-left box, returned as GLR-98213 with $227.33 due, and a single-column receipt with USD amounts, returned as SPD-2026-0714 with USD 157.50 due, each equal to the printed value.

1/1 assessed1/1 gradablePublished test setView Result →
The value must be derived
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

Datalab calculated a fuel cost that was not printed in the invoice. It returned £81.00 from 450 miles at £0.18 per mile, the exact result of multiplying the two values.

1/1 assessed1/1 gradablePublished test setView Result →
The field is absentNo published result
The value is directly availableNo published result
The value needs a supplied definitionNo published result
Nested & Repeated Fields2 pass2 no Result yet
ScenarioPublished outcomesTest coverageResult
A repeated set of records
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

Datalab extracted the repeated set without mixing values between items. All three invoice items were returned, with nothing dropped or duplicated.

1/1 assessed1/1 gradablePublished test setView Result →
The set continues across a page break
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

Datalab returned the invoice's ten line items as one list, six from the first page and four from the second, with every description, quantity, unit price and amount as printed and no item repeated at the page break.

1/1 assessed1/1 gradablePublished test setView Result →
The set contains a non-recordNo published result
The set is legitimately emptyNo published result
OCR2 pass
ScenarioPublished outcomesTest coverageResult
A cleanly scanned document
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

Datalab extracted accurate invoice values from the image-only scan. The scan and digital original both returned the source invoice's £1,754.40 total due and the same three line items.

1/1 assessed1/1 gradablePublished test setView Result →
A document mixing digital and scanned pages
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

Datalab returned all five requested values from the three-page invoice, including the vehicle registration and the recipient's name that appear only on its scanned page. Each value matched the document and was cited to the page it sits on, and the total due kept its pound sign.

1/1 assessed1/1 gradablePublished test setView Result →
Querying3 no Result yet
ScenarioPublished outcomesTest coverageResult
Aggregate across documentsNo published result
Filter records across documentsNo published result
The answer was never in the schemaNo published result
Review and Correction3 no Result yet
ScenarioPublished outcomesTest coverageResult
Correct a field valueNo published result
Corrected data goes downstreamNo published result
Repair a recordNo published result
Source Grounding2 no Result yet
ScenarioPublished outcomesTest coverageResult
The field is absentNo published result
Trace a value to its locationNo published result
Training1 no Result yet
ScenarioPublished outcomesTest coverageResult
A field the tool can only get right from examplesNo published result

Reading these Results

Published evidence and test coverage answer different questions.

Publication availability

Which scenarios have a Result?

A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.

Test coverage

What does each Result cover?

Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.

Scenario scope

Out of 19 scenarios

The summary counts each of the 19 scenarios once. A scenario fails when any of its published test cases failed. “No published Result yet” means nothing is public for that scenario; it does not say whether Datalab was tested on it.

Datalab in Structured Document Extraction | AI Demos