RAGFlow in Managed RAG
Scenario-level performance from current published Results.
14 of 15 scenarios published
Where RAGFlow failed
2 of 14 graded scenarios. Open a Result to see the conversation and the evidence.
A bad file in the upload
RAGFlow did not flag the two unreadable uploads as document failures. A truncated PDF and a password-protected PDF received the same success status as a readable comparison document.
View Result →The retrieved sources disagree with each other
RAGFlow chose one warranty period instead of flagging conflicting sources. Its answer gave 24 months, although it had retrieved documents stating 24 and 36 months.
View Result →How RAGFlow performed
Open a capability to explore its scenarios. Each row reports the test set in its published Result.
Citations1 pass
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A claim in the answer traced back to its source | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedRAGFlow’s out-of-hours Priority 1 answer cited text supporting all four claims, including acknowledgement within 90 minutes of the ticket being raised. The retrieved table giving 30 minutes was not cited. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Data Ingestion3 pass1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A bad file in the upload | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedRAGFlow did not flag the two unreadable uploads as document failures. A truncated PDF and a password-protected PDF received the same success status as a readable comparison document. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A cleanly scanned document | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedRAGFlow recovered the requested fact from an image-only calibration certificate. Its first-ranked result contained an as-found bias of -0.037 psu, matching the reference value of -0.037 psu. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A simple, clearly formatted table | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedRAGFlow preserved the structure of a simple table on retrieval. Its first result placed the reference value, 487, at the requested row and column; it returned the table rather than a separate answer naming the value. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| An ordinary digital text document | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedRAGFlow passes this plain-text document scenario. The ingested document contains the target sentence, and the default retrieval call returns that chunk first with a clear score gap. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Grounded Generation2 pass1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The retrieved material doesn't support an answer | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedRAGFlow declined to answer when the retrieved material did not support an answer. Across four responses to a paid parental leave question, it said the information was unavailable and supplied no entitlement figure or policy. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The retrieved material supports an answer | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedRAGFlow gave a fully supported answer to the out-of-hours acknowledgement question. It stated the policy’s 90-minute target, and the cited passage supported all three claims. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The retrieved sources disagree with each other | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedRAGFlow chose one warranty period instead of flagging conflicting sources. Its answer gave 24 months, although it had retrieved documents stating 24 and 36 months. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Knowledge Sync2 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Content is removed from the source | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedRAGFlow stopped using the supplier list after it was deleted. It had previously answered the supplier question with a citation to that list; afterwards, it said the information was not found and gave no citations. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Existing content is changed | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAfter an indexed document was edited and processed again, RAGFlow returned the new value and no longer returned the old one. Asked the same lead-time question before and after the change, it returned 11 working days before and 6 working days after, and a second full cycle repeated that result. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| New content is added to the source | No published result | ||
Retrieval4 pass
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Answer is available in the knowledge base | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedRAGFlow retrieved the passage answering the knowledge-base question. The first result contained the 90-minute out-of-hours acknowledgement target; the business-hours passage ranked fifth. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Answer is not available in the knowledge base | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedRAGFlow signalled the missing answer through lower relevance scores, not an explicit no-match message. Its ten returned passages did not address paid parental leave, and the highest score was 0.2865 versus 0.4826 for an answerable question. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Retrieval is scoped to part of the documents | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedRAGFlow respected the document restriction, returning three Nordic service passages without the depot note that held the answer. The depot note ranked first in both the unrestricted search and the UK-only comparison search. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The request uses different wording than the source | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedRAGFlow retrieved the supporting passage despite the question using different wording from the source. It ranked the annual-leave passage first in both requests. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Reading these Results
Published evidence and test coverage answer different questions.
Which scenarios have a Result?
A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.
What does each Result cover?
Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.
Out of 15 scenarios
The summary counts each of the 15 scenarios once. A scenario fails when any of its published test cases failed. “No published Result yet” means nothing is public for that scenario; it does not say whether RAGFlow was tested on it.