Edcafe AI in Quiz Generation
Scenario-level performance from current published Results.
13 of 16 scenarios published
Where Edcafe AI failed
8 of 13 graded scenarios. Open a Result to see the conversation and the evidence.
The learner's answer is partly correct
Edcafe AI recognised the learner’s answer as partly correct, but its personalised feedback did not explain the missing idea. It credited the series-circuit explanation and identified the omitted parallel case; the explanation of that case appeared in a stored model answer, not in the personalised feedback.
View Result →The learner's answer is partly correct
Edcafe AI recognised the learner’s answer as partly correct, but its personalised feedback did not explain the missing idea. It credited the series-circuit explanation and identified the omitted parallel case; the explanation of that case appeared in a stored model answer, not in the personalised feedback.
View Result →The learner checks whether they are improving
Edcafe AI did not give the learner a way to track improvement across attempts. The improved attempt replaced the earlier result rather than creating a history, and the teacher report kept only the first submission.
View Result →One topic is weaker than the others
Edcafe AI did not turn opposite learner histories into personalized practice. It correctly identified opposite weak topics for the two learners, but the finished result cards only offered Restart and the report surface had no practice control.
View Result →The learner returns to the material later
Edcafe AI served a returning learner the same blank quiz rather than practice tailored to earlier weaknesses. It did not refer to the earlier attempt or offer a practice recommendation.
View Result →A teacher gives the quiz to a class
Edcafe AI did not fully meet the requirements for giving a quiz to a class. Three learners completed the quiz through Google Classroom and had separate submissions, but the report did not identify their class or Google accounts.
View Result →A cleanly scanned document
Edcafe AI produced no quiz questions from the scanned chapter. It reported that it could not extract text from the PDF.
View Result →Important information appears in both text and a diagram
Edcafe AI covered text ideas but omitted core information taught only through the diagram. Its 30-question quiz never asked who eats whom in the food web.
View Result →How Edcafe AI performed
Open a capability to explore its scenarios. Each row reports the test set in its published Result.
Answer Feedback1 pass1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The learner gives a clearly wrong answer | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedEdcafe AI passed this scenario. The learner-facing feedback said the answer was incorrect, gave the rule that current stays the same in a series circuit, and addressed the stated misunderstanding rather than only showing the answer. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The learner's answer is partly correct | 1 Pass1 Fail0 Not gradable 1 of 2 test cases passed · 1 failedWhat happenedEdcafe AI recognised the learner’s answer as partly correct, but its personalised feedback did not explain the missing idea. It credited the series-circuit explanation and identified the omitted parallel case; the explanation of that case appeared in a stored model answer, not in the personalised feedback. | 2/2 assessed2/2 gradablePublished test set | View Result → |
Class Reports1 pass
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The teacher checks what the class struggled with | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedEdcafe AI made the class's shared weakness in parallel circuits visible alongside stronger performance on series circuits. Answer-level feedback flagged off-topic parallel responses, and learner summaries pointed to parallel circuits as the area to improve. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Grading2 pass1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The learner answers correctly in their own words | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedEdcafe AI passes this own-words meaning-grading case. It marked Learner D Excellent Understanding Demonstrated, and the report showed a 0% average across four submissions while three learners were labelled excellent, so the average does not reflect this answer's wording. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The learner answers some fixed-answer questions correctly and others incorrectly | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedEdcafe AI accurately graded the mixed-answer attempt, marking six answers correct and four incorrect. The learner result and teacher report both gave 60% (6/10), the total required by the answer key. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The learner's answer is partly correct | 1 Pass1 Fail0 Not gradable 1 of 2 test cases passed · 1 failedWhat happenedEdcafe AI recognised the learner’s answer as partly correct, but its personalised feedback did not explain the missing idea. It credited the series-circuit explanation and identified the omitted parallel case; the explanation of that case appeared in a stored model answer, not in the personalised feedback. | 2/2 assessed2/2 gradablePublished test set | View Result → |
Learner Progress1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The learner checks whether they are improving | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedEdcafe AI did not give the learner a way to track improvement across attempts. The improved attempt replaced the earlier result rather than creating a history, and the teacher report kept only the first submission. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Personalized Practice2 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| One topic is weaker than the others | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedEdcafe AI did not turn opposite learner histories into personalized practice. It correctly identified opposite weak topics for the two learners, but the finished result cards only offered Restart and the report surface had no practice control. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The learner returns to the material later | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedEdcafe AI served a returning learner the same blank quiz rather than practice tailored to earlier weaknesses. It did not refer to the earlier attempt or offer a practice recommendation. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Quiz Delivery1 pass1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A learner takes a quiz on their own | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedEdcafe AI passes this scenario. One learner completed the quiz, submitted once, and the teacher report saved that attempt. Before submission, the quiz showed no answer information; after submission, the result screen showed the score and review. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A teacher gives the quiz to a class | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedEdcafe AI did not fully meet the requirements for giving a quiz to a class. Three learners completed the quiz through Google Classroom and had separate submissions, but the report did not identify their class or Google accounts. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Quiz Generation2 fail3 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A cleanly scanned document | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedEdcafe AI produced no quiz questions from the scanned chapter. It reported that it could not extract text from the PDF. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Important information appears in both text and a diagram | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedEdcafe AI covered text ideas but omitted core information taught only through the diagram. Its 30-question quiz never asked who eats whom in the food web. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The user asks the quiz to test a particular learning goal | No published result | ||
| The user turns a recorded lecture into a quiz | No published result | ||
| The user turns study material into a quiz | No published result | ||
Reading these Results
Published evidence and test coverage answer different questions.
Which scenarios have a Result?
A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.
What does each Result cover?
Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.
Out of 15 scenarios
The summary counts each of the 15 scenarios once. A scenario fails when any of its published test cases failed. “No published Result yet” means nothing is public for that scenario; it does not say whether Edcafe AI was tested on it.