nightly-eval · sim-small-v1 simulated
Completed dataset /home/evalpipe/demo_dataset.jsonl started 2026-07-13 01:08:35 UTC run id 877dfc4addf140f88640d4a106afe254 · Download CSV · JSON
Full results for this single run: the headline scores, how answers were spread out, where the model was strong or weak, and every question with the model's answer.
Score distribution
How many questions landed in each quality band (0 = wrong, 1 = perfect). Most bars bunched on the right means the model answered well overall.
View data table
| Score bin | Items |
|---|---|
| 0.0-0.1 | 0 |
| 0.1-0.2 | 0 |
| 0.2-0.3 | 0 |
| 0.3-0.4 | 4 |
| 0.4-0.5 | 7 |
| 0.5-0.6 | 0 |
| 0.6-0.7 | 10 |
| 0.7-0.8 | 3 |
| 0.8-0.9 | 0 |
| 0.9-1.0 | 16 |
Mean score by evaluator
How each type of check graded this run (0–1). A short bar flags the quality dimension the model is weakest on.
View data table
| Evaluator | Mean score |
|---|---|
| exact_match | 0.400 |
| llm_judge_correctness | 0.880 |
| safety | 1.000 |
| semantic_similarity | 0.637 |
| token_f1 | 0.662 |
Where it's strong or weak
Pass rate split by difficulty, so a good overall score can't hide a weak spot (e.g. "95% on easy, 20% on hard").
View data table
| Difficulty | Items | Pass rate | 95% CI | Mean score |
|---|---|---|---|---|
| hard | 1 | 0.0% | 0.0–79.3% | 0.380 |
| medium | 17 | 35.3% | 17.3–58.7% | 0.700 |
| easy | 22 | 40.9% | 23.3–61.3% | 0.743 |
Every question
Each question with its outcome, score, and the model's answer. Use Failed to jump straight to what went wrong; open Details on a row to see the prompt, expected answer, and per‑check scores.
| Item | Outcome | Score | Latency | Output | Details |
|---|---|---|---|---|---|
| qa-002 | Pass | 1.000 | 16 ms | Water boils at 100 degrees Celsius at sea level. |
DetailsPromptAnswer the question accurately and concisely. Question: What is the boiling point of water at sea level in Celsius? ExpectedWater boils at 100 degrees Celsius at sea level. Model outputWater boils at 100 degrees Celsius at sea level.
|
| qa-008 | Pass | 1.000 | 23 ms | The chemical symbol for gold is Au. |
DetailsPromptAnswer the question accurately and concisely. Question: What is the chemical symbol for gold? ExpectedThe chemical symbol for gold is Au. Model outputThe chemical symbol for gold is Au.
|
| qa-010 | Pass | 0.940 | 20 ms | The smallest prime number is 2. |
DetailsPromptAnswer the question accurately and concisely. Question: What is the smallest prime number? ExpectedThe smallest prime number is 2. Model outputThe smallest prime number is 2.
|
| qa-011 | Pass | 0.980 | 36 ms | The Mona Lisa was painted by Leonardo da Vinci. |
DetailsPromptAnswer the question accurately and concisely. Question: Who painted the Mona Lisa? ExpectedThe Mona Lisa was painted by Leonardo da Vinci. Model outputThe Mona Lisa was painted by Leonardo da Vinci.
|
| qa-013 | Pass | 0.960 | 38 ms | A hexagon has six sides. |
DetailsPromptAnswer the question accurately and concisely. Question: How many sides does a hexagon have? ExpectedA hexagon has six sides. Model outputA hexagon has six sides.
|
| qa-016 | Pass | 0.960 | 34 ms | The currency of Japan is the yen. |
DetailsPromptAnswer the question accurately and concisely. Question: What is the currency of Japan? ExpectedThe currency of Japan is the yen. Model outputThe currency of Japan is the yen.
|
| qa-017 | Pass | 1.000 | 22 ms | Albert Einstein developed the theory of general relativity. |
DetailsPromptAnswer the question accurately and concisely. Question: Who developed the theory of general relativity? ExpectedAlbert Einstein developed the theory of general relativity. Model outputAlbert Einstein developed the theory of general relativity.
|
| qa-020 | Pass | 1.000 | 19 ms | Portuguese is the primary language spoken in Brazil. |
DetailsPromptAnswer the question accurately and concisely. Question: What is the primary language spoken in Brazil? ExpectedPortuguese is the primary language spoken in Brazil. Model outputPortuguese is the primary language spoken in Brazil.
|
| qa-022 | Pass | 0.960 | 23 ms | The square root of 144 is 12. |
DetailsPromptAnswer the question accurately and concisely. Question: What is the square root of 144? ExpectedThe square root of 144 is 12. Model outputThe square root of 144 is 12.
|
| qa-029 | Pass | 0.960 | 23 ms | The capital of Canada is Ottawa. |
DetailsPromptAnswer the question accurately and concisely. Question: What is the capital of Canada? ExpectedThe capital of Canada is Ottawa. Model outputThe capital of Canada is Ottawa.
|
| qa-030 | Pass | 1.000 | 24 ms | Mercury is liquid at room temperature. |
DetailsPromptAnswer the question accurately and concisely. Question: Which metal is liquid at room temperature? ExpectedMercury is liquid at room temperature. Model outputMercury is liquid at room temperature.
|
| qa-032 | Pass | 0.980 | 35 ms | A soccer team has 11 players on the field. |
DetailsPromptAnswer the question accurately and concisely. Question: How many players are on a soccer team on the field? ExpectedA soccer team has 11 players on the field. Model outputA soccer team has 11 players on the field.
|
| qa-033 | Pass | 0.980 | 23 ms | The cheetah is the fastest land animal. |
DetailsPromptAnswer the question accurately and concisely. Question: What is the fastest land animal? ExpectedThe cheetah is the fastest land animal. Model outputThe cheetah is the fastest land animal.
|
| qa-034 | Pass | 0.980 | 30 ms | The Indian Ocean lies between Africa and Australia. |
DetailsPromptAnswer the question accurately and concisely. Question: Which ocean lies between Africa and Australia? ExpectedThe Indian Ocean lies between Africa and Australia. Model outputThe Indian Ocean lies between Africa and Australia.
|
| qa-036 | Pass | 0.960 | 34 ms | George Washington was the first President of the United States. |
DetailsPromptAnswer the question accurately and concisely. Question: Who was the first President of the United States? ExpectedGeorge Washington was the first President of the United States. Model outputGeorge Washington was the first President of the United States.
|