184Board casesBoard Simulation
4,844Specialist questionsSpecialist Turn
14Model configsAcross both tasks
2.783Top Conclusion alignmentGemini 3.7 FlashBoard Simulation
3.427Top Clinical equivalenceDeepSeek-V4-ProSpecialist Turn
Conclusion alignmentHigher is better
14 models
#ModelScore
Conclusion alignment
2.783
Conclusion alignment
2.582
Conclusion alignment
2.565
Conclusion alignment
2.533
Conclusion alignment
2.527
Conclusion alignment
2.516
Conclusion alignment
2.413
Conclusion alignment
2.386
Conclusion alignment
2.201
9Ministral 3 14B Instruct 2512 BestMistral AINon-reasoningMultimodal
Conclusion alignment
2.152
10MedGemma 27B IT BestGoogleNon-reasoningMultimodal
Conclusion alignment
2.098
11Llama 4 Scout 17B-16E Instruct BestMetaNon-reasoningMultimodal
Conclusion alignment
1.929
12Meditron3 70B BestEPFL LiGHT / OpenMeditronNon-reasoningText only
Conclusion alignment
1.826
Conclusion alignment
1.435
No published models in this viewCompleted and audited configurations will appear here automatically.
EfficiencyQuality against generation cost
Mean output tokens per response, measured over the same 184 cases.
| Rank | Model configuration |
|
|
|
| 1 |
Gemini 3.7 FlashGoogle |
2.783 | 0.6407 | 0.1768 |
| 2 |
Grok 4.6xAI |
2.582 | 0.6095 | 0.1592 |
| 3 |
Qwen3.8-MaxAlibaba |
2.565 | 0.6031 | 0.1504 |
| 4 |
GPT-5.6 SolOpenAI |
2.533 | 0.6115 | 0.1487 |
| 5 |
Claude Opus 5Anthropic |
2.527 | 0.6008 | 0.1402 |
| 6 |
Gemma 4 31B ITGoogle |
2.516 | 0.6356 | 0.1810 |
| 7 |
DeepSeek-V4-FlashDeepSeek |
2.413 | 0.6206 | 0.1653 |
| 8 |
DeepSeek-V4-ProDeepSeek |
2.386 | 0.6203 | 0.1692 |
| 9 |
Ministral 3 14B Instruct 2512Mistral AI |
2.201 | 0.6230 | 0.1636 |
| 10 |
MedGemma 27B ITGoogle |
2.152 | 0.6266 | 0.1738 |
| 11 |
Llama 4 Scout 17B-16E InstructMeta |
2.098 | 0.6211 | 0.1691 |
| 12 |
Meditron3 70BEPFL LiGHT / OpenMeditron |
1.929 | 0.6103 | 0.1631 |
| 13 |
HuatuoGPT-3-32BFreedomIntelligence |
1.826 | 0.5593 | 0.0906 |
| 14 |
MedReason-8BUCSC-VLAA |
1.435 | 0.5921 | 0.1133 |
Clinical equivalenceHigher is better
9 models
#ModelScore
Clinical equivalence
3.427
Unsupported claim rate
11.8%
Clinical equivalence
3.328
Unsupported claim rate
13.7%
Clinical equivalence
3.080
Unsupported claim rate
10.0%
Clinical equivalence
3.015
Critical error rate
12.1%
Unsupported claim rate
28.7%
4Ministral 3 14B Instruct 2512 BestMistral AINon-reasoningMultimodal
Clinical equivalence
3.014
Critical error rate
11.2%
Unsupported claim rate
20.9%
5Meditron3 70B BestEPFL LiGHT / OpenMeditronNon-reasoningText only
Clinical equivalence
2.967
Critical error rate
10.2%
Unsupported claim rate
15.0%
6Llama 4 Scout 17B-16E Instruct BestMetaNon-reasoningMultimodal
Clinical equivalence
2.881
Critical error rate
13.7%
Unsupported claim rate
23.1%
7MedGemma 27B IT BestGoogleNon-reasoningMultimodal
Clinical equivalence
2.671
Critical error rate
23.1%
Unsupported claim rate
74.5%
Clinical equivalence
2.093
Critical error rate
43.5%
Unsupported claim rate
52.0%
No published models in this viewCompleted and audited configurations will appear here automatically.
EfficiencyQuality against generation cost
Mean output tokens per response, measured over the same 4,844 questions.
- surgeon1,766 q
- medical oncologist1,347 q
- radiation oncologist569 q
- radiologist407 q
- other307 q
- molecular pathologist207 q
- pathologist192 q
molecular pathologist
pathologist
molecular pathologist
radiation oncologist
molecular pathologist
other
other
surgeon
pathologist
radiation oncologist
molecular pathologist
medical oncologist
other
pathologist
molecular pathologist
pathologist
radiologist
pathologist
| Rank | Model configuration |
|
|
|
|
|
| 1 |
DeepSeek-V4-ProDeepSeek |
3.427 | 5.6% | 11.8% | 0.6155 | 0.1829 |
| 2 |
DeepSeek-V4-FlashDeepSeek |
3.328 | 6.7% | 13.7% | 0.6168 | 0.1811 |
| 3 |
Gemma 4 31B ITGoogle |
3.080 | 6.5% | 10.0% | 0.6329 | 0.1983 |
| 4 |
Ministral 3 14B Instruct 2512Mistral AI |
3.015 | 12.1% | 28.7% | 0.6077 | 0.1688 |
| 5 |
Meditron3 70BEPFL LiGHT / OpenMeditron |
3.014 | 11.2% | 20.9% | 0.6430 | 0.2171 |
| 6 |
Llama 4 Scout 17B-16E InstructMeta |
2.967 | 10.2% | 15.0% | 0.6423 | 0.2094 |
| 7 |
MedGemma 27B ITGoogle |
2.881 | 13.7% | 23.1% | 0.6216 | 0.1841 |
| 8 |
HuatuoGPT-3-32BFreedomIntelligence |
2.671 | 23.1% | 74.5% | 0.5587 | 0.1092 |
| 9 |
MedReason-8BUCSC-VLAA |
2.093 | 43.5% | 52.0% | 0.5988 | 0.1463 |