Register and share your invite link to earn from video plays and referrals.

Artificial Analysis
@ArtificialAnlys
Independent analysis of AI
Joined January 2024
665 Following    130.3K Followers
Announcing MLCR-AA, our leaderboard for Wisedocs' MLCR (Medical Long Context Reasoning) benchmark for reasoning over long medical case files. We run the hardest, held-out question tiers, with Claude Fable 5 achieving the top score of 64.4% MLCR-AA tests models on realistic synthetic medical and insurance case files built by the team at @wisedocsai, based on the record review work their platform specializes in. Our leaderboard runs the private hold-out set of 60 questions from the two hardest categories: Expert, which requires specialist medical reasoning across a full case file, and Compound, which packs several independent questions into a single query. Each question is answered against a complete case file of ~70-150 pages, and graded by a three-model judge panel on completeness and accuracy alongside a concision test limiting verbose responses compared to expert answers. Accuracy verifies that the response is well-grounded in the source documents and case context while completeness assesses whether the model produced the same essential details that were included in expert-annotated responses. The concision test verifies that models are not producing excessively verbose content (>5x the length of expert responses). Recent Claude releases from @AnthropicAI lead MLCR-AA, with Claude Fable 5 at 64.4%, and Claude Opus 5 (scoring 53.9% to 59.4% across efforts). and Kimi K3 (max) from @Kimi_Moonshot is the leading open weights model at 38.3%. Key results from MLCR-AA: ➤ Medical record review is partly achievable with AI today, but at high cost and with room to improve: the leading model (Claude Fable 5) achieves a score of 64.4% at a cost of $1 per task, the median model scores <15% ➤ Models stay faithful to source documents but miss key details required for a complete response: nearly 40% of models tested score above 80% for accuracy, the vast majority score below 50% for completeness. Models are largely right about what they do report, and omit a lot of information. GPT-5.6 Terra (max) records the highest accuracy at 93.7% and still places 10th, held back by completeness ➤ Anthropic models lead due to completeness rather than accuracy: the highest-scoring models are from Anthropic, but their accuracy is comparable to or below the strongest OpenAI models. The separation comes from covering the full scope of the expert reference answer We would like to thank Wisedocs for building MLCR and for their collaboration in bringing it to Artificial Analysis!
Show more