Methodology

We don't run our own scores. Every number comes from a public benchmark, linked on each page with its date. We pick, summarize and explain.

Sources

• Norwegian, German, Dutch, Spanish, Portuguese: EuroEval generative leaderboards (Alexandra Institute; formerly ScandEval). Retrieved 2026-10-15.
• Mandarin: SuperCLUE general leaderboard, Jul 2026 edition.
• Korean: Horangi 4 (W&B Korea), via AIB, as of 2026-08-03; plus KMMLU.
• Norwegian extra: NorEval (University of Oslo) for reference.

How we rank (EuroEval languages)

• Overall: EuroEval's own rank score (lower is better), which aggregates all tasks.
• Writing: mean primary score of summarization and linguistic-acceptability (grammar) datasets, plus simplification where available.
• Understanding: mean primary score of reading comprehension, knowledge and common-sense datasets.
• Models count for a category only if they have results on every dataset in it. Scores tagged "val" are on validation splits; "zero-shot" means no examples given.

Known gaps

• Translation: no public per-model, per-language leaderboard covering all 7 languages with current models. WMT26 and Intento publish findings, but not as a matching per-language ranking.
• Agent / tool-use tasks: EuroEval's chat leaderboards include tool use, but that data isn't exposed in a form we can reuse yet. SuperCLUE's total includes agentic coding and task planning.
• Mandarin: SuperCLUE ranks Chinese-developed models only. Korean: the newest frontier models aren't on Horangi 4 yet.
• Scores aren't comparable across languages.

Updates

Monthly: python3 scripts/fetch_euroeval.py refreshes EuroEval; Mandarin and Korean are edited by hand in data/external.json; then python3 scripts/build.py.