Arabic Arena
A speech-to-text leaderboard scored on many test sets at once: MSA, Gulf, Egyptian, Levantine, Maghrebi, code-switching, noisy speech, and simulated 8 kHz telephone and call-center channels. One test set can mislead; the average of many is harder to game. Lower is better.
Click a column to sort; the best value in each column is highlighted. Small grey numbers are 95 % bootstrap intervals (WER) and, under the rank, the rank range. Values in parentheses are the number of sets finished so far. Closed APIs and fine-tunes with gated weights are unranked references, scored on the re-hosted sets only. † marks models trained on the train split of one of the benchmark corpora, so their score there is not zero-shot (hover for details). Halluc. % is the share of noise-only clips for which the model wrote any words; it is not part of the Arena score.
Per-set results
Charts
Arena WER
Lower is better.
Accuracy vs speed
RTFx on one RTX 5070 Ti (log scale). Down and to the right is better.
Phone-channel robustness
WER on the same source sets: clean audio, then the telephony channel, then the call-center condition.
Where models break
Hallucination on noise-only audio
Share of 300 clips with no target speech (line silence, hold hiss, mains hum, distant babble, office, street and transport noise; half sent through a phone codec) for which the model still wrote words. Lower is better. Not part of the Arena score.
How scores are computed
- Per set: corpus WER and CER, meaning total errors over total reference words or characters, after the same normalization is applied to references and hypotheses (Arabic: diacritics, alef/hamza/ta-marbuta spellings and punctuation are unified and digits are written out as words).
- Per track: the unweighted mean over that track's sets.
- Arena: .
- Uncertainty: 95% intervals from a paired bootstrap over utterances (1000 resamples per set, the same resample for every model). The rank range is how far a model's Arena rank moves across resamples; models whose ranges overlap are not reliably separated.
- Telephony: 8 kHz, a 300–3400 Hz band, a G.711, GSM, AMR-NB, G.726 or Opus codec, and packet loss. Call-center adds babble, office or street noise and reverb at the caller before that channel.
- Decoding: greedy, language forced to the suite language, batch inference on one RTX 5070 Ti (16 GB).
- Models: open-weight models are ranked, all run locally; closed APIs and fine-tunes with gated weights are listed as unranked references.