Egyptian and Nigerian English Join the Open ASR Leaderboard
Hugging Face announced a collaboration with Liva AI to add private conversational speech datasets for Egyptian and Nigerian English accents to the Open ASR Leaderboard. The splits contain 1.34 hours of Egyptian English and 1.55 hours of Nigerian English, with spontaneous speech and punctuated, cased transcriptions that retain disfluencies. The leaderboard will report accents grouped as US versus non-US, and as Kachru’s Inner Circle versus Outer and Expanding Circles.
Hugging Face says the additional private data broadens accent representation in ASR evaluations and is intended to make evaluations more trustworthy by reducing benchmark gaming. The new accent groupings also provide a more detailed view of the leaderboard’s coverage.
A few months ago, we added private data to the Open ASR Leaderboard: 11 splits from Appen and DataOceanAI covering scripted and conversational speech from five accents (blog). The community response was great, with many highlighting the importance of held-out sets for trustworthy evaluations. Since then, we added one more private split for Indian-accented English with Voice Arena (blog).
We’re excited to announce a new collaboration with Liva AI to add even more private data with two new accents: Egyptian 🇪🇬 and Nigerian 🇳🇬.
| Dataset | Accent | Duration [h] | Style | Transcription |
|---|---|---|---|---|
| Liva AI Conversational EG | Egyptian | 1.34 | Conversational, spontaneous | Punctuated, cased, with disfluencies. |
| Liva AI Conversational NG | Nigerian | 1.55 | Conversational, spontaneous | Punctuated, cased, with disfluencies. |
This is part of our continued effort to improve representation in ASR evals (more accents and more languages) and to improve trust (new private data to minimize benchmaxxing, and new data providers). We’re excited to see the community continuing to push ASR performance for the world 🤗