Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Egyptian and Nigerian English Join the Open ASR Leaderboard

AI summary

Hugging Face announced a collaboration with Liva AI to add private conversational speech datasets for Egyptian and Nigerian English accents to the Open ASR Leaderboard. The splits contain 1.34 hours of Egyptian English and 1.55 hours of Nigerian English, with spontaneous speech and punctuated, cased transcriptions that retain disfluencies. The leaderboard will report accents grouped as US versus non-US, and as Kachru’s Inner Circle versus Outer and Expanding Circles.

In-depth insight

Hugging Face says the additional private data broadens accent representation in ASR evaluations and is intended to make evaluations more trustworthy by reducing benchmark gaming. The new accent groupings also provide a more detailed view of the leaderboard’s coverage.

Back to Articles

A few months ago, we added private data to the Open ASR Leaderboard: 11 splits from Appen and DataOceanAI covering scripted and conversational speech from five accents (blog). The community response was great, with many highlighting the importance of held-out sets for trustworthy evaluations. Since then, we added one more private split for Indian-accented English with Voice Arena (blog).

We’re excited to announce a new collaboration with Liva AI to add even more private data with two new accents: Egyptian 🇪🇬 and Nigerian 🇳🇬.

DatasetAccentDuration [h]StyleTranscription
Liva AI Conversational EGEgyptian1.34Conversational, spontaneousPunctuated, cased, with disfluencies.
Liva AI Conversational NGNigerian1.55Conversational, spontaneousPunctuated, cased, with disfluencies.

This is part of our continued effort to improve representation in ASR evals (more accents and more languages) and to improve trust (new private data to minimize benchmaxxing, and new data providers). We’re excited to see the community continuing to push ASR performance for the world 🤗

#Hugging Face#Liva AI#Open ASR Leaderboard

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.