Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Apple researchers detect code-switching directly from speech encoders

AI summary

A paper from Apple Machine Learning Research proposes a lightweight method that detects code-switching — alternating languages within one utterance — directly from speech, without relying on accurate multilingual transcription. The approach uses frame-level language information in pretrained speech encoders: it identifies language-selective hidden-state dimensions from monolingual recordings and aggregates their frame-level activations into a code-switching prediction. With a small code-switching set used to tune a decision rule, the authors report the Whisper-small encoder reaches a mean F1 of 0.798 across 14 language pairs in CS-FLEURS, versus 0.776 for a CNN and 0.751 for a HuBERT linear baseline trained for monolingual language identification.

In-depth insight

The authors say pretrained speech encoders already encode the information needed to identify code-switching utterances, offering a lightweight tool for building more representative multilingual corpora.

research area Methods and Algorithms, research area Speech and Natural Language Processing

content type paperpublished October 2026

Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models

AuthorsJie Chi, Zakaria Aldeneh, Masha Fedzechkina, Maureen de Seyssel, Natalie Schluter

View publication

Code-switching (CS), the alternation between languages within an utterance, is common in multilingual speech but underrepresented in pretraining corpora. Increasing CS coverage requires finding relevant utterances in large speech collections so that they can be included in training data. One way to find such utterances is to transcribe the speech and detect language alternation in the text, but this depends on obtaining accurate multilingual transcriptions. We propose a lightweight method that detects CS directly from speech. It uses frame-level language information in pretrained speech encoders to determine whether both languages are present within an utterance. Specifically, we identify language-selective hidden-state dimensions using monolingual recordings and aggregate their frame-level activations into a CS prediction. With a small CS set used to tune a decision rule, we show that the Whisper-small encoder achieves a mean F1 of 0.798 across 14 language pairs in CS-FLEURS, compared with 0.776 for a CNN and 0.751 for a HuBERT linear baseline trained for monolingual language identification. Our findings suggest that pretrained speech encoders already encode the information needed to identify CS utterances, and they provide a lightweight tool for building more representative multilingual corpora.

#Apple#Apple Machine Learning Research#Whisper-small#HuBERT#CS-FLEURS

Original · 2 min readRead the original
Group chat

No comments yet. Start the conversation.