[Paper] SpidR-Adapt: A Universal Speech Representation Model for Few-Shot Adaptation

Source: arXiv - 2512.21204v1

Overview

SpidR‑Adapt is a new universal speech‑representation model that can learn a new language from just a handful of unlabeled audio hours—a scale comparable to what infants hear when they start speaking. By framing low‑resource speech learning as a meta‑learning problem, the authors achieve 100× better data efficiency than conventional self‑supervised approaches, making rapid language adaptation practical for real‑world products.

Key Contributions

• Meta‑learning formulation for speech adaptation – treats each language as a “task” and learns how to adapt quickly to new tasks.
• Multi‑task Adaptive Pre‑Training (MAdaPT) – a bi‑level optimization framework that jointly optimizes a universal encoder and language‑specific adapters.
• First‑Order Bi‑level Optimization (FOBLO) – a lightweight heuristic that sidesteps the expensive second‑order gradients normally required for meta‑learning.
• Interleaved supervision – alternates self‑supervised and supervised objectives during meta‑training, yielding a stable and robust initialization.
• Architecture‑agnostic – works with any backbone (e.g., wav2vec 2.0, HuBERT), so existing pipelines can be upgraded without redesign.
• Open‑source release – code, pretrained checkpoints, and evaluation scripts are publicly available.

Methodology

1. Base Encoder – a standard self‑supervised speech model (e.g., wav2vec 2.0) is first trained on a large multilingual corpus.
2. Task Definition – each target language constitutes a separate adaptation task.
3. Bi‑level Optimization
   • Inner loop: fine‑tune a tiny language‑specific adapter on a few minutes/hours of unlabeled audio from the target language.
   • Outer loop: update the universal encoder’s parameters so that, after the inner adaptation, performance on a held‑out validation set improves.
4. FOBLO Approximation – instead of computing full second‑order gradients, the authors use a first‑order approximation that treats the inner‑loop updates as fixed, dramatically reducing compute.
5. Interleaved Supervision – during meta‑training, the model alternates between a contrastive self‑supervised loss and a supervised phoneme classification loss (available for a small set of high‑resource languages). This stabilizes training and yields a better starting point for adaptation.

Results & Findings

• Data efficiency: comparable or better scores are achieved with <1 hour of target‑language audio, a >100× reduction in required data.
• Speed: adaptation completes in under 10 minutes on a single GPU.
• Generalization: the same meta‑trained encoder works across 20+ languages, demonstrating true universality.

Practical Implications

• Rapid deployment of voice assistants in emerging markets: a product team can add a new language to an existing speech stack with a few hours of recorded user utterances, no need for costly transcriptions.
• Low‑resource research: researchers can experiment with under‑represented languages without building massive corpora, accelerating linguistic diversity in AI.
• Edge devices: because the adapter modules are tiny (a few thousand parameters), they can be shipped as lightweight patches, keeping the bulk of the model on the server.
• Continuous learning: the bi‑level framework naturally supports on‑device fine‑tuning as more unlabeled audio streams in, enabling “learning while listening” scenarios.
• Plug‑and‑play upgrade: any existing wav2vec 2.0/HuBERT pipeline can be swapped for the SpidR‑Adapt encoder without architectural changes, preserving downstream task heads (ASR, speaker ID, etc.).

Limitations & Future Work

• Dependence on a strong multilingual base – the meta‑learning gains diminish if the initial encoder is trained on a narrow language set.
• Adapter size vs. performance trade‑off – while adapters are lightweight, extremely constrained environments may still find the extra parameters non‑trivial.
• Evaluation limited to phoneme‑level and language‑model probes; downstream ASR word error rates were not reported.
• Future directions include extending the framework to multimodal adaptation (e.g., audio‑visual speech), exploring online FOBLO for continual learning, and testing real‑time on‑device adaptation with privacy‑preserving constraints.

Authors

• Mahi Luthra
• Jiayi Shen
• Maxime Poli
• Angelo Ortiz
• Yosuke Higuchi
• Youssef Benchekroun
• Martin Gleize
• Charles‑Eric Saint‑James
• Dongyan Lin
• Phillip Rust
• Angel Villar
• Surya Parimi
• Vanessa Stark
• Rashel Moritz
• Juan Pino
• Yann LeCun
• Emmanuel Dupoux

Paper Information

• arXiv ID: 2512.21204v1
• Categories: cs.CL, cs.AI
• Published: December 24, 2025
• PDF: Download PDF