[Paper] MuseCPBench: an Empirical Study of Music Editing Methods through Music Context Preservation

Source: arXiv - 2512.14629v1

Overview

MuseCPBench presents the first systematic benchmark for measuring Music Context Preservation (MCP)—the ability of music‑editing models to keep the “unchanged” parts of a track intact while altering a target attribute (e.g., timbre, instrument, genre). By exposing inconsistencies in existing evaluation practices, the authors give developers a reliable yardstick for comparing and improving music‑editing tools used in film scoring, game audio pipelines, and streaming services.

Key Contributions

• MCP Benchmark (MuseCPBench): A curated dataset and evaluation suite covering four musical‑facet categories (rhythm, harmony, timbre, high‑level structure).
• Unified Metrics: Introduces a set of objective and perceptual metrics (spectral distance, pitch‑class similarity, rhythmic continuity, listener‑study scores) that can be applied uniformly across models.
• Comprehensive Baseline Comparison: Evaluates five representative music‑editing approaches (GAN‑based, diffusion, VAE, transformer, and rule‑based pipelines) on the benchmark.
• Diagnostic Analyses: Breaks down performance by facet, model architecture, and editing operation, revealing systematic preservation gaps (e.g., rhythm often drifts in timbre‑transfer models).
• Open‑Source Release: Provides code, pretrained checkpoints, and a web demo so the community can reproduce results and plug in new models.

Methodology

1. Dataset Construction – The authors assembled 1,200 multi‑instrument tracks from public stems (e.g., MedleyDB, DSD100) and annotated them with ground‑truth facet labels (tempo, chord progression, instrument timbre, song sections).
2. Editing Scenarios – Four editing tasks were defined:
   • Timbre Transfer: swap a target instrument while keeping melody and rhythm.
   • Instrument Substitution: replace a whole track (e.g., piano → synth) without altering harmonic content.
   • Genre Transformation: change production style (e.g., pop → lo‑fi) while preserving melodic contour.
   • Structural Editing: reorder sections (intro, verse, chorus) while keeping local musical details.
3. Evaluation Pipeline – For each edited output, the benchmark computes:
   • Objective Scores: spectral convergence, pitch‑class histogram similarity, onset‑offset alignment, and segment‑level structural similarity.
   • Perceptual Scores: crowdsourced listening tests asking participants to rate “how much of the original musical context feels unchanged.”
4. Baseline Implementations – The five models were either taken from the original papers or re‑implemented following the authors’ public code, ensuring a fair comparison under the same data splits and hyper‑parameters.

Results & Findings

• Rhythmic fidelity is the most robust facet across all models (average preservation > 0.85).
• Harmony suffers the most in genre‑transformation pipelines, with average chord‑class similarity dropping to 0.58.
• Diffusion models excel at timbre changes but still introduce subtle timing jitter, reflected in lower onset‑alignment scores.
• The rule‑based structural editor, while simple, outperforms learned models on preserving high‑level song sections, highlighting that “hard‑coded” musical knowledge can still be valuable.

Ablation studies show that adding a context‑preservation loss (e.g., contrastive similarity between original and edited non‑target stems) improves MCP scores by 5–10 % across the board.

Practical Implications

• Audio Engineers & Game Sound Designers can now benchmark their in‑house editing tools against a community standard, ensuring that automated timbre swaps won’t unintentionally shift groove or harmonic intent.
• Streaming Platforms looking to generate “personalized” versions of tracks (e.g., instrument‑specific stems for karaoke) can select models with proven MCP scores, reducing the risk of user‑perceived quality loss.
• Tool Vendors (DAW plugins, AI‑powered audio suites) can integrate MuseCPBench as a regression test, catching regressions in context preservation before release.
• Research & Development benefit from the open‑source metric suite to quickly prototype new loss functions or architecture tweaks aimed at specific facets (e.g., a “rhythm‑preserving” regularizer for genre conversion).

Limitations & Future Work

• Genre Coverage – The benchmark currently focuses on Western popular music; non‑Western scales, micro‑tonality, and traditional instruments are under‑represented.
• Subjectivity in Perceptual Scores – While crowdsourced ratings provide valuable insight, they can be influenced by listener expertise and playback environment; a more controlled lab study could refine these numbers.
• Scalability – Evaluating large diffusion models on the full dataset is computationally expensive; future work may explore proxy metrics that correlate well with full MCP scores.
• Extension to Real‑Time Editing – The current benchmark evaluates offline edits; extending the suite to measure latency and streaming‑compatible preservation would be valuable for interactive applications.

By exposing where today’s music‑editing models fall short, MuseCPBench sets a clear roadmap for building AI tools that respect the musical context—an essential step toward trustworthy, production‑ready audio generation.

Authors

• Yash Vishe
• Eric Xue
• Xunyi Jiang
• Zachary Novack
• Junda Wu
• Julian McAuley
• Xin Xu

Paper Information

• arXiv ID: 2512.14629v1
• Categories: cs.SD, cs.AI
• Published: December 16, 2025
• PDF: Download PDF