{ "id": "ep39zb3f5q", "slug": "music-source-separation", "title": "Music source separation", "updated": "2026-10-15", "topic_path": [ { "id": "technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/topics/technology" }, { "id": "technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing" }, { "id": "technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai" } ], "geo": [ { "id": "geo.nongeo.t1946.technology.computing.ai", "label": "Non-geographic · 1946 to 2000: Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.technology.computing.ai", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t1946", "label": "Non-geographic · 1946 to 2000", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946" }, { "id": "geo.nongeo.t1946.technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.technology" }, { "id": "geo.nongeo.t1946.technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.technology.computing" }, { "id": "geo.nongeo.t1946.technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.technology.computing.ai" } ] }, { "id": "geo.nongeo.t2001.technology.computing.ai", "label": "Non-geographic · 2001 to 2020: Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t2001", "label": "Non-geographic · 2001 to 2020", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001" }, { "id": "geo.nongeo.t2001.technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology" }, { "id": "geo.nongeo.t2001.technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing" }, { "id": "geo.nongeo.t2001.technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai" } ] } ], "excerpt": "Music source separation is a signal processing and machine learning method that splits a mixed music recording into stems such as vocals, drums, bass, and other instruments.", "snippet": "Music source separation is a signal processing and machine learning method that splits a mixed music recording into stems such as vocals, drums, bass, and other instruments.", "node": "technology.computing.ai", "markdown": "# Music source separation\n\nMusic source separation is a signal processing and machine learning method that splits a mixed music recording into individual component stems such as vocals, drums, bass, and other instruments. A typical system takes a waveform as input and produces separated waveforms for four stems: vocals, drums, bass, and other.[1](https://sigsep.github.io/open-unmix/) Separated stems enable karaoke-version generation, intelligent music editing and remixing, vocal pitch estimation, and music transcription.[2](https://export.arxiv.org/pdf/2309.02612v2.pdf)\n\n| Key fact | Detail |\n| --- | --- |\n| Standard input/output | Waveform in; four stereo stems out (vocals, drums, bass, other) at 44.1 kHz[1](https://sigsep.github.io/open-unmix/) • [3](https://github.com/facebookresearch/demucs) |\n| Core mechanism | Learned masks on the mixture spectrogram, or direct waveform modeling[4](https://ar5iv.labs.arxiv.org/html/1909.01174) |\n| Headline accuracy | 9.00 to 11.99 dB SDR on MUSDB18-HQ for current state-of-the-art models[3](https://github.com/facebookresearch/demucs) • [5](https://ar5iv.labs.arxiv.org/html/2310.01809) |\n| Standard benchmark | MUSDB18: 150 full-length tracks, about 10 hours, with isolated four-stem ground truth[6](https://sigsep.github.io/datasets/musdb.html) |\n| Compute | Demucs on CPU runs at roughly 1.5 times track duration; Spleeter runs 100 times faster than real time on one GPU[3](https://github.com/facebookresearch/demucs) • [7](https://www.theoj.org/joss-papers/joss.02154/10.21105.joss.02154.pdf) |\n| Main failure modes | Bleed between stems, failure on sources playing in unison, and additive or subtractive artifacts[8](https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/107959/Benetos%20Perceptual%20errors%20in%20music%202025%20Accepted.pdf?isAllowed=y&sequence=2) |\n\n## How it works\n\nMost music source separation methods learn a mask per source on the mixture spectrogram \\( S := \\mathrm{STFT}(s) \\), where STFT is the Short-Time Fourier Transform. The estimated sources are recovered by inverting the transform: \\( \\hat{s}_{i} := \\mathrm{ISTFT}(\\sigma_{i} \\cdot S) \\), where \\( \\sigma_{i} \\) is the mask for source \\( i \\).[4](https://ar5iv.labs.arxiv.org/html/1909.01174) The mask can be binary, valued in \\( \\{0,1\\} \\), or a soft assignment valued in \\( [0,1] \\). Mask-based spectrogram methods perform very well without requiring large models.[4](https://ar5iv.labs.arxiv.org/html/1909.01174)\n\nAn alternative family operates directly on the waveform, avoiding the spectrogram step entirely; Demucs is a convolutional and recurrent waveform model of this kind.[4](https://ar5iv.labs.arxiv.org/html/1909.01174) Hybrid models run parallel branches in the time and frequency domains and combine them.[9](https://mdx-workshop.github.io/proceedings/defossez.pdf)\n\nQuality is measured with the source-to-distortion ratio (SDR), source-to-interference ratio (SIR), and source-to-artifacts ratio (SAR), a set of metrics introduced by Vincent et al. that decomposes each separated signal into target, interference, and artifact components; these are implemented in the BSS Eval and mir_eval toolkits.[10](https://www.merl.com/publications/docs/TR2025-036.pdf) A scale-invariant variant, SI-SDR, aligns the target with the estimate using only a rescaling factor rather than an FIR filter.[10](https://www.merl.com/publications/docs/TR2025-036.pdf) On MUSDB, SDR is reported as the median across the median SDR over all 1-second chunks in each song, as defined by SiSEC18.[11](https://arxiv.org/pdf/2211.08553.pdf)\n\n## How it is done\n\nOpen-Unmix ships three end-to-end pretrained models (umxl, umxhq, umx) that take waveform inputs and output separated waveforms for the four stems.[1](https://sigsep.github.io/open-unmix/) The umxhq model is trained on MUSDB18-HQ at full 22050 Hz bandwidth, while umx is trained on the AAC-compressed MUSDB18, bandwidth-limited to 16 kHz.[1](https://sigsep.github.io/open-unmix/)\n\nBecause models such as Hybrid Demucs are large and memory-consuming, a full song is separated by chunking it into smaller segments, running the model piece by piece, and rearranging the outputs, with overlap between chunks to accommodate edge artifacts.[12](https://docs.pytorch.org/audio/main/tutorials/hybrid%5Fdemucs%5Ftutorial.html) Hybrid Transformer models support a maximum segment length of 7.8 seconds.[3](https://github.com/facebookresearch/demucs) The final step is stem export; Demucs writes the four stereo wav files at 44.1 kHz into a per-model, per-track directory.[3](https://github.com/facebookresearch/demucs)\n\nDemucs needs at least 3 GB of GPU RAM, or about 7 GB with default arguments, and a --segment option reduces memory; on CPU, processing time is roughly 1.5 times the duration of the track.[3](https://github.com/facebookresearch/demucs) Spleeter's 4-stem model processes 100 seconds of stereo audio in under 1 second on a [GeForce RTX 2080](https://www.edgepedia.org/geforce-rtx-2080), separating the musdb18 test set (about 3 h 27 m of audio) in under 2 minutes.[7](https://www.theoj.org/joss-papers/joss.02154/10.21105.joss.02154.pdf)\n\n## Origin\n\nOne of the earliest techniques exploiting spatial position for music source separation was Independent Component Analysis (ICA).[13](https://inria.hal.science/hal-01945345v1/document) The REpeating Pattern Extraction Technique (REPET), introduced by Z. Rafii and B. Pardo in IEEE Transactions on Audio Speech and Language Processing in 2012, took a different route: it identifies periodically repeating segments in the audio, compares them to a repeating segment model, and extracts the repeating background (accompaniment) from the non-repeating foreground (vocals) via time-frequency masking.[14](https://doi.org/10.1109/tasl.2012.2213249) A 2014 chapter by Zafar Rafii, Antoine Liutkus, and Bryan Pardo generalized REPET to background/foreground separation in audio beyond music, such as speech mixed with repeating background noise.[15](https://doi.org/10.1007/978-3-642-55016-4_14)\n\nSystematic benchmarking began with the Signal Separation Evaluation Campaign (SiSEC), launched in 2007 and continued until 2018; its music separation track was later succeeded by the crowd-based Music Demixing challenge.[10](https://www.merl.com/publications/docs/TR2025-036.pdf) The MUSDB18 dataset, comprising 150 full-length music tracks totaling 10 hours, was released in 2017 according to the Open-Unmix paper,[16](https://github.com/sigsep/open-unmix-paper-joss/blob/master/paper.md) though another account states it was introduced during SiSEC in 2018.[17](https://transactions.ismir.net/articles/10.5334/tismir.171)\n\nThe deep learning era produced several reference systems. Open-Unmix, a reference implementation based on a bi-directional LSTM model, was published in 2019 by Fabian-Robert Stöter and colleagues in the Journal of Open Source Software.[18](https://doi.org/10.21105/joss.01667) Demucs, a convolutional and recurrent waveform model with extra unlabeled data remixed, was published in 2019 by Alexandre Défossez and colleagues.[19](https://doi.org/10.48550/arxiv.1909.01174) Spleeter, a fast command-line tool with pre-trained [TensorFlow](https://www.edgepedia.org/tensorflow) models, was published in 2020 by Romain Hennequin and colleagues in the Journal of Open Source Software.[20](https://doi.org/10.21105/joss.02154) The Music Demixing Challenge 2021 was documented in 2022 by Yuki Mitsufuji and colleagues in Frontiers in Signal Processing.[21](https://doi.org/10.3389/frsip.2021.808395) MoisesDB, a dataset for source separation beyond 4 stems, was published in 2023 by Igor Pereira and colleagues.[22](https://doi.org/10.48550/arxiv.2307.15913)\n\n## Variants\n\nSpectrogram-mask models learn masks on the time-frequency representation, as described above; they are state of the art among compact models but suffer phase inconsistency, which makes drum and bass attack sounds hollow.[4](https://ar5iv.labs.arxiv.org/html/1909.01174) • [9](https://mdx-workshop.github.io/proceedings/defossez.pdf) Time-domain models such as Wave-U-Net, ConvTasNet, and Demucs build their networks directly on the waveform input.[2](https://export.arxiv.org/pdf/2309.02612v2.pdf) Demucs outperformed the then state-of-the-art Wave-U-Net by 1.6 points of SDR on the musdb benchmark.[4](https://ar5iv.labs.arxiv.org/html/1909.01174)\n\nHybrid models run two parallel branches, one temporal and one spectral; Hybrid Demucs adds compressed residual branches with dilated convolutions, LSTM, and local attention.[9](https://mdx-workshop.github.io/proceedings/defossez.pdf) Hybrid Transformer Demucs (HT Demucs) replaces the innermost layers of the hybrid bi-U-Net with a cross-domain Transformer Encoder, using self-attention within one domain and cross-attention across domains.[11](https://arxiv.org/pdf/2211.08553.pdf)\n\nBand-split models take a different approach: Band-Split RNN (BSRNN) splits the spectrogram into subband spectrograms with predefined bandwidths adjusted per instrument type, transforms them to equal-dimension features, and uses stacked residual RNN layers for cross-band and cross-sequence modeling, producing complex-valued time-frequency masks per subband.[23](https://ar5iv.labs.arxiv.org/html/2209.15174) Fine-grained splitting at low frequencies and coarse-grained splitting at high frequencies improves frequency resolution while saving computation.[23](https://ar5iv.labs.arxiv.org/html/2209.15174) BS-RoFormer and Mel-Band RoFormer extend the band-split idea with rotary transformer layers.[2](https://export.arxiv.org/pdf/2309.02612v2.pdf) • [5](https://ar5iv.labs.arxiv.org/html/2310.01809) Notable models including Demucs, Spleeter, ByteSep, and KUIELab-MDX-Net are all variations of a U-Net, while Band-Split RNN is one of the few state-of-the-art systems that does not rely on a U-Net setup.[24](https://ar5iv.labs.arxiv.org/html/2406.18747v2)\n\nDiffusion-based separators are a recent addition: a denoising score-matching diffusion model can serve as a last-stage generative refinement on top of a pretrained deterministic separator, and multi-track latent diffusion models perform separation and generation jointly.[25](https://arxiv.org/html/2412.06965) • [26](https://ar5iv.labs.arxiv.org/html/2409.12346v4)\n\n## Applications\n\nPublished work identifies karaoke-version generation, intelligent music editing and remixing, vocal pitch estimation, and music transcription as the main applications enabled by music source separation.[2](https://export.arxiv.org/pdf/2309.02612v2.pdf) MoisesDB extends the setting beyond four stems with 240 tracks from 45 artists covering twelve musical genres, organized in a two-level hierarchical taxonomy of stems, totaling over 14 hours.[27](https://ar5iv.labs.arxiv.org/html/2307.15913)\n\n## Limitations and alternatives\n\nSeparation quality degrades in characteristic ways. Bleed occurs when the separated output contains a non-target stem in addition to the target stem, due to timbral ambiguity or spectral/pitch overlap of sources.[8](https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/107959/Benetos%20Perceptual%20errors%20in%20music%202025%20Accepted.pdf?isAllowed=y&sequence=2) In experiments, all tested models failed to separate sources playing in unison; in unison cases the less loud instrument is typically absent from its output stem and appears in the louder non-target stem.[8](https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/107959/Benetos%20Perceptual%20errors%20in%20music%202025%20Accepted.pdf?isAllowed=y&sequence=2) Artifacts may be additive (Noise) or subtractive (Spectral Degradation), with spectral degradation typically a byproduct of misclassification or unison.[8](https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/107959/Benetos%20Perceptual%20errors%20in%20music%202025%20Accepted.pdf?isAllowed=y&sequence=2) [Performance](https://www.edgepedia.org/performance) also varies across musical genres and audio components.[28](https://journals.tubitak.gov.tr/cgi/viewcontent.cgi?article=4048&context=elektrik)\n\nAgainst classical alternatives, deep learning dominates. A study on MusDB-HQ comparing FastICA, NMF, and DUET with Hybrid Demucs, Spleeter, Open-Unmix, and Wave-U-Net found machine learning models superior.[28](https://journals.tubitak.gov.tr/cgi/viewcontent.cgi?article=4048&context=elektrik) Similarly, in SiSEC evaluations, classical signal processing methods were clearly outperformed by machine learning methods but remained useful as fast, simple baselines.[16](https://github.com/sigsep/open-unmix-paper-joss/blob/master/paper.md)\n\nEvaluation itself has a known gap: SDR-based metrics do not fully track listener-rated quality, an issue identified as open in recent reviews.[29](https://aaltodoc.aalto.fi/items/182bb4d2-7459-40c7-abb8-0910cf2657ab) Recent progress has converged on spectrogram-domain models combining explicit frequency-band partitioning with transformer-based sequence modeling, of which BS-RoFormer represents the current state of the art on MUSDB18-HQ.[29](https://aaltodoc.aalto.fi/items/182bb4d2-7459-40c7-abb8-0910cf2657ab) Since late 2023, generative refinement has matured: a diffusion refinement stage improves a pretrained deterministic separator, and Consistency Distillation reduces its inference to a single step; the U-Net plus diffusion configuration runs at 570 ms per stem (RTF 0.048), more than 30 times faster than MSDM at 4.6 s, while U-Net plus CD at \\( T = 1 \\) runs at 228 ms (RTF 0.019).[25](https://arxiv.org/html/2412.06965) Multi-track latent diffusion models go further, handling partial or arrangement generation via inpainting, such as adding a guitar to existing bass and drums tracks.[26](https://ar5iv.labs.arxiv.org/html/2409.12346v4)\n\n## References\n\n1. [Open-Unmix documentation (SigSep)](https://sigsep.github.io/open-unmix/)\n2. [BS-RoFormer: Band-Split Rotary Transformer for Music Source Separation (arXiv 2309.02612)](https://export.arxiv.org/pdf/2309.02612v2.pdf)\n3. [facebookresearch/demucs README (official repository)](https://github.com/facebookresearch/demucs)\n4. [Music Source Separation with extra unlabeled data remixed (Demucs introducing paper, arXiv 1909.01174)](https://ar5iv.labs.arxiv.org/html/1909.01174)\n5. [Mel-Band RoFormer for Music Source Separation (arXiv 2310.01809)](https://ar5iv.labs.arxiv.org/html/2310.01809)\n6. [MUSDB18 | SigSep](https://sigsep.github.io/datasets/musdb.html)\n7. [Spleeter: a fast and efficient music source separation tool with pre-trained models (JOSS)](https://www.theoj.org/joss-papers/joss.02154/10.21105.joss.02154.pdf)\n8. [Perceptual errors in music source separation (ISMIR 2025 proceedings, QMUL)](https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/107959/Benetos%20Perceptual%20errors%20in%20music%202025%20Accepted.pdf?isAllowed=y&sequence=2)\n9. [Hybrid Spectrogram and Waveform Source Separation (Hybrid Demucs, MDX Workshop 2021)](https://mdx-workshop.github.io/proceedings/defossez.pdf)\n10. [30+ Years of Source Separation Research: Achievements and Future Challenges (MERL TR2025-036; also arXiv 2501.11837)](https://www.merl.com/publications/docs/TR2025-036.pdf)\n11. [Hybrid Transformers for Music Source Separation (HT Demucs, arXiv 2211.08553)](https://arxiv.org/pdf/2211.08553.pdf)\n12. [Music Source Separation with Hybrid Demucs (PyTorch tutorial)](https://docs.pytorch.org/audio/main/tutorials/hybrid%5Fdemucs%5Ftutorial.html)\n13. [Overview of methods for music source separation (HAL)](https://inria.hal.science/hal-01945345v1/document)\n14. [Z. Rafii, B. Pardo (2012). REpeating Pattern Extraction Technique (REPET): A Simple Method for Music/Voice Separation. IEEE Transactions on Audio Speech and Language Processing.](https://doi.org/10.1109/tasl.2012.2213249)\n15. [Zafar Rafii, Antoine Liutkus, Bryan Pardo (2014). REPET for Background/Foreground Separation in Audio. Signals and communication technology.](https://doi.org/10.1007/978-3-642-55016-4_14)\n16. [Open-Unmix: A Reference Implementation for Music Source Separation (JOSS paper)](https://github.com/sigsep/open-unmix-paper-joss/blob/master/paper.md)\n17. [The Sound Demixing Challenge 2023 – Music Demixing Track (TISMIR)](https://transactions.ismir.net/articles/10.5334/tismir.171)\n18. [Fabian-Robert Stöter and colleagues (2019). Open-Unmix - A Reference Implementation for Music Source Separation. The Journal of Open Source Software.](https://doi.org/10.21105/joss.01667)\n19. [Défossez, Alexandre and colleagues (2019). Demucs: Deep Extractor for Music Sources with extra unlabeled data remixed. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1909.01174)\n20. [Romain Hennequin and colleagues (2020). Spleeter: a fast and efficient music source separation tool with pre-trained models. The Journal of Open Source Software.](https://doi.org/10.21105/joss.02154)\n21. [Yuki Mitsufuji and colleagues (2022). Music Demixing Challenge 2021. Frontiers in Signal Processing.](https://doi.org/10.3389/frsip.2021.808395)\n22. [Pereira, Igor and colleagues (2023). Moisesdb: A dataset for source separation beyond 4-stems. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.2307.15913)\n23. [Music Source Separation with Band-split RNN (arXiv 2209.15174)](https://ar5iv.labs.arxiv.org/html/2209.15174)\n24. [A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stems (arXiv 2406.18747)](https://ar5iv.labs.arxiv.org/html/2406.18747v2)\n25. [Improving Music Source Separation with Diffusion and Consistency Refinement (arXiv 2412.06965)](https://arxiv.org/html/2412.06965)\n26. [Simultaneous Music Separation and Generation Using Multi-Track Latent Diffusion Models (arXiv 2409.12346, IRCAM)](https://ar5iv.labs.arxiv.org/html/2409.12346v4)\n27. [MoisesDB: A Dataset for Source Separation beyond 4-Stems (arXiv 2307.15913)](https://ar5iv.labs.arxiv.org/html/2307.15913)\n28. [A comparative study of blind source separation methods (Turkish Journal of Electrical Engineering)](https://journals.tubitak.gov.tr/cgi/viewcontent.cgi?article=4048&context=elektrik)\n29. [A review of deep learning architectures for music source separation (Aalto thesis)](https://aaltodoc.aalto.fi/items/182bb4d2-7459-40c7-abb8-0910cf2657ab)\n\n---\n*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Artificial intelligence and data*\n\n*Initially written Sep 29, 2026 · Edited: Oct 15, 2026*\n\n*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*\n\nLicense: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license\n", "same_as": [], "url": "https://www.edgepedia.org/music-source-separation", "markdown_url": "https://www.edgepedia.org/music-source-separation.md", "license": { "name": "Edgepedia Community License 1.0", "url": "https://www.edgepedia.org/edgepedia/license", "summary": "Free with credit, commercial use included. AI training is open to everyone. For other uses, organizations over USD 100M in revenue or 100M monthly users license separately.", "spdx": "LicenseRef-Edgepedia-Community-1.0" }, "credit": "\"Music source separation\", Edgepedia (EdgeChat), https://www.edgepedia.org/music-source-separation. Edgepedia Community License 1.0.", "credit_md": "\"[Music source separation](https://www.edgepedia.org/music-source-separation)\", Edgepedia (EdgeChat), [https://www.edgepedia.org/music-source-separation](https://www.edgepedia.org/music-source-separation). [Edgepedia Community License 1.0](https://www.edgepedia.org/edgepedia/license).", "credit_html": "\"Music source separation\", Edgepedia (EdgeChat), https://www.edgepedia.org/music-source-separation. Edgepedia Community License 1.0.", "speakable": "Music source separation is a signal processing and machine learning method that splits a mixed music recording into stems such as vocals, drums, bass, and other instruments." }