Separating a mix back into its parts
Once a song is mixed down to two channels, the individual instruments are gone as separate entities. What remains is the sum of everything, and pulling one element back out of a sum is not an operation with an exact answer.
For years the only available trick was channel cancellation, which relies on vocals usually being panned to the centre. Subtract one channel from the other and anything identical in both disappears. It works, in the sense that the voice gets quieter, and it also removes the kick drum, the bass and the snare, which are centred for the same reason, and it leaves you with a mono file.
A trained separation model works differently. It has learned, from a large number of songs where both the mix and the separate parts were available, what a human voice looks like as a pattern of energy across frequency and time. Given a new mix it predicts which parts of that pattern belong to the voice, and the prediction is made per frequency and per moment rather than per channel. The drums stay, the bass stays, and the result stays in stereo.
Separating a file
- Drop in one or more MP3, WAV, M4A, OGG or FLAC files.
- Choose What to extract, an instrumental, the vocals, or both.
- Run it. The first run downloads the model, and separation itself takes real time.
- Download the WAV, or the zip when you asked for both.
What the model is actually doing
The audio is first decoded to stereo at 44.1 kHz, which is the rate the model expects. Mono material is duplicated into both channels and anything with more than two channels is reduced to two.
That waveform is then converted into a spectrogram, a picture of how much energy sits at each frequency at each instant. This is the representation the model works in, and it is why the approach can separate things that overlap in time: two sounds happening simultaneously usually occupy different frequencies, and where they overlap the model has learned which is more likely to be which.
The model outputs a mask, a value per frequency per frame saying what proportion belongs to the voice. Applying the mask and converting back to a waveform produces the separated audio. The other part is what remains.
Why the output is uncompressed
The separated audio arrives as raw samples and is written directly to a WAV.
Encoding it to MP3 at that point would layer compression artefacts on top of separation artefacts, and the two do not hide each other. Separation leaves faint spectral residue in unusual places, precisely the kind of thing a lossy encoder handles badly, and the combination sounds noticeably worse than either alone.
The files are large as a result. If you want them smaller, convert afterwards as a deliberate step with MP3 to WAV in reverse, once you have confirmed the separation is good enough to keep.
What to expect, honestly
The instrumental will have faint traces of the voice in it, mostly in loud sections and on reverb tails, because reverb spreads a voice across time and frequency in ways that are hard to attribute. The acapella will carry a little of the backing for the same reason.
Sparse arrangements separate better than dense ones. An acoustic recording with a clear vocal comes out very cleanly. A modern production with heavy compression, layered harmonies and effects on everything is a much harder problem.
None of this is a limitation of this particular tool so much as of the task. Reconstructing parts from a mix is inference, and no amount of processing recovers information that the mixdown discarded.
Related audio tools
If the source is quiet or uneven before you start, Normalize Audio gives the model a better input to work from, and Remove Audio Noise helps with a noisy recording. If one channel of your source is dead, fix that first with Convert Stereo to Mono, since a broken stereo image confuses the separation.
To get the words as text rather than as audio, Transcribe Audio runs a different model in the same local way, and works noticeably better on a separated vocal than on a full mix. The rest are on the audio hub.