Rohan Paul

StepFun has released StepAudio 3 Music, a model that turns a text description and lyrics into a finished song. The interesting engineering result is that better audio reconstruction did not necessarily produce better music generation. The technical report compares single-codebook and residual vector quantization approaches. The final tokenizer uses one token stream at 50 Hz with 65,536 entries, followed by a flow-matching DiT renderer. Why this matters: the representation has to preserve sound AND give the autoregressive model a sequence it can predict reliably. Optimizing the codec in isolation can miss that trade-off. (In comment you will find some audio examples to give this architecture some context)