A real, honest-to-goodness voltage regulator has three parts: a fixed voltage reference, an error amplifier, and a pass element.
Most people only put eyeballs on the final, all-wrapped-up-in-a-tidy-IC-package version, typified by the LM7812, or with a couple of extra gain-set resistors, the LM317. These chips have a working bandwidth about about 2 kHz, as they are designed to 1) reduce 120 Hz ripple and 2) be rock stable no matter what abuse they are subjected to. As a result at audio frequencies and above they are pretty much noise generators...
Knowing this, many people have set out to build better regulators for audio work.
The most obvious route is to build a high performance LM7812 from discrete components. (Most excellent review
here.) AD797 for the error amplifier, high stability, low noise voltage reference, etc. The trick though, is stability. The LM7812 is low bandwidth not because it's too cheap to manage anything better, but because...