non at all would be the best . Since this can be a bit problematic you could use some kind of stepped device using resistors. I tried both series and ladder konfigurations and both were a lot better than my standard alps pot. The first I build had 1dB steps wich turned out to be much too fine. Now I use one with 3dB steps wich is allright and gives a usefull attenuation range. I never tried the solid state devices or relais arreys but these could give similar results.
I use a home made stepped attenuator, it is more transparent than any pot (expensive or inexpensive) it has 2db steps and
covers a 24 db range, (never required more attenuation),
The trick is to initially build it with fairly inexpensive metal film resistors, then when you've had it for some time you will probably have just a few positions that you regularly use, identify the resistor values at these settings and buy some high quality vishay or whatever is flavour of the month and use them to replace the resistors at your preferred settings.
DIYers typically use Alps or Panasonic for availability reasons. Alps is perhaps the most commonly used high end pot for audio in the world. Noble is very popular with high end equipment makers but is hard to obtain in small quantities. The previously mentioned Penny and Giles is a British company that caters primarily to the pro audio and mixer market. It is rare to see their products elsewhere.
Horible quality control and fagile to solder to. The most overpriced pot I have ever seen. This pot drove a very well know preamp designer to stepped attenuators with relays. I would not use them if the were free. I believe they were over $150 last time I priced one.