On the other hand if your speaker cable is 180 Meters long, then you might have a real problem with a 500 kHz carrier frequency
It is not only depending upon cable length. If your ripple is symmetrical (i.e.. the output stage is a full bridge) then you can't radiate a strong electric field component. And if your wire is properly twisted, the magnetic field component will also be cancelled by large amounts.
BTW: My PWM amp had a carrier suppression of 83 dB (i.e. less than 5 mV at an output voltage capability of +- 45 Volts). For an amp used in active speakers I would go for less carrier suppression in favour of less impact on the audio signal.
I agree wit NP that the sound quality problems might come from the modulation and switching part of the amp. We have to take care about timing- AND amplitude- errors because both influence linearity. Additionally a class-d amp has often worse PSU rejection than linear amps. So one has to be careful and not be tempted to use a crappy PSU just because of the good efficiency of the amp. I think that is the cause for some people (some colleagues in the P.A. business) claiming that class-d amps had bad LF reproduction (my own experience is the contrary.) The output filter per se should not be that much of a problem, apart from reducing the slew capabilities of the amp.