De-powering unused sub-circuits. Good pratice?

Is it a good idea to de-power unused sub-circuits?

For example. If I was making an integrated amplifier that included say a phono stage, a dac and a Bluetooth module.

Would it be a good idea to switch the output of a say +/- 15vdc power supply so that it is only powering the module in use at the time.

For example if I was using the phono stage; The phono stage would be powered but the dac and Bluetooth modules would not be.

Is this unnecessary complexity?
Would I be better to just leave all 3 powered all the time?
 
This is DIY so do it your way. I have a DIY preamp with multiple retulated power supplies that can be switched off. Suppose you are listening to CD and no longer need the momentary input switching feature? disconnect! You don't need the headphone amp powered? disconnect! You don't need the Bal->SE conversion feature? disconnect! Those front panel lights bugging you? disconnect!

If you can hear the noise levels go down as certain stuff is disconnected then it was worth it, if not, its still fun to be able to do it.
 
A possible problem with powering down (and back up), is the potential for power up thumps. Have a plan for that and I see no problem other than added complexity.

Speculation: Many failures happen at power up. From a reliability perspective it might be best to minimize power cycles.