I've been thinking recently, that all kinds of issues with the playback that give harder time for the brain to interpret the sound will give some sort of listening fatigue and take away from the perception.
...What ever helps you enjoy the music.
I came to same conclusion.