Guess what? It's not, the "slightly bland" is the giveaway - distortion can make something sound sort of clean, but it is still most certainly distortion. Trouble is, standard measuring procedures don't pick it up ...
I am like a dog with a bone now and must find out . The output impedance and input loading will be checked , as few variables as possible . The output resistance can be whatever I like . 90 m ohm is where I start . I bet every headphone will have it's preference . In theory even 10R should suit a 32R load ( very old theory ) . Damping factors like that encourage a bit of bloom . It might also help the amp a bit .
One thing I might try is a null test . My analyzer will do that quite well as it isn't referenced to ground . Clip input ground to source and output to phones . Use tones and music . See what comes out . Try different phones also .