Try those if the PF won't do the trick. We prefer 2.35mA if good. Enough to give normal Vbe, current noise less in Q1.
Maybe its oscillation. The mA difference in the meters make me suspicious for that. Is there the C1 installed that decouples?
Do you use a transformer for the on-board rectification or some DC line directly to C1 for feeding the reg?
Looks like a DC line feed. If that's the case it has to go directly to C1. And be at least 10V for 5V output.