To give you a ballpark figure:
Suppose you have 30 dB main amp voltage gain, two 90 dB, 1 W, 1 m, 8 ohm loudspeakers, a room with 1 m reverberation radius and you want to keep the noise level below 0 dB(A). With one loudspeaker, you could then allow 10^(-90/10) W=1 nW of A-weighted noise, but as you have two of them, you need to reduce that to 500 pW per loudspeaker. In 8 ohm, that corresponds to an RMS voltage of sqrt(500 pW*8 ohm)~=63.25 uV RMS. At the input of the main amp, the allowable voltage level is 10^(30/20) times smaller, or 2 uV RMS A-weighted.
You measure 1.2 mV, which is 600 times as high as what I just calculated, giving you a noise sound pressure level of around 55.56 dB, which is quite high.
Maybe things are not as bad as they seem, because you don't specify whether you measured a RMS, quasi-peak or quasi peak-to-peak value and over what bandwidth you measured. If you haven't limited your measurement bandwidth, a large part of the noise may actually be ultrasonic and will only annoy your dog, cat or any bats living in your house.
About the generator: you can measure the preamp noise with input shorted, or with the input terminated by a resistor. Why would you need a generator?