THD measurements - how to correctly wire attenuator for amplifier output and audio interface balanced input?

As others have noted, pin 1 should be connected to pin 3 in the XLR and connected to RCA ground, and pin 2 in the XLR should be connected to RCA positive.

As PRR noted, all of the noise is 60Hz power line noise and its harmonics, plus the resultant intermodulation distortion. That can be from external noise picked up from AC power sources and it is also quite possible that your amplifier's power supply and wiring arrangement is producing that noise.

What amplifier are you measuring? Post pictures of your setup and internal shot of the amplifier that you are measuring.
 
The basic interface schematic is incorrect, see Jan’s comments.

The load connects to the amplifier speaker terminals and the attenuator signal also connects to the amplifier speaker terminals. CN1/2 need 4 wire connections, two to load and the other two to the attenuator. In low power amplifiers you may not see any differences however larger powers amplifiers you will see the different in the measurements.

Checking the test system first is the #1 test. What does the spectrum signal look like? Is it free of any AC LINE NOISE? Make this test by using an open input to the XLR, then shorted pin #2 to pin #3 now short Pins #1, 2 & 3. Now this is the lowest noise you can expect from this instrument.

As you add other cables, sources and loads the measurement noise floor will be degraded. The XLR input connector if correctly used only sees the signal from pin #2 to pin #3. Driving pin #1 with NOISE should have no degration to the signal measurements.

I have made my living designing & measuring audio/analog and other type products. Never assume that it is OK, measure twice, cut once.

Duke
 
Here are attached results that I am getting when connecting PIN1 and PIN2 of the XLR mic input for the audio interface:

(just so people get the idea of what I mean by interference - it can be seen as "grass" on those graphs)
That's a good proof that you did it wrong.
Have you actually checked the correct way to connect XLR connectors?
We have this app here called 'Google', you just type in a search term and it tells you where the beef it.
You should check it out sometimes ;-)

Jan
 
Also keep in mind some amplifiers ( - ) output are not connected to the chassie power ground.
Your interface is a mic preamp with mic level inputs, even with the pad you may still have too much input
level.

Still thinking you need some isolation between the amp and the interface.
Indeed, (-) output should NOT be connected to power ground. It should be connected to signal ground.
Connect it to power ground and you're almost certainly get hum/noise.
It's really all very logical if you think about it.

Jan
 
According to this article here: https://khronscave.blogspot.com/2019/10/55-focusrite-scarlett-2i2-teardown.html

The NJM2122's in these preamps are wired as the two input opamps in a three-opamp instrumentation amplifier. The third opamp is one half of an NJM4565 (U18 in this case).

View attachment 1130149
First of all, IMHO, your measurements do not look so bad, the amp -whatever it is- seems rather noiseless. What caught my attention was that you can't make it work using pins #2 and #3. Given the input of your soundcard, it should be wrong wiring, but honestly I would stay with the scheme you are using, it seems OK.