Mixing outputs of two pre amps

Hi,

I am currently building a Tube amp based on the Audio Note EL84. The main amp is functioning well. I have also constructed a Audio Note L3 phono pre amp. Also functioning well.

To complete my project I am now designing a PCB with which I can switch between different audio sources like CD player, Record player, Streaming device and Aux. The CD player, Streaming device and Aux use a simple NE5532 pre amp like https://xtronic.org/circuit/preamplifier-circuit-diagram-op-amp-12v/. Instead of the fixed resistors I use digital resistors on an arduino microcontroller to set the gain for each source. Also works fine. The sources are separated with 4 relays three relays connect directly to the NE5532 one relay get input from

But now the challenge/question. I have wired the two pre amps directly to the input of the EL84 main amp. This means when I use the Phono tube amp this input signal will be on the output of the NE5532. The other way around is not applicable as the relay of the Phono source is closed in that condition.

Will placing an input of the NE5532 output when I am not using it ham the pre amp? I tried to place two diodes on the output of the pre amp but found out that that is not working. Is there another method to make the input one way instead of diodes?

Please see the schematic attached if my description is not clear.
Input schematic.jpg