Sounds more like a wrong connection. If you look at the potentiometer from the front (knob) the left pin goes to ground, the right to the RCA connector and the center pin to the amplifier's input.
If that does not cure the fault, post a schematic of your amplifier.