The Toslink DAC probably needs the output capacitor to block DC voltage. But, it was setup for an input resistance of 100k minimum (15.9 Hz cutoff). That would even be marginal. Use this
high pass filter calculator . You want the cutoff frequency to be much lower than 20 Hz, because you want phase shift in the audio band to be minimal. For usual solid state 10k loads, they typically use a 10uF coupling cap, which gets the cutoff frequency down to 1.59 Hz. It doesn't need to be that low.
With pots, there is a fixed resistance facing the coupling capacitor, when AC coupled. The high pass filter will then have a fixed cutoff frequency. The wiper connects to the following stage. Buffers after the pot do help a great deal, in my experience.
If you want to DC couple, as in your original idea, you would use JFET input op amps and dual supplies for lowest DC offset. I would also use adjustable regulators (e.g. LM317/LM337), so that you can get the rail voltages symmetrical. Buffer-pot-buffer arrangement.
I just use a PGA2310 based volume control. I like it. They are not available on Ali any longer. They now have the improved (assuming)
MUSES72320 units. You get remote control. Not very DIY. There are some
DIY projects here using the MUSES72320 or MUSES72323.