You always, 100% of the time measure DC offset with open outputs (unless tube, then it doesn't matter and they must have a load), and also shorted inputs. Service manuals are quite clear on this. The reason? Noise, and meters can react differently to AC on the DC scale. So noise may throw off your readings considering you are measuring in the mV range. If your meter only has a 100 mV or 1 V minimum full range sensitivity, you need a new meter. Especially if you look at the accuracy specs.
So to answer your question, always with shorted inputs.
So to answer your question, always with shorted inputs.