They use that along with gap pads to cool the drivers.
You could drop the boards in a glass jar with solvent (acetone) and agitate them to break up what you can't scrape off. Unless there is a reason that you cannot work around it, I'd leave it. If you don't, you'll have to replace it.
It may be the base fluid used in the compound you're using. I use Dow 340 and it comes out 100% in a single wash. Remove as much as you can before washing and see if that helps. It could also be the type of washing machine you use. A high efficiency machine may not beat the clothes as hard as one with an agitator.
I have nac for using CRC brake cleaner for everything, including laying waste to huge **** spiders here in Fl.. Nothing, I repeat nothing kills bugs faster than the red can of CRC!~ Works great for removing old compound too..