PCB mounting

I'm nearing completion of my BrianGT 3875 project, and it turns out I forgot to order PCB stand offs. I would like to use 4-40 aluminum, but that's not an option locally and I don't want to pay $8 to ship $2 of parts. Radio Shack has some 4-40 standoffs, but the metal ones look steel (correct me if I'm wrong) and the plastic ones are too short. The local Ace has a good selection of 6-32 aluminum and nylon spacers that would work great. Is there any reason I couldn't drill/ream out the mounting holes a bit? Would I crack the board? This seems like a silly question, but I haven't found many options searching.