Will Neodymium magnets in surround affect driver performance

Hi all,
So have finally finished my first ever build and decided after some close calls with drunk mates getting over excited to buy a metal grill to keep it safe. I've done some research into different ways on how to fix the grill on and really liked the idea of using magnets to make it easily removable. I have bought a metal grill that fits snugly and was going to stack some magnets to give a little extra space for excursion. The plan is to put the magnets in the nail holes on the driver surround as they fit perfectly and look discreet

My question is, will having extra strong magnets that close to the driver effect its performance/movement in any way? I know the magnet fields don't reach very far but I just wanted to check before going ahead with it.

Thanks!