It's hard to separate the magnetic field from the potentially inductor-influencing pieces of ferrous metal that creates it.
However, if you check the technical performance (firstly frequency response) of the speaker after assembly, you'll find how much influence there is. It may be very small - insignificant.
well Zaph|Audio - ZMV5 - MCM / Vifa 5" System shows a single steel screw changing inductance by 10%. To some 10% is significant. To others not. YMMV, batteries not included etc.