Crossover Networks

My Audio Engineering Textbook (written by Marshall Leach) indicates that the phase response of the driver should be taken into account. If a zobel or voice coil inductance compensation network is used and the crossover points are sufficiently far from the loudspeaker resonance can the phase response of the driver be ignored? In other words, could a perfect phase response be had with a 2nd order crossover with the polarity of the midrange switched.