it's bad design PLL's filter. I've got the same problems. Just read Crystal's design documents about suggested board layout, use best caps... traces short as possible...
Good luck
Same thing is was happening on one of my prototypes. I believe it is related to EMI or interference on ground loops. If you post your schematic/layout we might be able to help some more.
Crystal's AN159 document p.2nd-- recommended board layout... RC soldered almost "on pin" (0605)... PLL's ground and Vcc via R-C, reciever trafo+RS485... Now it's okay
Or maybe reference layout SLEU010A from Texas?
The same problems I was with PCM2702/2906...
