Parasitic interference is a common limitation in laser interferometers, arising from unwanted beams that corrupt the phase measurement and degrade displacement sensitivity. In this work, we present a unified framework for the characterization and mitigation of parasitic interference in heterodyne interferometers. Parasitic beams are classified into two types based on the orientation of their corresponding phase vectors, and their noise contribution is modeled as a function of polarization, relative amplitude, and phase of the parasitic beam. Central to this framework is the concept of coupling coefficients, which quantify the interferometer’s susceptibility to parasitic interference and can be readily computed using Jones calculus for any optical configuration. The two types of parasitic interference motivate distinct mitigation strategies: differential interferometry and balanced detection, complemented by polarization control and high-quality beam splitters. The models and mitigation strategies are validated experimentally in both a simplified Mach-Zehnder interferometer and a differential interferometer used for optomechanical inertial sensing, demonstrating reductions in parasitic phase noise by up to four orders of magnitude and achieving sub-picometer displacement sensitivity at frequencies as low as 9 mHz under ambient atmospheric pressure.