Gravitational wave (GW) observations place stringent constraints on fourth-order gravity. In the vanishing mass limit, cancellations among the massless graviton and extra massive spin-2 ghost and scalar modes prevent recovery of the general relativity quadrupole formula, in direct conflict with precision binary measurements. Using pulsar binaries and GW170817, we derive a lower bound $m\gtrsim {10}^{-11}eV$ on the massive modes and also obtain constraints on ghost-free fourth-order gravity. Our results serve the first GW test of fourth-order gravity and can be extended to alternative gravity theories.