We investigate the Landau-Zener description of resonant axion-photon conversion in neutron star magnetospheres. We find that this picture often fails for axion conversions to millimeter-to-optical band photons due to the characteristic resonance width exceeding the size of the conversion region. This comparison of scales yields a simple criterion for evaluating the validity of the Landau-Zener formula. We verify this criterion numerically, and show that when invalid, the Landau-Zener conversion probability may significantly deviate from the numerical result. In light of these findings, we revise constraints on axions from neutron star optical-band polarization searches.