Strong electronic correlations generally require non-perturbative treatment. Local correlations are captured by dynamical mean-field theory while nonlocal correlations can be treated with diagrammatic extensions such as the Dual Fermion approach. Dual Fermion is built on physically motivated, but in principle uncontrolled approximations, so careful benchmarking is needed to understand the strengths and limitations of the method. In this work, we benchmark ladder Dual Fermion and dynamical mean-field theory for the Falicov-Kimball model with the exact classical Monte Carlo solution. We focus on the thermodynamics, electronic structure and susceptibility, especially at the combined frequency and momentum structure, and find that Dual Fermion clearly outperforms dynamical mean-field theory. Somewhat surprisingly, Dual Fermion is not as accurate for the relation between orbital density versus chemical potential in the doped system. These results demonstrate the need for rigorous benchmarking of diagrammatic extensions of dynamical mean-field theory for models with inequivalent orbitals, which is essential for modelling materials.