When the parameters of a physical system are varied, the eigenvalues of observables can undergo crossings and avoided crossings among themselves. It is relevant to be aware of such points since important physical processes often occur there. In a recent paper [M. Bhattacharya and C. Raman, Phys. Rev. Lett. 97, 140405 (2006)] we introduced a powerful algebraic solution to the problem of finding (avoided) crossings in atomic and molecular spectra. This was done via a mapping to the problem of locating the roots of a polynomial in the parameters of interest. In this article we describe our method in detail. Given a physical system that can be represented by a matrix, we show how to find a bound on the number of (avoided) crossings in its spectrum, the scaling of this bound with the size of the Hilbert space and the parametric dependencies of the Hamiltonian, the interval in which the (avoided) crossings all lie in parameter space, the number of crossings at any given parameter value, and the minimum separation between the (avoided) crossings. We also show how the crossings can reveal the symmetries of the physical system, how (avoided) crossings can always be found without solving for the eigenvalues, how they may sometimes be found even in case the Hamiltonian is not fully known, and how crossings may be visualized in a more direct way than displayed by the spectrum. In the accompanying paper [M. Bhattacharya and C. Raman, Phys. Rev. A 75, 033406 (2007)] we detail the application of these techniques to atoms and molecules.