We study the superradiant phase transition of a mesoscopic Fermi gas comprising between a few tens and a few thousand ${}^6$Li atoms in a high-finesse cavity across a wide range of densities. We observe a non-monotonic variation of the superradiant threshold as a function of density, with a minimum reached when the Fermi and recoil wavevectors are comparable. The minimum corresponds to a crossover between Fermi pressure-assisted ordering and Pauli blocking of photon scattering, in good agreement with theory. This interpretation is confirmed by a study of the atom-number dependence of the ordering threshold and photon number scaling. Our system opens the perspective of studying few-fermion systems with strong and coherent light-matter coupling.