Harnessing the interaction between light and matter at the quantum level has been a central theme in atomic physics and quantum optics, with applications from quantum computation to quantum metrology. Combining complex interactions with photonic synthetic materials provides an opportunity to investigate novel quantum phases and phenomena, forging insightful connections to condensed matter physics. Here we explore many-body phenomena with a single artificial atom coupled to the many discrete modes of a photonic crystal. This experiment reaches the ultrastrong light-matter coupling regime using the circuit quantum electrodynamics paradigm, by galvanically coupling a highly nonlinear fluxonium qubit to a tight-binding lattice of microwave resonators. In this regime, the transport of a single photon becomes a many-body problem, owing to the strong participation of multi-photon bound states arising from interactions that break particle number conservation. Leveraging the effective photon-photon interactions mediated by the qubit, the transport of multiple photons gives rise to complex multimode dynamics, enabling the generation of a continuous reservoir of strongly correlated photons, providing a potentially useful resource for distributing remote entanglement. This work opens exciting prospects for exploring nonlinear quantum optics at the single-photon level and stabilizing entangled many-body phases of light.