Inertialess elastic instabilities can profoundly affect transport in complex-fluid flows, from mixing in microfluidic devices and displacement through porous media to polymer processing. While such instabilities are well understood in flows with curved streamlines, their origin in rectilinear flows like the prototypical Kolmogorov flow remains unresolved. Here, we report the first experimental evidence of the center-mode elastic instability, at vanishing inertia, in a confined quasi-two-dimensional viscoelastic Kolmogorov flow, driven by electromagnetic forcing. The instability is characterized by the emergence of ``arrow-shaped" structures in the velocity field, which fluctuates chaotically as the forcing strength increases. We reproduce these observations in simulations of the Oldroyd-B model and trace the dynamics in the confined flow to the travelling arrowhead structures that emerge from the center-mode instability of unbounded Kolmogorov flow. Our work establishes a generic route to purely elastic instability in rectilinear flows, broadening the conditions under which elastic turbulence and enhanced mixing can arise.