James Webb Space Telescope (JWST) has opened up a new chapter in infrared astronomy. Besides the discovery and a deeper understanding of various astrophysical sources, JWST can also uncover the non-gravitational nature of dark matter (DM). If DM is QCD axion or an eV-scale Axion-like particle (ALP), Milky Way DM can decay into two photons and produce a distinct line signature in the spectroscopic observations made by JWST. Using the latest NIRSpec IFU spectroscopic observations from JWST, we put the strongest bound on the photon coupling for QCD axion/,ALP DM in the mass range between 0.47 and 2.55,eV. In particular, we are able to probe a new mass range for ALP DM between $\sim$ 0.47 eV to 0.78 eV beyond what can be probed by globular cluster observations. We constrain well-motivated and UV complete models of QCD axion and ALP DM, including predictions from some models derived from string theory and/,or various Grand Unification scenarios. Future JWST observations of DM-rich systems with a better understanding of the astrophysical and instrumental backgrounds can thus enable us to potentially discover QCD axion and ALP DM. The datasets used in this work are available at: .