Microwave Kinetic Inductance Detectors (MKIDs) are superconducting photon-counting detectors that simultaneously measure the energy and arrival time of individual photons on large-format arrays. The energy resolving power R = E/∆E is the primary figure of merit for spectrophotometric applications, and can be limited in practice by the assumptions underlying the Wiener optimal filter used to estimate photon energy from the detector timestream. The standard approach applies a coordinate transform followed by the optimal filter, an estimator that is optimal only for stationary noise, linear response, and energy-independent pulse shapes. These are assumptions that MKIDs structurally violate. We present Venom (Very Efficient Neural Optimal-filter for MKIDs), a selective state space model based on the Mamba architecture that replaces the coordinate transform and optimal filter with a single learned sequence model operating directly on raw in-phase/quadrature (IQ) timestream data. A single-layer SSM backbone processes IQ samples at the downsampled FPGA output rate, typically 1–2 MHz; an attention-pooling energy head regresses photon energy with a Gaussian negative-log-likelihood loss. The model carries 3,314 trainable parameters and targets deployment on the MKIDGen3 RFSoC readout platform. To deal with the small calibration datasets typical of MKID experiments and help the model avoid classifying instead of assigning a photon energy, we train on synthetic pulses drawn from a streaming-PCA plus shape-preserving (PCHIP) covariance interpolator built from the calibration data, allowing continuous sampling of photon energy between calibration wavelengths. We validate Venom on two published MKID data sets with the same base model: an InHf bilayer resonator at ten wavelengths from 254 to 1310 nm (Zobrist et al. 2022) and a PtSi resonator at five wavelengths from 808 to 1310 nm (Zobrist et al. 2019). All numbers are reported on a stratified 15% held-out validation set that the model and the synthetic-pulse generator never saw during training. On the InHf detector, Venom reaches a mean KDE resolving power of 26.1 versus 24.8 for the published per-wavelength optimal-filter baseline of Zobrist et al. (2022). This is a 5% improvement using a single 3,314-parameter model in place of ten per-laser filter templates, with VENOM leading at six of ten wavelengths by up to 27%. The R of 42.4 at 254 nm is the highest R ever recorded for a UVOIR MKID suitable for use in an array. On the PtSi detector, where the published analysis used a single shared 920 nm optimal-filter template, Venom reaches a mean KDE R = 19.5 vs. R = 11.8 for the shared-template filter on the same held-out subset, a 66% improvement with per-wavelength improvements up to 100%. This large increase for PtSi points to the possibility of the model uncovering either a fault with our previous optimal filtering approach on PtSi, or potential new physics related to pulse shape variations related to photon absorption depth.