Determining crystal symmetry from powder X-ray diffraction (PXRD) is a central problem in materials characterization, yet multiple space groups produce indistinguishable patterns, and automated methods plateau at low accuracy on real data. The open question is where the recoverable symmetry information is lost. We cast symmetry determination as decoding through a noisy measurement channel and measure its capacity directly. On our real-data benchmark, a Bayes-optimal decoder that is given the pattern of systematically absent reflections recovers 67% Top-1 at the benchmark’s class distribution, yet end-to-end model accuracy is only ∼10%. An explicit twostage decomposition localizes nearly all of this gap to identifying truly absent reflections in noisy, degraded diffraction patterns. Once the absence pattern is known, mapping it to the correct symmetry contributes little additional error. To probe this limit with a strong decoder, we introduce a physics-informed transformer with an explicit sin 2 θ coordinate channel, physics-aware positional encoding, and multi-task outputs that separate geometric rule learning from holistic pattern recognition. We train the model on a synthetic-distribution curriculum chosen to reduce train–test overlap risk. This model is competitive with or stronger than published baselines and still hits the same wall, showing that the limit is structural. We also show that several recent state-of-the-art models share this wall. Mapping residual errors onto the directed acyclic graph of maximal symmetry-reducing (translationengleiche) subgroups shows physically structured errors that are local on the hierarchy and biased toward lower-symmetry descendants, a signature of erased absence cues. The same hierarchy makes classical Pawley verification tractable by bounding the search. Further improvements in automated symmetry determination will require more reliable recovery of systematic absences from experimental data.