Inverse design has transformed vast physical parameter spaces into a substrate for emergent functionality, raising the tantalizing prospect of migrating intelligence from the digital domain into the physical world itself. Sensing is a natural first setting for such a migration program, because the analog-to-digital interface imposes a hard bottleneck: by the data-processing inequality, information not captured by the hardware is irrecoverable by any downstream algorithm. Existing approaches improve information capture through either sensor hardware optimization or adaptive measurement strategies operating on fixed hardware, but rarely both in concert. On our benchmarks, the criterion for a good sensing geometry depends on the policy that will follow it: a geometry selected by a fixed-schedule or clairvoyant proxy forfeits a large fraction of the attainable adaptive value. We formulate the joint problem as joint dynamic programming (joint-DP), a unified optimization over sensor geometry and an adaptive measurement policy defined by a Bellman recursion. A hierarchy of relaxations, which trades Bellman optimality of the inner policy for scale while keeping the outer hardware gradient exact, carries the framework from small discrete POMDPs to freeform photonic topologies with 90,000 design pixels.