The laws of thermodynamics are a cornerstone for describing nanoscale and open quantum systems. However, formulating these laws for systems under continuous feedback control and under experimentally relevant conditions is challenging. In this work, we lay out a formalism for the laws of thermodynamics in an open quantum system under continuous measurement and feedback described by a Quantum Fokker Planck Master Equation. We derive expressions for work, heat, and measurement-induced energy changes, and we investigate entropy production and fluctuation theorems. We illustrate our results with a continuous version of a measurement-driven Szilard engine, as well as a work extraction scheme in a two-level system under bang-bang control. Our results provide insights into the energetics as well as the irreversibility of classical and quantum systems under continuous feedback control.