- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Letter
  • Open Access

Accelerated motional cooling with deep reinforcement learning

Bijita Sarma*, Sangkha Borah, A Kani, and Jason Twamley

  • Okinawa Institute of Science and Technology Graduate University, Okinawa 904-0495, Japan

  • *bijita.sarma@oist.jp
PDF

Phys. Rev. Research 4, L042038 – Published 29 November, 2022

DOI: https://doi.org/10.1103/PhysRevResearch.4.L042038

Abstract

Achieving fast cooling of motional modes is a prerequisite for leveraging such bosonic quanta for high-speed quantum information processing. In this Letter, we address the aspect of reducing the time limit for cooling, below that constrained by the conventional sideband cooling techniques, and propose a scheme to apply deep reinforcement learning (DRL) to achieve this. In particular, we have numerically demonstrated how the scheme can be used effectively to accelerate the dynamic motional cooling of a macroscopic magnonic sphere, and how it can be uniformly extended to more complex systems, for example, a tripartite opto-magno-mechanical system, to obtain cooling of the motional mode below the time bound of coherent cooling. While conventional sideband cooling methods do not work beyond the well-known rotating wave approximation (RWA) regimes, our proposed DRL scheme can be applied uniformly to regimes operating within and beyond the RWA, and thus, this offers a new and complete toolkit for rapid control and generation of macroscopic quantum states for application in quantum technologies.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (58)

  1. C. Schäfermeier, H. Kerdoncuff, U. B. Hoff, H. Fu, A. Huck, J. Bilek, G. I. Harris, W. P. Bowen, T. Gehring, and U. L. Andersen, Quantum enhanced feedback cooling of a mechanical oscillator using nonclassical light, Nat. Commun. 7, 13628 (2016).
  2. J. Guo, R. Norte, and S. Gröblacher, Feedback Cooling of a Room Temperature Mechanical Oscillator close to its Motional Ground State, Phys. Rev. Lett. 123, 223602 (2019).
  3. Y.-S. Park and H. Wang, Resolved-sideband and cryogenic cooling of an optomechanical resonator - Nature Physics, Nat. Phys. 5, 489 (2009).
  4. J. B. Clark, F. Lecocq, R. W. Simmonds, J. Aumentado, and J. D. Teufel, Sideband cooling beyond the quantum backaction limit with squeezed light - Nature, Nature (London) 541, 191 (2017).
  5. M. Frimmer, J. Gieseler, and L. Novotny, Cooling Mechanical Oscillators by Coherent Control, Phys. Rev. Lett. 117, 163601 (2016).
  6. M. Abdi, P. Degenfeld-Schonburg, M. Sameti, C. Navarrete-Benlloch, and M. J. Hartmann, Dissipative Optomechanical Preparation of Macroscopic Quantum Superposition States, Phys. Rev. Lett. 116, 233604 (2016).
  7. S. Bose, K. Jacobs, and P. L. Knight, Scheme to probe the decoherence of a macroscopic object, Phys. Rev. A 59, 3204 (1999).
  8. W. Marshall, C. Simon, R. Penrose, and D. Bouwmeester, Towards Quantum Superpositions of a Mirror, Phys. Rev. Lett. 91, 130401 (2003).
  9. A. Schliesser, O. Arcizet, R. Rivière, G. Anetsberger, and T. J. Kippenberg, Resolved-sideband cooling and position measurement of a micromechanical oscillator close to the Heisenberg uncertainty limit - Nature Physics, Nat. Phys. 5, 509 (2009).
  10. J. Manley, M. D. Chowdhury, D. Grin, S. Singh, and D. J. Wilson, Searching for Vector Dark Matter with an Optomechanical Accelerometer, Phys. Rev. Lett. 126, 061301 (2021).
  11. J. E. Bourassa, N. Quesada, I. Tzitrin, A. Száva, T. Isacsson, J. Izaac, K. K. Sabapathy, G. Dauphinais, and I. Dhand, Fast simulation of bosonic qubits via gaussian functions in phase space, PRX Quantum 2, 040315 (2021).
  12. X. Zhang, C.-L. Zou, L. Jiang, and H. X. Tang, Cavity magnomechanics, Sci. Adv. 2, e1501286 (2016).
  13. D. Lachance-Quirion, Y. Tabuchi, A. Gloppe, K. Usami, and Y. Nakamura, Hybrid quantum systems based on magnonics, Appl. Phys. Express 12, 070101 (2019).
  14. Y.-P. Wang and C.-M. Hu, Dissipative couplings in cavity magnonics, J. Appl. Phys. 127, 130901 (2020).
  15. J. Li, S.-Y. Zhu, and G. S. Agarwal, Magnon-Photon-Phonon Entanglement in Cavity Magnomechanics, Phys. Rev. Lett. 121, 203601 (2018).
  16. X. Zhang, C.-L. Zou, L. Jiang, and H. X. Tang, Strongly Coupled Magnons and Cavity Microwave Photons, Phys. Rev. Lett. 113, 156401 (2014).
  17. Y.-P. Wang, G.-Q. Zhang, D. Zhang, T.-F. Li, C.-M. Hu, and J. Q. You, Bistability of Cavity Magnon Polaritons, Phys. Rev. Lett. 120, 057202 (2018).
  18. M. Asjad, N. E. Abari, S. Zippilli, and D. Vitali, Optomechanical cooling with intracavity squeezed light, Opt. Express 27, 32427 (2019).
  19. J.-H. Gan, Y.-C. Liu, C. Lu, X. Wang, M. K. Tey, and L. You, Intracavity-squeezed optomechanical cooling, Laser Photonics Rev. 13, 1900120 (2019).
  20. H.-K. Lau and A. A. Clerk, Ground-State Cooling and High-Fidelity Quantum Transduction via Parametrically Driven Bad-Cavity Optomechanics, Phys. Rev. Lett. 124, 103602 (2020).
  21. A. Joshi, K. Noh, and Y. Y. Gao, Quantum information processing with bosonic qubits in circuit qed, Quantum Sci. Technol. 6, 033001 (2021).
  22. I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep Learning, (MIT Press, Cambridge, 2016), Vol 1.
  23. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, The MIT Press, 2 edition, 2018.
  24. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis, Mastering the game of Go with deep neural networks and tree search, Nature (London) 529, 484 (2016).
  25. D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, Y. Chen, T. Lillicrap, F. Hui, L. Sifre, G. van den Driessche, T. Graepel, and D. Hassabis, Mastering the game of Go without human knowledge, Nature (London) 550, 354 (2017).
  26. G. Carleo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld, N. Tishby, L. Vogt-Maranto, and L. Zdeborová, Machine learning and the physical sciences, Rev. Mod. Phys. 91, 045002 (2019).
  27. M. Bukov, A. G. R. Day, D. Sels, P. Weinberg, A. Polkovnikov, and P. Mehta, Reinforcement Learning in Different Phases of Quantum Control, Phys. Rev. X 8, 031086 (2018).
  28. T. Fösel, P. Tighineanu, T. Weiss, and F. Marquardt, Reinforcement Learning with Neural Networks for Quantum Feedback, Phys. Rev. X 8, 031084 (2018).
  29. S. Borah, B. Sarma, M. Kewming, G. J. Milburn, and J. Twamley, Measurement-Based Feedback Quantum Control with Deep Reinforcement Learning for a Double-Well Nonlinear Potential, Phys. Rev. Lett. 127, 190403 (2021).
  30. C. Sommer, M. Asjad, and C. Genes, Prospects of reinforcement learning for the simultaneous damping of many mechanical modes, Sci. Rep. 10, 1 (2020).
  31. Z. T. Wang, Y. Ashida, and M. Ueda, Deep Reinforcement Learning Control of Quantum Cartpoles, Phys. Rev. Lett. 125, 100401 (2020).
  32. M. Y. Niu, S. Boixo, V. N. Smelyanskiy, and H. Neven, Universal quantum control through deep reinforcement learning, npj Quantum Inf. 5, 33 (2019).
  33. X.-M. Zhang, Z. Wei, R. Asad, X.-C. Yang, and X. Wang, When does reinforcement learning stand out in quantum control? A comparative study on state preparation, npj Quantum Inf. 5, 85 (2019).
  34. H. Xu, L. Wang, H. Yuan, and X. Wang, Generalizable control for multiparameter quantum metrology, Phys. Rev. A 103, 042615 (2021).
  35. J. Mackeprang, D. B. R. Dasari, and J. Wrachtrup, A reinforcement learning approach for quantum state engineering, Quantum Mach. Intell. 2, 5 (2020).
  36. T. Haug, W.-K. Mok, J.-B. You, W. Zhang, C. E. Png, and L.-C. Kwek, Classifying global state preparation via deep reinforcement learning, Mach. Learn.: Sci. Technol. 2, 01LT02 (2021).
  37. S.-F. Guo, F. Chen, Q. Liu, M. Xue, J.-J. Chen, J.-H. Cao, T.-W. Mao, M. K. Tey, and L. You, Faster State Preparation across Quantum Phase Transition Assisted by Reinforcement Learning, Phys. Rev. Lett. 126, 060401 (2021).
  38. M. Bilkis, M. Rosati, R. M. Yepes, and J. Calsamiglia, Real-time calibration of coherent-state receivers: Learning by trial and error, Phys. Rev. Res. 2, 033295 (2020).
  39. R. Porotti, D. Tamascelli, M. Restelli, and E. Prati, Coherent transport of quantum states by deep reinforcement learning, Commun. Phys. 2, 61 (2019).
  40. Y. Tabuchi, S. Ishino, T. Ishikawa, R. Yamazaki, K. Usami, and Y. Nakamura, Hybridizing Ferromagnetic Magnons and Microwave Photons in the Quantum Limit, Phys. Rev. Lett. 113, 083603 (2014).
  41. See Supplemental Material at http://link.aps.org/supplemental/10.1103/PhysRevResearch.4.L042038 for a brief theory of reinforcement learning, technical details of the implementation and additional results, which includes Refs.  [12, 15, 16, 17, 23, 24, 25, 40, 42, 49, 50, 52, 53, 54, 55, 56, 57, 58].
  42. T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, arXiv:1801.01290 (2018).
  43. T. Seberson, T. Seberson, P. Ju, J. Ahn, J. Bang, T. Li, T. Li, T. Li, T. Li, F. Robicheaux, and F. Robicheaux, Simulation of sympathetic cooling an optically levitated magnetic nanoparticle via coupling to a cold atomic gas, J. Opt. Soc. Am. B 37, 3714 (2020).
  44. C. C. Rusconi, V. Pöchhacker, K. Kustura, J. I. Cirac, and O. Romero-Isart, Quantum Spin Stabilized Magnetic Levitation, Phys. Rev. Lett. 119, 167202 (2017).
  45. P. Huillery, T. Delord, L. Nicolas, M. Van Den Bossche, M. Perdriat, and G. Hétet, Spin mechanics with levitating ferromagnetic particles, Phys. Rev. B 101, 134415 (2020).
  46. S. Forstner, E. Sheridan, J. Knittel, C. L. Humphreys, G. A. Brawley, H. Rubinsztein-Dunlop, and W. P. Bowen, Ultrasensitive optomechanical magnetometry, Adv. Mater. 26, 6348 (2014).
  47. K. Xia, M. R. Vanner, and J. Twamley, An opto-magneto-mechanical quantum interface between distant superconducting qubits, Sci. Rep. 4, 5571 (2015).
  48. C. Yu, J. Janousek, E. Sheridan, D. L. McAuslan, H. Rubinsztein-Dunlop, P. K. Lam, Y. Zhang, and W. P. Bowen, Optomechanical Magnetometry with a Macroscopic Resonator, Phys. Rev. Appl. 5, 044007 (2016).
  49. T. M. Hoang, J. Ahn, J. Bang, and T. Li, Electron spin control of optically levitated nanodiamonds in vacuum, Nat. Commun. 7, 12250 (2016).
  50. T. Delord, P. Huillery, L. Schwab, L. Nicolas, L. Lecordier, and G. Hétet, Ramsey Interferences and Spin Echoes from Electron Spins Inside a Levitating Macroscopic Particle, Phys. Rev. Lett. 121, 053602 (2018).
  51. K. E. Khosla, G. A. Brawley, M. R. Vanner, and W. P. Bowen, Quantum optomechanics beyond the quantum coherent oscillation regime, Optica 4, 1382 (2017).
  52. J. Achiam, Spinning Up in Deep Reinforcement Learning, 2021, [Online; accessed 15. Jul. 2021].
  53. G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, OpenAI Gym, arXiv:1606.01540 (2016).
  54. A. Raffin, A. Hill, M. Ernestus, A. Gleave, A. Kanervisto, and N. Dormann, Stable baselines3, https://github.com/DLR-RM/stable-baselines3, 2019.
  55. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai et al., Pytorch: An imperative style, high-performance deep learning library, F. d' Alché-Buc, and editors, Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc., 2019.
  56. P. C. Fletcher and R. O. Bell, Ferrimagnetic resonance modes in spheres, J. Appl. Phys. 30, 687 (1959).
  57. Y.-D. Wang and A. A. Clerk, Using Interference for High Fidelity Quantum State Transfer in Optomechanics, Phys. Rev. Lett. 108, 153603 (2012).
  58. K. Bergmann, H. Theuer, and B. W. Shore, Coherent population transfer among quantum states of atoms and molecules, Rev. Mod. Phys. 70, 1003 (1998).

Outline

Information

Phys. Rev. Research 4, L042038– Published 29 November, 2022
  • Received 28 January 2022
  • Accepted 1 November 2022
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation