- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Letter
  • Open Access

Optimal control of quantum thermal machines using machine learning

Ilia Khait1, Juan Carrasquilla2,3, and Dvira Segal1,4,*

  • 1Department of Physics, University of Toronto, 60 Saint George St., Toronto, Ontario M5S 1A7, Canada
  • 2Vector Institute, MaRS Centre, Toronto, Ontario M5G 1M1, Canada
  • 3Department of Physics & Astronomy, University of Waterloo, Waterloo, Ontario N2L 3G1, Canada
  • 4Department of Chemistry and Centre for Quantum Information and Quantum Control, University of Toronto, 80 Saint George St., Toronto, Ontario M5S 3H6, Canada

  • *dvira.segal@utoronto.ca
PDF

Phys. Rev. Research 4, L012029 – Published 11 March, 2022

DOI: https://doi.org/10.1103/PhysRevResearch.4.L012029

Abstract

We develop a deep learning (DL) framework assisted by differentiable programming for discovery of optimal quantum control protocols under hard constraints. To that end, we use neural network representations to our protocols, whose learning process is done with exact gradients. We find high-quality solutions to the optimization problem of finite-time thermodynamical process in a quantum thermal machine. Using this DL algorithm, we show that a previously employed, intuitive energetic cost of the thermal machine driving suffers from a fundamental flaw, which we resolve with an alternative construction for the cost function. Our DL-quantum control framework can be utilized to solve other quantum dynamics and thermodynamics problems.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (80)

  1. C. Lanczos, The Variational Principles of Mechanics (University of Toronto Press, Toronto, 1970).
  2. A. Lipson, S. G. Lipson, and H. Lipson, Optical Physics (Cambridge University Press, Cambridge, 2010), 4th ed.
  3. D. A. Fedorov, B. Peng, N. Govind, and Y. Alexeev, VQE method: A short survey and recent developments Mater. Theory 6, 2 (2022).
  4. J. Gemmer, M. Michel, and G. Mahler, Quantum Thermodynamics: Emergence of Thermodynamic Behavior within Composite Quantum Systems (Springer, Berlin, Heidelberg, 2009), 2nd ed., Vol. 784.
  5. S. C. Kaushik, S. K. Tyagi, and P. Kumar, Finite Time Thermodynamics of Power and Refrigeration Cycles (Springer International Publishing, Cham, Switzerland, 2018).
  6. M. Esposito, R. Kawai, K. Lindenberg, and C. Van den Broeck, Efficiency at Maximum Power of Low-Dissipation Carnot Engines, Phys. Rev. Lett. 105, 150603 (2010).
  7. M. Esposito, R. Kawai, K. Lindenberg, and C. Van den Broeck, Finite-time thermodynamics for a single-level quantum dot, Europhys. Lett. 89, 20003 (2010).
  8. M. I. Kamien and N. L. Schwartz, Dynamic Optimization: The Calculus of Variations and Optimal Control in Economics and Management, 2nd ed. (Dover Publications, Mineola, 2013), https://books.google.ca/books?id=liLCAgAAQBAJ.
  9. Y. Rezek, P. Salamon, K. H. Hoffmann, and R. Kosloff, The quantum refrigerator: The quest for absolute zero, Europhys. Lett. 85, 30008 (2009).
  10. K. H. Hoffmann, P. Salamon, Y. Rezek, and R. Kosloff, Time-optimal controls for frictionless cooling in harmonic traps, Europhys. Lett. 96, 60015 (2011).
  11. P. Doria, T. Calarco, and S. Montangero, Optimal Control Technique for Many-Body Quantum Dynamics, Phys. Rev. Lett. 106, 190501 (2011).
  12. O. Abah and E. Lutz, Optimal performance of a quantum Otto refrigerator, Europhys. Lett. 113, 60002 (2016).
  13. T. Fösel, P. Tighineanu, T. Weiss, and F. Marquardt, Reinforcement Learning with Neural Networks for Quantum Feedback, Phys. Rev. X 8, 031084 (2018).
  14. Z. T. Wang, Y. Ashida, and M. Ueda, Deep Reinforcement Learning Control of Quantum Cartpoles, Phys. Rev. Lett. 125, 100401 (2020).
  15. M. Y. Niu, S. Boixo, V. N. Smelyanskiy, and H. Neven, Universal quantum control through deep reinforcement learning, npj Quantum Inf. 5, 33 (2019).
  16. R. Porotti, D. Tamascelli, M. Restelli, and E. Prati, Coherent transport of quantum states by deep reinforcement learning, Commun. Phys. 2, 61 (2019).
  17. M. Dalgaard, F. Motzoi, J. J. Sørensen, and J. Sherson, Global optimization of quantum dynamics with AlphaZero deep exploration, npj Quantum Inf. 6, 6 (2020).
  18. V. Nguyen, S. Orbell, D. T. Lennon, H. Moon, F. Vigneau, L. C. Camenzind, L. Yu, D. M. Zumbühl, G. A. D. Briggs, M. A. Osborne, and N. Ares, Deep reinforcement learning for efficient measurement of quantum devices, npj Quantum Inf. 7, 100 (2021).
  19. V. B. Sørdal and J. Bergli, Deep reinforcement learning for quantum Szilard engine optimization, Phys. Rev. A 100, 042314 (2019).
  20. N. D. Pozza, L. Buffoni, S. Martina, and F. Caruso, Quantum reinforcement learning: The maze problem, arXiv:2108.04490.
  21. J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne et al., jax: composable transformations of Python+NumPy programs (2018), http://github.com/google/jax.
  22. A. Choromanska, M. Henaff, M. Mathieu, G. B. Arous, and Y. LeCun, The loss surfaces of multilayer networks, arXiv:1412.0233.
  23. F. Hioe, Theory of generalized adiabatic following in multilevel systems, Phys. Lett. A 99, 150 (1983).
  24. T. A. Laine and S. Stenholm, Adiabatic processes in three-level systems, Phys. Rev. A 53, 2501 (1996).
  25. A. A. Melnikov, H. Poulsen Nautrup, M. Krenn, V. Dunjko, M. Tiersch, A. Zeilinger, and H. J. Briegel, Active learning machine learns to create new quantum experiments, Proc. Natl. Acad. Sci. USA 115, 1221 (2018).
  26. M. Bukov, A. G. R. Day, D. Sels, P. Weinberg, A. Polkovnikov, and P. Mehta, Reinforcement Learning in Different Phases of Quantum Control, Phys. Rev. X 8, 031086 (2018).
  27. J. Wallnöfer, A. A. Melnikov, W. Dür, and H. J. Briegel, Machine learning for long-distance quantum communication, PRX Quantum 1, 010301 (2020).
  28. C. Beeler, U. Yahorau, R. Coles, K. Mills, S. Whitelam, and I. Tamblyn, Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning, Phys. Rev. E 104, 064128 (2021).
  29. A. Jasinski, J. Montaner, R. C. Forrey, B. H. Yang, P. C. Stancil, N. Balakrishnan, J. Dai, R. A. Vargas-Hernández, and R. V. Krems, Machine learning corrected quantum dynamics calculations, Phys. Rev. Research 2, 032051 (2020).
  30. S. Deffner and S. Campbell, Quantum Thermodynamics: An Introduction to the Thermodynamics of Quantum Information (Morgan & Claypool Publishers, San Rafael, 2019).
  31. S. Bhattacharjee and A. Dutta, Quantum thermal machines and batteries, Eur. Phys. J. B 94, 239 (2021).
  32. R. Kosloff and A. Levy, Quantum heat engines and refrigerators: continuous devices, Annu. Rev. Phys. Chem. 65, 365 (2014).
  33. S. Vinjanampathy and J. Anders, Quantum thermodynamics, Contemp. Phys. 57, 545 (2016).
  34. A. Das and V. Mukherjee, Quantum-enhanced finite-time Otto cycle, Phys. Rev. Research 2, 033083 (2020).
  35. J. Klatzow, J. N. Becker, P. M. Ledingham, C. Weinzetl, K. T. Kaczmarek, D. J. Saunders, J. Nunn, I. A. Walmsley, R. Uzdin, and E. Poem, Experimental Demonstration of Quantum Effects in the Operation of Microscopic Heat Engines, Phys. Rev. Lett. 122, 110601 (2019).
  36. R. B. S, V. Mukherjee, U. Divakaran, and A. del Campo, Universal finite-time thermodynamics of many-body quantum machines from Kibble-Zurek scaling, Phys. Rev. Research 2, 043247 (2020).
  37. X. Chen, A. Ruschhaupt, S. Schmidt, A. del Campo, D. Guéry-Odelin, and J. G. Muga, Fast Optimal Frictionless Atom Cooling in Harmonic Traps: Shortcut to Adiabaticity, Phys. Rev. Lett. 104, 063002 (2010).
  38. J. Deng, Q.-h. Wang, Z. Liu, P. Hänggi, and J. Gong, Boosting work characteristics and overall heat-engine performance via shortcuts to adiabaticity: quantum and classical systems, Phys. Rev. E 88, 062122 (2013).
  39. E. Torrontegui, S. Ibáñez, S. Martínez-Garaot, M. Modugno, A. del Campo, D. Guéry-Odelin, A. Ruschhaupt, X. Chen, and J. G. Muga, Chapter 2: Shortcuts to adiabaticity, Adv. At. Mol. Opt. Phys. 62, 117 (2013).
  40. X. Chen and J. G. Muga, Transient energy excitation in shortcuts to adiabaticity for the time-dependent harmonic oscillator, Phys. Rev. A 82, 053403 (2010).
  41. J. G. Muga, X. Chen, S. Ibáñez, I. Lizuain, and A. Ruschhaupt, Transitionless quantum drivings for the harmonic oscillator, J. Phys. B: At., Mol. Opt. Phys. 43, 085509 (2010).
  42. Y.-Y. Cui, X. Chen, and J. G. Muga, Transient particle energies in shortcuts to adiabatic expansions of harmonic traps, J. Phys. Chem. A 120, 2962 (2016).
  43. O. Abah, J. Roßnagel, G. Jacob, S. Deffner, F. Schmidt-Kaler, K. Singer, and E. Lutz, Single-Ion Heat Engine at Maximum Power, Phys. Rev. Lett. 109, 203006 (2012).
  44. A. C. Santos and M. S. Sarandy, Superadiabatic controlled evolutions and universal quantum computation, Sci. Rep. 5, 15775 (2015).
  45. R. Kosloff and Y. Rezek, The quantum harmonic otto cycle, Entropy 19, 136 (2017).
  46. K. Funo, N. Lambert, B. Karimi, J. P. Pekola, Y. Masuyama, and F. Nori, Speeding up a quantum refrigerator via counterdiabatic driving, Phys. Rev. B 100, 035407 (2019).
  47. D. Guéry-Odelin, A. Ruschhaupt, A. Kiely, E. Torrontegui, S. Martínez-Garaot, and J. G. Muga, Energy consumption for shortcuts to adiabaticity, Rev. Mod. Phys. 91, 045001 (2019).
  48. H. Zhou, Y. Ji, X. Nie, X. Yang, X. Chen, J. Bian, and X. Peng, Experimental Realization of Shortcuts to Adiabaticity in a Nonintegrable Spin Chain by Local Counterdiabatic Driving, Phys. Rev. Applied 13, 044059 (2020).
  49. K. Ono, S. N. Shevchenko, T. Mori, S. Moriyama, and F. Nori, Analog of a Quantum Heat Engine Using a Single-Spin Qubit, Phys. Rev. Lett. 125, 166802 (2020).
  50. L. Dupays, D. C. Spierings, A. M. Steinberg, and A. del Campo Delta-kick cooling, time-optimal control of scale-invariant dynamics, and shortcuts to adiabaticity assisted by kicks, Phys. Rev. Research 3, 033261 (2021).
  51. J. Carrasquilla, Machine learning for quantum matter, Adv. Phys.: X 5, 1797528 (2020).
  52. A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind, Automatic differentiation in machine learning: a Survey, J. Mach. Learning Res. 18, 1 (2018).
  53. F. Schäfer, M. Kloc, C. Bruder, and N. Lörch, A differentiable programming method for quantum control, Mach. Learning: Sci. Technology 1, 035009 (2020).
  54. A. del Campo, Shortcuts to Adiabaticity by Counterdiabatic Driving, Phys. Rev. Lett. 111, 100502 (2013).
  55. M. Beau, J. Jaramillo, and A. Del Campo, Scaling-up quantum heat engines efficiently via shortcuts to adiabaticity, Entropy 18, 168 (2016).
  56. L. Coopmans, D. Luo, G. Kells, B. K. Clark, and J. Carrasquilla, Protocol discovery for the quantum control of Majoranas by differentiable programming and natural evolution strategies, PRX Quantum 2, 020332 (2021).
  57. J. Roßnagel, O. Abah, F. Schmidt-Kaler, K. Singer, and E. Lutz, Nanoscale Heat Engine Beyond the Carnot Limit, Phys. Rev. Lett. 112, 030602 (2014).
  58. J. Roßnagel, S. T. Dawkins, K. N. Tolazzi, O. Abah, E. Lutz, F. Schmidt-Kaler, and K. Singer, A single-atom heat engine, Science 352, 325 (2016).
  59. Y. Rezek and R. Kosloff, Irreversible performance of a quantum harmonic heat engine, New J. Phys. 8, 83 (2006).
  60. M. V. Berry, Transitionless quantum driving, J. Phys. A: Math. Theor. 42, 365303 (2009).
  61. See Supplemental Material at http://link.aps.org/supplemental/10.1103/PhysRevResearch.4.L012029 for derivation of the energetic cost function, discussion of a additional ansatzes and a description of the neural network optimization.
  62. O. Abah and E. Lutz, Performance of shortcut-to-adiabaticity quantum engines, Phys. Rev. E 98, 032121 (2018).
  63. K. Husimi, Miscellanea in elementary quantum mechanics, II, Prog. Theor. Phys. 9, 381 (1953).
  64. M. A. Lohe, Exact time dependence of solutions to the time-dependent Schrödinger equation, J. Phys. A: Math. Theor. 42, 35307 (2009).
  65. Y. Zheng, S. Campbell, G. De Chiara, and D. Poletti, Cost of counterdiabatic driving and work output, Phys. Rev. A 94, 042132 (2016).
  66. S. Campbell and S. Deffner, Quantum Thermodynamics, Phys. Rev. Lett. 118, 100601 (2017).
  67. D. P. Kingma and J. Ba (2014), adam: a method for stochastic optimization, arXiv:1412.6980 (2017).
  68. We use the nonadiabatic parameter QNA*=ω12+ω222ω1ω2, which alters Eqs. (3) and (4) (see Refs. [12, 43] for details).
  69. O. Abah and E. Lutz, Energy efficient quantum machines, Europhys. Lett. 118, 40005 (2017).
  70. O. Abah and M. Paternostro, Shortcut-to-adiabaticity Otto engine: a twist to finite-time thermodynamics, Phys. Rev. E 99, 022110 (2019).
  71. O. Abah, M. Paternostro, and E. Lutz, Shortcut-to-adiabaticity quantum Otto refrigerator, Phys. Rev. Research 2, 023120 (2020).
  72. Criticisms to this approach, albeit from a different aspect, were raised in Ref. [32].
  73. A. Del Campo, J. Goold, and M. Paternostro, More bang for your buck: Super-adiabatic quantum engines, Sci. Rep. 4, 6208 (2014).
  74. K. Funo, J.-N. Zhang, C. Chatou, K. Kim, M. Ueda, and A. del Campo, Universal Work Fluctuations During Shortcuts to Adiabaticity by Counterdiabatic Driving, Phys. Rev. Lett. 118, 100602 (2017).
  75. E. Torrontegui, I. Lizuain, S. González-Resines, A. Tobalina, A. Ruschhaupt, R. Kosloff, and J. G. Muga, Energy consumption for shortcuts to adiabaticity, Phys. Rev. A 96, 022133 (2017).
  76. A. Tobalina, J. Alonso, and J. G. Muga, Energy consumption for ion-transport in a segmented Paul trap, New J. Phys. 20, 065002 (2018).
  77. A. Tobalina, I. Lizuain, and J. G. Muga, Vanishing efficiency of a speeded-up ion-in-Paul-trap Otto engine, Europhys. Lett. 127, 20005 (2019).
  78. P. Sgroi, G. M. Palma, and M. Paternostro, Reinforcement Learning Approach to Nonequilibrium Quantum Thermodynamics, Phys. Rev. Lett. 126, 020601 (2021).
  79. A. F. de Almeida, R. Moreira, and T. Rodrigues, Synthetic organic chemistry driven by artificial intelligence, Nature Rev. Chem. 3, 589 (2019).
  80. M. M. Müller, R. S. Said, F. Jelezko, T. Calarco, and S. Montangero, One decade of quantum optimal control in the chopped random basis, arXiv:2104.07687 (2021).

Outline

Information

Phys. Rev. Research 4, L012029– Published 11 March, 2022
  • Received 16 August 2021
  • Revised 29 November 2021
  • Accepted 25 January 2022
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation