- Physical Review Journals
All JournalsPhysics Magazine

Physical Review E

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Learning to crawl: Benefits and limits of centralized versus distributed control

Luca Gagliardi* and Agnese Seminara†

  • Machine Learning Genoa Center and Department of Civil Chemical and Environmental Engineering, University of Genoa, villa Cambiaso, via Montallegro 1, 16145, Italy

  • *Contact author: luca.gagliardi@edu.unige.it
  • †Contact author: agnese.seminara@unige.it
PDF

Phys. Rev. E 113, 044404 – Published 3 April, 2026

DOI: https://doi.org/10.1103/9snq-xgqh

Abstract

We present a model of a crawler consisting of several suction units distributed along a straight line and connected by springs. The suction units are rudimentary proprioceptors-actuators, which sense binary states of compression versus elongation of the springs and can either adhere or remain idle. Muscular contraction is not controlled by the crawler but follows an endogenous, stereotyped wave. The crawler is tasked to learn patterns of adhesion that generate thrust in response to the wave of contraction. Using tabular Q-learning we demonstrate that crawling can be learned by trial and error and we ask what are the benefits and limitations of distributed versus centralized learning architectures. We find that by centralizing proprioceptive feedback and control, the crawler leverages long-range correlations in the dynamics and ride the endogenous wave smoothly. The ensuing benefits are measured in terms of both speed and robustness to failure, although they come at increased computational cost. At the opposite extreme, purely distributed feedback and control only leverages local information and yield a jerkier and slower crawling, although computationally cheap. Intermediate levels of centralization can negotiate fast and robust crawling while avoiding excessive computational burden, demonstrating the computational benefits of a hierarchical organization of crawling. Our model unveils the trade-offs among crawling speed, robustness to failure, computational cost, and information exchange that may shape biological solutions for crawling and could inspire the design of robotic crawlers.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (62)

  1. W. N. Frost, Neuroethology: Generating complex behavioral sequences with distributed nerve nets, Curr. Biol. 33, R398 (2023).
  2. J. Bielecki, S. K. Dam Nielsen, G. Nachman, and A. Garm, Associative learning in the box jellyfish Tripedalia cystophora, Curr. Biol. 33, 4150 (2023).
  3. C. A. Freas and K. Cheng, Neuroecology beyond the brain: Learning in Echinodermata, Learn. Behav. 50, 20 (2022).
  4. T. Kano, D. Kanauchi, H. Aonuma, E. G. Clark, and A. Ishiguro, Decentralized control mechanism for determination of moving direction in brittle stars with penta-radially symmetric body, Front. Neurorobot. 13, 66 (2019).
  5. E. G. Clark, D. Kanauchi, T. Kano, H. Aonuma, D. E. G. Briggs, and A. Ishiguro, The function of the ophiuroid nerve ring: How a decentralized nervous system controls coordinated locomotion, J. Exp. Biol. 222, 192104 (2018).
  6. T. Po, E. Kanso, and M. J. McHenry, Cooperative transport in sea star locomotion, Curr. Biol. 34, 2551 (2024).
  7. S. Heydari, A. Johnson, O. Ellers, M. J. McHenry, and E. Kanso, Sea star inspired crawling and bouncing, J. R. Soc. Interface 17, 20190700 (2020).
  8. C. A. Allard, W. A. Valencia-Montoya, and N. W. Bellono, Cephalopod chemotactile sensation, Curr. Biol. 33, R1081 (2023).
  9. J. A. Mather and L. Dickel, Cephalopod complex cognition, Curr. Opin. Behav. Sci. 16, 131 (2017).
  10. G. Levy, T. Flash, and B. Hochner, Arm coordination in octopus crawling involves unique motor control strategies, Curr. Biol. 25, 1195 (2015).
  11. T. Gutnick, R. A. Byrne, B. Hochner, and M. Kuba, Octopus vulgaris uses visual information to determine the location of its arm, Curr. Biol. 21, 460 (2011).
  12. L. Zullo, G. Sumbre, C. Agnisola, T. Flash, and B. Hochner, Nonsomatotopic organization of the higher motor centers in octopus, Curr. Biol. 19, 1632 (2009).
  13. M. Dickinson, C. Farley, R. Full, M. Koehl, R. Kram, and S. Lehman, How animals move: An integrative view, Science 288, 100 (2000).
  14. K. Nishikawa, A. A. Biewener, P. Aerts, A. N. Ahn, H. J. Chiel, M. A. Daley, T. L. Daniel, R. J. Full, M. E. Hale, T. L. Hedrick, et al., Neuromechanics: An integrative approach for understanding motor control, Integr. Comp. Biol. 47, 16 (2007).
  15. A. J. Ijspeert, Central pattern generators for locomotion control in animals and robots: A review, Neural Netw. 21, 642 (2008).
  16. T. Umedachi, K. Takeda, T. Nakagaki, R. Kobayashi, and A. Ishiguro, Fully decentralized control of a soft-bodied robot inspired by true slime mold, Biol. Cybern. 102, 261 (2010).
  17. J. C. Spagna, D. I. Goldman, P.-C. Lin, D. E. Koditschek, and R. J. Full, Distributed mechanical feedback in arthropods and robots simplifies control of rapid running on challenging terrain, Bioinspir. Biomimet. 2, 9 (2007).
  18. I. D. Neveln, A. Tirumalai, and S. Sponberg, Information-based centralization of locomotion in animals and robots, Nat. Commun. 10, 3655 (2019).
  19. K. Yasui, T. Kano, E. M. Standen, H. Aonuma, A. J. Ijspeert, and A. Ishiguro, Decoding the essential interplay between central and peripheral control in adaptive locomotion of amphibious centipedes, Sci. Rep. 9, 18288 (2019).
  20. K. Yasui, A. Gupta, Q. Fu, S. Suzuki, J. Hainer, L. Paez, K. Lutek, J. Arreguit, T. Kano, E. M. Standen, et al., Multisensory feedback makes swimming circuits robust against spinal transection and enables terrestrial crawling in elongate fish, Proc. Natl. Acad. Sci. USA 122, e2422248122 (2025).
  21. S. Suzuki, T. Kano, A. J. Ijspeert, and A. Ishiguro, Spontaneous gait transitions of sprawling quadruped locomotion by sensory-driven body–limb coordination mechanisms, Front. Neurorob. 15, 645731 (2021).
  22. R. J. Full and D. E. Koditschek, Templates and anchors: Neuromechanical hypotheses of legged locomotion on land, J. Exp. Biol. 202, 3325 (1999).
  23. A. Ayali, A. Borgmann, A. Büschges, E. Couzin-Fuchs, S. Daun-Gruhn, and P. Holmes, The comparative investigation of the stick insect and cockroach models in the study of insect locomotion, Curr. Opin. Insect Sci. 12, 1 (2015).
  24. D. E. Koditschek, R. J. Full, and M. Buehler, Mechanical aspects of legged locomotion control, Arthropod Struct. Dev. 33, 251 (2004).
  25. K. Pearson, Ö. Ekeberg, and A. Büschges, Assessing sensory function in locomotor systems using neuro-mechanical simulations, Trends Neurosci. 29, 625 (2006).
  26. S. Mishra, W. M. Van Rees, and L. Mahadevan, Coordinated crawling via reinforcement learning, J. R. Soc. Interface 17, 20200198 (2020).
  27. A. Jan Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal, Dynamical movement primitives: Learning attractor models for motor behaviors, Neural Comput. 25, 328 (2013).
  28. B. Hartl, M. Levin, and A. Zöttl, Neuroevolution of decentralized decision-making in N-bead swimmers leads to scalable and robust collective locomotion, Commun. Phys. 8, 194 (2025).
  29. Y. Tanaka, K. Ito, T. Nakagaki, and R. Kobayashi, Mechanics of peristaltic locomotion and role of anchoring, J. R. Soc. Interface 9, 222 (2012).
  30. B. Chong, J. He, D. Soto, T. Wang, D. Irvine, G. Blekherman, and D. I. Goldman, Multilegged matter transport: A framework for locomotion on noisy landscapes, Science 380, 509 (2023).
  31. S. Grillner, Biological pattern generation: The cellular and computational logic of networks in motion, Neuron 52, 751 (2006).
  32. Q. Wen, M. D. Po, E. Hulme, S. Chen, X. Liu, S. Kwok, M. Gershow, A. Leifer, V. Butler, C. Fang-Yen, et al., Proprioceptive coupling within motor neurons drives C. elegans forward locomotion, Neuron 76, 750 (2012).
  33. J. H. Boyle, S. Berri, and N. Cohen, Gait modulation in C. elegans: An integrated neuromechanical model, Front. Comput. Neurosci. 6, 1 (2012).
  34. C. Pehlevan, P. Paoletti, and L. Mahadevan, Integrative neuromechanics of crawling in D. melanogaster larvae, eLife 5, e11031 (2016).
  35. J. Nakanishi, J. Morimoto, G. Endo, G. Cheng, S. Schaal, and M. Kawato, Learning from demonstration and adaptation of biped locomotion, Rob. Auton. Syst. 47, 79 (2004).
  36. N. G. Hatsopoulos and W. H. W. Jr., Resonance tuning in rhythmic arm movements, J. Mot. Behav. 28, 3 (1996).
  37. J. Lynch, J. Gau, S. Sponberg, and N. Gravish, Dimensional analysis of spring-wing systems reveals performance metrics for resonant flapping-wing flight, J. R. Soc. Interface 18, 20200888 (2021).
  38. J. C. Tuthill and E. Azim, Proprioception, Curr. Biol. 28, R194 (2018).
  39. R. S. Sutton and A. Barto, Reinforcement Learning: An Introduction, 2nd ed., Adaptive Computation and Machine Learning (The MIT Press, Cambridge, MA, 2020).
  40. S. Mahadevan, Average reward reinforcement learning: Foundations, algorithms, and empirical results, Mach. Learn. 22, 159 (1996).
  41. A. Plaat, Deep Reinforcement Learning (Springer Nature, Singapore, 2022).
  42. D. L. Young and C. Eccles, in Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications II, edited by T. Pham, L. Solomon, and K. Rainey (SPIE, Bellingham, WA, 2020), p. 44.
  43. L. Gagliardi, Git repo: Control a 1D crawler (2025), https://github.com/PimLb/control_1D_crawler.git.
  44. See Supplemental Material at http://link.aps.org/supplemental/10.1103/9snq-xgqh for animations of different crawling policies.
  45. W. Stern-Tomlinson, M. Nusbaum, L. Perez, and W. J. Kristan, A kinematic study of crawling behavior in the leech,Hirudo medicinalis, J. Comp. Physiol. A 158, 593 (1986).
  46. B. Hochner, L. Zullo, T. Shomrat, G. Levy, and N. Nesher, Embodied mechanisms of motor control in the octopus, Curr. Biol. 33, R1119 (2023).
  47. S. Chen, Y. Cao, M. Sarparast, H. Yuan, L. Dong, X. Tan, and C. Cao, Soft crawling robots: Design, actuation, and locomotion, Adv. Mater. Technol. 5, 1900837 (2020).
  48. T. Sato, W. Watanabe, and A. Ishiguro, in Proceedings of the IEEE International Conference on Robotics and Automation (IEEE, Los Alamitos, CA, 2010), pp. 709–714.
  49. B. Mazzolai, A. Mondini, F. Tramacere, G. Riccomi, A. Sadeghi, G. Giordano, E. Del Dottore, M. Scaccia, M. Zampato, and S. Carminati, Octopus-inspired soft arm with suction cups for enhanced grasping tasks in confined environments, Adv. Intell. Syst. 1, 1900041 (2019).
  50. S. Sareh, K. Althoefer, M. Li, Y. Noh, F. Tramacere, P. Sareh, B. Mazzolai, and M. Kovac, Anchoring like octopus: biologically inspired soft artificial sucker, J. R. Soc. Interface 14 (2017).
  51. J. Peters and S. Schaal, Reinforcement learning of motor skills with policy gradients, Neural Netw. 21, 682 (2008).
  52. H. V. Hasselt, Double Q-learning, Advances in Neural Information Processing Systems 23, 1 (2010).
  53. M. Kim, J.-S. Kim, M.-S. Choi, and J.-H. Park, Adaptive discount factor for deep reinforcement learning in continuing tasks with uncertainty, Sensors 22, 7266 (2022).
  54. Y. Kuramoto, in International Symposium on Mathematical Problems in Theoretical Physics, edited by H. Araki (Springer-Verlag, Berlin, 1975), Vol. 39, pp. 420–422.
  55. S. H. Strogatz, From Kuramoto to Crawford: Exploring the onset of synchronization in populations of coupled oscillators, Physica D 143, 1 (2000).
  56. G. Jékely, F. Keijzer, and P. Godfrey-Smith, An option space for early neural evolution, Philos. Trans. R. Soc. B 370, 20150181 (2015).
  57. D. Arendt, M. A. Tosches, and H. Marlow, From nerve net to nerve ring, nerve cord and brain—Evolution of the nervous system, Nat. Rev. Neurosci. 17, 61 (2016).
  58. F. Keijzer and A. Arnellos, The animal sensorimotor organization: A challenge for the environmental complexity thesis, Biol. Philos. 32, 421 (2017).
  59. M. J. Wells, Proprioception and visual discrimination of orientation in Octopus, J. Exp. Biol. 37, 489 (1960).
  60. H. Bagheri, A. Hu, S. Cummings, C. Roy, R. Casleton, A. Wan, N. Erjavic, S. Berman, M. M. Peet, D. M. Aukes, et al., New insights on the control and function of octopus suckers, Adv. Intell. Syst. 2, 1900154 (2020).
  61. L. Van Giesen, P. B. Kilian, C. A. Allard, and N. W. Bellono, Molecular basis of chemotactile sensation in octopus, Cell 183, 594 (2020).
  62. L. Gagliardi and A. Seminara, Codes and scripts to reproduce results from present publication (2025), https://github.com/PimLb/control_1D_crawler.

Outline

Information

Phys. Rev. E 113, 044404– Published 3 April, 2026
  • Received 11 June 2025
  • Accepted 26 February 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation