Collective phenomena in fluid environments exhibit fascinating dynamics, observed in fish schools, bird flocks, and bacteria colonies. What are the benefits for individuals or groups in collective swimming? In this paper, we introduce inverse reinforcement learning (IRL) as a new approach to address this question. In IRL, rewards of individuals or a group are inferred through observations of their states and actions. This is in contrast with forward optimization or reinforcement learning approaches, where an objective is specified and states and actions are inferred. The algorithm is demonstrated for a group of four artificial swimmers in a two-dimensional viscous flow described by the Navier-Stokes equations. We examine the capabilities of IRL in the discovery of group and individual objectives in the collective behavior of the swimmers. We demonstrate the ability to distill a reward function from efficient swimming patterns. The reward functions are interpreted using sensitivity analysis and symbolic regression techniques. In this paper, we use synthetic data for observations and trajectories, generated by a policy that is learned first by performing forward reinforcement learning with the goal of maximizing swimming efficiency. The IRL methodologies are readily applicable to real-world and experimental data of natural swimmers. The study introduces a new direction for bioinspired fluid mechanics by, not a priori specification of objective functions, but discovering them from observations of natural and artificial swimmers.

Close

PrevNext

View figure in article

• Biological fluid dynamics
• Swimming
• Vortex dynamics
• Immersed boundary methods
• Machine learning