Steering Smart Active Particles via Deep Reinforcement Learning

Active matter consists of entities that harness energy from their surroundings to propel themselves. Throughout evolution, biological active systems—ranging from microorganisms to schools of fish—have developed sophisticated navigation strategies for efficiently locating food, evading predators, and engaging in collective behaviors. There is currently an enormous interest in physics, biology, and engineering, to design autonomous synthetic agents, capable of executing complex tasks such as microsurgery, drug delivery, and environmental remediation. Despite nature offering paradigms of active systems navigating in adverse conditions, we still lack a straightforward approach to replicate and integrate these autonomous behaviors into synthetic counterparts. The overarching objective of this thesis is to formulate systematic methodologies for the design and discovery of optimal navigation strategies for active agents. We leverage the recent advancements in artificial intelligence, specifically deep reinforcement learning, to design "smart" agents that are capable of learning robust strategies solely from their interactions with the surrounding environment. We first tackle the fundamental problem of optimal point-to-point navigation for a self-propelled agent which can freely steer in environments hosting complex flow or force fields. Our method, for the first time, enables the determination of the asymptotically optimal trajectory without reward shaping, offering a robust alternative to traditional analytical techniques. Next we investigate the problem of efficient foraging and design a smart run-and-tumble agent which can "see" and collect nutrients from its surroundings. Our designed agents demonstrate superior foraging efficiency and enhanced survival capabilities compared to conventional strategies for unknown environments. Notably, our smart run-and-tumbler, without any prior knowledge, not only develops motion patterns that closely parallels chemotactic bacteria, but also learns a strikingly similar tumble rate distribution albeit with distinct differences. Finally, we expand from individual agents to collective dynamics by examining multiple intelligent agents tackling the optimal evacuation problem. Here we combine deep reinforcement learning with self-play to enable a crowd of "pressure-aware" agents to collaboratively optimize evacuation for various environments. After training, the model exhibits intriguing combination of remarkably simple, interpretable strategies such as queuing, re-queuing, and zipper-merger dynamics, significantly reducing fatalities and surpassing standard benchmarks for evacuation efficiency. The results and design choices of this thesis establish a robust benchmark for future theoretical developments. Our findings offer a systematic approach to examine whether natural strategies, such as turtle migration or Bacterial foraging, converge toward global optimality. For synthetic agents, our methods can extend to autonomous robots and smart active particles designed for efficient removal of waste, toxins, or microplastics. Finally, our framework for optimal evacuation can be generalized for designing and enhancing public spaces, as well as the rigorous assessment of current safety guidelines in crowd management.