Low-Reynolds-Number Locomotion via Reinforcement Learning

Author

Zonghao Zou

Date of Award

5-2022

Document Type

Thesis - SCU Access Only

Publisher

Santa Clara : Santa Clara University, 2022.

Degree Name

Master of Science (MS)

Department

Mechanical Engineering

First Advisor

On Shun Pak

Abstract

There are growing interests in the development of artificial microscopic machines that can perform complex maneuvers like swimming microorganisms for potential biomedical applications. At the microscopic scales, the dominance of viscous over inertial forces imposes stringent constraints on locomotion. More recently, reinforcement learning has been used as an alternative approach to enable a machine to learn effective locomotory gaits for net translation based on its interaction with the surroundings. In this thesis, we first demonstrates the use of reinforcement learning to generate net mechanical rotation at low Reynolds numbers without requiring prior knowledge of locomotion. For a three-sphere configuration, the reinforcement learning recovers the strategy proposed by Dreyfus et al.. As the number of spheres increases, multiple effective rotational strategies emerge from the learning process. However, given sufficiently long learning processes, all machines considered converge to a single type of rotational policies that consist of traveling waves of actuation, suggesting its optimality of the strategy in generating net rotation at low Reynolds numbers. After demonstrating the capability to produce net mechanical rotation, we will next focus on utilizing deep reinforcement learning to develop effective locomotory strategies for translation, rotation and combined motions. The AI-powered swimmer can switch between various locomotory gaits adaptively to navigate towards target locations. The multimodal navigation strategy is reminiscent of gait-switching behaviors adopted by swimming microorganisms. We show that the strategy advised by AI is robust to flow perturbations and versatile in enabling the swimmer to perform complex tasks such as path tracing without being explicitly programmed. Taken together, our results demonstrate the vast potential of these AI-powered swimmers for applications in unpredictable, complex fluid environments.

Recommended Citation

Zou, Zonghao, "Low-Reynolds-Number Locomotion via Reinforcement Learning" (2022). Mechanical Engineering Master's Theses. 46.
https://scholarcommons.scu.edu/mech_mstr/46