Using deep reinforcement learning to model storm petrel behaviors

Storm petrels are crazy little oceanic birds that often “hop” on the water’s surface. Nobody knows why. It’s a difficult thing to study–the birds are doing this behavior in the middle of the ocean. Do they perform this behavior to save energy via weight support? Do they use their feet as an “anchor” in high wind? We wanted to know if there was a mechanistic reason for this behavior. And we thought that perhaps machine learning could help us test it.

Using MATLABs reinforcement learning toolbox and Simscape, we built a model that could “teach itself” to fly. The boundary conditions were based on the quasi-steady aerodynamics taken from my 2016 morphing wings paper (estimated and scaled to the wings of a petrel) and anatomical data collected of actual petrel specimens I collected at the Burke Museum in Seattle. Youtube videos were used to estimate the kinematics of the wings and legs as the birds pattered across the water.

Sea pattering DRL model
Sea anchoring DRL model

Reward functions were assigned to each sim: pattering (aka forward motion) and sea anchoring (remaining stationary). After several thousand attempts, the models more or less started to look like real birds pattering across the surface of the water.

Our results show that the wings provided most of the lift force while the feet provided the forward force via hydrodynamic drag. Interestingly, at higher wind speeds, the tail appears to take on an outsized role in force production.

I find it really fascinating that we are able to use machine learning to test difficult behavioral hypotheses. Assuming that the test organism is trying to maximize efficiency (i.e. cost of transport) and that our model matches the physics of the real world, it seems very reasonable to think we could test all kinds of crazy behaviors.

Unfortunately this paper is paywalled, but I’m happy to email it to you! Please just send me an email.

Leave a Reply

Your email address will not be published. Required fields are marked *