AI Case Study
Researchers at Harvard University and the University of Tübingen track animals’ movements in the lab using deep learning
Researchers at Harvard University and the University of Tübingen leveraged deep learning to track animals in labs and understand their behaviour. They developed a software, DeepLabCut, which tracks features from the digits of mice based on transfer learning with deep neural networks. The software has achieved excellent results with minimal training data, according to the scientists.
Industry
Public And Social Sector
Education And Academia
Project Overview
"Now, a team of researchers from the Rowland Institute at Harvard, Harvard University, and the University of Tübingen is turning to artificial intelligence technology to solve the problem.
The software they developed, dubbed DeepLabCut, harnesses new learning techniques to track features from the digits of mice, to egg-laying behavior in Drosophila, and beyond.
The solution came in what is called “transfer learning,” or applying an already-trained network to a different problem, similar to the way scientists believe biological systems learn.
Using a state-of-the-art algorithm for tracking human movement called DeeperCut, the Mathises were able to show that deep learning could be highly data-efficient. The new software’s name is a nod to DeeperCut’s authors.
Just as a child does not need to develop another visual system from scratch in order to recognize a novel object, but relies on thousands of hours of experience and adapts them to recognize new objects, DeepLabCut is pretrained on thousands of images containing natural objects, images of hammers, cats, dogs, foods, and more.
With that pretraining in place, the software needed only 100 examples of mice performing an odor-guided navigation experiment to recognize specific mouse body parts as well as humans could.
The team was also able to apply the technology to mice making reaching movements, and, in collaboration with Kevin Cury, a neuroscientist from Columbia University, to flies laying eggs in a 3-D chamber.
The software toolbox can be used with minimal to no coding experience and is freely available at mousemotorlab.org/deeplabcut.
Reported Results
"“We were very impressed by the success of the transfer-learning approach and the versatility of DeepLabCut,” Mackenzie Mathis said. “With only a few hundred frames of training data, we were able to get accurate and robust tracking across a myriad of experimental conditions, animals, and behaviors.”"
Technology
Function
R And D
Core Research And Development
Background
"Understanding the brain, in part, means understanding how behavior is created. To reverse-engineer how neural circuits drive behavior requires accurate and vigorous tracking of behavior, yet the increasingly complex tasks animals perform in the laboratory have made that challenging.
The notion of using software to track animal movements was born partly of necessity. Both Mackenzie and Alexander Mathis had tried using traditional techniques, which typically involve placing tracking markers on animals and using heuristics such as object segmentation, with mixed success.
Such techniques are often sensitive to the choice of analysis parameters, and markers or tattoos are invasive and can hinder natural behaviors, or may be impossible to place on very small or wild animals, they said."
Benefits
Data