AI-integrated microscopes perform quicker imaging and handle more data

Scientists have begun using a method known as light-field microscopy to capture certain rapid biological processes in 3D to identify the swift neuronal signals in a fish brain. However, the images are usually of poor quality, and it can take days or hours to translate large quantities of data into 3D volumes and movies.

A representation of a neural network provides a backdrop to a fish larva’s beating heart. Image Credit: Tobias Wüstefeld.

EMBL researchers have now merged artificial intelligence (AI) algorithms with two cutting-edge microscopy methods—a breakthrough that reduces image processing time from days to mere seconds while confirming that the resulting images are precise and crisp. The results were published in the journal Nature Methods.

Ultimately, we were able to take ‘the best of both worlds’ in this approach. AI enabled us to combine different microscopy techniques, so that we could image as fast as light-field microscopy allows and get close to the image resolution of light-sheet microscopy.”

Nils Wagner, PhD Student, Technical University of Munich

Wagner is one of the two lead authors of the study. While light-sheet and light-field microscopy sound identical, they have distinct challenges and advantages.

Light-field microscopy produces large 3D images that enable researchers to quantify and detect extraordinarily fine motions at high speeds, like a fish larva’s beating heart. However, this method generates large volumes of data that can take days to process, and the resulting images are typically low in resolution.

Researchers can photograph samples at higher resolution by focusing on a single 2D plane of a given sample at a time using light-sheet microscopy. Light-sheet microscopy provides images that are faster to process than light-field microscopy, but the evidence is not as detailed since it only captures data from a single 2D plane at a time.

To focus on the advantages of each technique, EMBL researchers devised a method that employs light-field microscopy to image large 3D samples and light-sheet microscopy to develop AI algorithms, which then generate an accurate 3D image of the sample.

“If you build algorithms that produce an image, you need to check that these algorithms are constructing the right image. In the new study, the researchers used light-sheet microscopy to make sure the AI algorithms were working. This makes our research stand out from what has been done in the past,” said Anna Kreshuk, the group leader at EMBL whose team contributed machine learning skills to the project.

According to Robert Prevedel, the EMBL group leader whose group introduced the novel hybrid microscopy platform, the real bottleneck in creating better microscopes is always computing rather than optics technology. That’s why, in 2018, he and Anna wanted to team up.

Our method will be really key for people who want to study how brains compute. Our method can image an entire brain of a fish larva, in real time.”

Robert Prevedel, Group Leader, European Molecular Biology Laboratory

He and Anna believe that this technique should be adapted to fit with various kinds of microscopes, helping scientists to examine dozens of different samples and see even more, much quicker.

It may, for example, aid in the discovery of genes involved in heart development or in the simultaneous measurement of the activity of thousands of neurons.

The researchers plan to investigate whether the technique can be extended to larger species, such as mammals, in the future. Fynn Beuttenmüller, a Ph.D. student and also study co-lead author in the Kreshuk group at EMBL Heidelberg, has no doubts about AI’s strength.

Computational methods will continue to bring exciting advances to microscopy.”

Fynn Beuttenmüller, Study Co-Lead Author and PhD Student, Kreshuk group, European Molecular Biology Laboratory Heidelberg