Applications of deep learning in electron microscopy

Determination of the interface between amorphous insulator and crystalline 4H–SiC in transmission electron microscope image by using convolutional neural network

(2021) Hironori Yoshioka et al.   AIP Advances

CDAE: A Cascade of Denoising Autoencoders for Noise Reduction in the Clustering of Single-Particle Cryo-EM Images

(2021) Houchao Lei et al.   Frontiers in Genetics

DeepAlign, a 3D Alignment Method based on Regionalized Deep Learning for Cryo-EM

(2021) A. Jiménez-Moreno et al.   JOURNAL OF STRUCTURAL BIOLOGY

CryoDRGN: reconstruction of heterogeneous cryo-EM structures using neural networks

(2021) Ellen D. Zhong et al.   NATURE METHODS

Anomalous nanoparticle surface diffusion in LCTEM is revealed by deep learning-assisted analysis

(2021) Vida Jamali et al.   PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA

Charting the low-loss region in electron energy loss spectroscopy with machine learning

(2021) Laurien I. Roest et al.   ULTRAMICROSCOPY

CASSPER is a semantic segmentation-based particle picking algorithm for single-particle cryo-electron microscopy

(2021) Blesson George et al.   Communications Biology

Democratising deep learning for microscopy with ZeroCostDL4Mic

(2021) Lucas von Chamier et al.   Nature Communications

Exploring order parameters and dynamic processes in disordered systems via variational autoencoders

(2021) Sergei V. Kalinin et al.   Science Advances

Learning-based defect recognition for quasi-periodic HRSTEM images

(2021) Nik Dennler et al.   MICRON

Alignment of electron optical beam shaping elements using a convolutional neural network

(2021) E. Rotunno et al.   ULTRAMICROSCOPY

ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data

(2020) Foivos I. Diakogiannis et al.   ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING

Practical Considerations for Atomistic Structure Modeling with Cryo-EM Maps

(2020) Doo Nam Kim et al.   Journal of Chemical Information and Modeling

Deep Learning Enabled Strain Mapping of Single-Atom Defects in Two-Dimensional Transition Metal Dichalcogenides with Sub-Picometer Precision

(2020) Chia-Hao Lee et al.   NANO LETTERS

A Brief Survey on Semantic Segmentation with Deep Learning

(2020) Shijie Hao et al.   NEUROCOMPUTING

High-Throughput Cryo-EM Enabled by User-Free Preprocessing Routines

(2020) Yilai Li et al.   STRUCTURE

Deep Learning to Predict Protein Backbone Structure from High-Resolution Cryo-EM Density Maps

(2020) Dong Si et al.   Scientific Reports

Partial Scanning Transmission Electron Microscopy with Deep Learning

(2020) Jeffrey M. Ede et al.   Scientific Reports

AI-enabled high-resolution scanning coherent diffraction imaging

(2020) Mathew J. Cherukara et al.   APPLIED PHYSICS LETTERS

A deep learning approach for determining the chiral indices of carbon nanotubes from high-resolution transmission electron microscopy images

(2020) G.D. Förster et al.   CARBON

Atomic column heights detection in metallic nanoparticles using deep convolutional learning

(2020) Marco Ragone et al.   COMPUTATIONAL MATERIALS SCIENCE

MicrographCleaner: A python package for cryo-EM micrograph cleaning using deep learning

(2020) Ruben Sanchez-Garcia et al.   JOURNAL OF STRUCTURAL BIOLOGY

Machine Learning to Reveal Nanoparticle Dynamics from Liquid-Phase TEM Videos

(2020) Lehan Yao et al.   ACS Central Science

Understanding important features of deep learning models for segmentation of high-resolution transmission electron microscopy images

(2020) James P. Horwath et al.   npj Computational Materials

Towards data-driven next-generation transmission electron microscopy

(2020) Steven R. Spurgeon et al.   NATURE MATERIALS

Analysis of complex, beam-sensitive materials by transmission electron microscopy and associated techniques

(2020) Martha Ilett et al.   PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES

Topaz-Denoise: general deep denoising models for cryoEM and cryoET

(2020) Tristan Bepler et al.   Nature Communications

Enhancing the signal-to-noise ratio and generating contrast for cryo-EM images with convolutional neural networks

(2020) Eugene Palovcak et al.   IUCrJ

Virus identification in electron microscopy images by residual mixed attention network

(2020) Chi Xiao et al.   COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE

DeepTracer for fast de novo cryo-EM protein structure modeling and special studies on CoV-related complexes

(2020) Jonas Pfab et al.   PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA

Deep learning for scanning electron microscopy: Synthetic data for the nanoparticles detection

(2020) A. Yu. Kharin   ULTRAMICROSCOPY

A comprehensive review of past and present image inpainting methods

(2020) Jireh Jam et al.   COMPUTER VISION AND IMAGE UNDERSTANDING

Deep learning analysis of defect and phase evolution during electron beam-induced transformations in WS2

(2019) Artem Maksov et al.   npj Computational Materials

Towards calibration-invariant spectroscopy using deep learning

(2019) M. Chatzidakis et al.   Scientific Reports

Deep Learning for Validating and Estimating Resolution of Cryo-Electron Microscopy Density Maps †

(2019) Todor Avramov et al.   MOLECULES

Automatic detection, localization and segmentation of nano-particles with deep learning in microscopy images

(2019) Ayse Betul Oktay et al.   MICRON

AutoCryoPicker: an unsupervised learning approach for fully automated single particle picking in Cryo-EM images

(2019) Adil Al-Azzawi et al.   BMC BIOINFORMATICS

Lab on a beam—Big data and artificial intelligence in scanning transmission electron microscopy

(2019) Sergei V. Kalinin et al.   MRS BULLETIN

Improving electron micrograph signal-to-noise with an atrous convolutional encoder-decoder

(2019) Jeffrey M. Ede et al.   ULTRAMICROSCOPY

Reducing the U-Net size for practical scenarios: Virus recognition in electron microscopy images

(2019) Damian J. Matuszewski et al.   COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE

A Super-Clustering Approach for Fully Automated Single Particle Picking in Cryo-EM

(2019) Al-Azzawi et al.   Genes

DeepRes: a new deep-learning- and aspect-based local resolution method for electron-microscopy maps

(2019) Erney Ramírez-Aportela et al.   IUCrJ

Building and exploring libraries of atomic defects in graphene: Scanning transmission electron and scanning tunneling microscopy study

(2019) Maxim Ziatdinov et al.   Science Advances

Atomic Mechanisms for the Si Atom Dynamics in Graphene: Chemical Transformations at the Edge and in the Bulk

(2019) Maxim Ziatdinov et al.   ADVANCED FUNCTIONAL MATERIALS

Modelling structures in cryo-EM maps

(2019) Sony Malhotra et al.   CURRENT OPINION IN STRUCTURAL BIOLOGY

Deep Learning for Generic Object Detection: A Survey

(2019) Li Liu et al.   INTERNATIONAL JOURNAL OF COMPUTER VISION

Real-time cryo-electron microscopy data preprocessing with Warp

(2019) Dimitry Tegunov et al.   NATURE METHODS

Positive-unlabeled convolutional neural networks for particle picking in cryo-electron micrographs

(2019) Tristan Bepler et al.   NATURE METHODS

Resolution enhancement in scanning electron microscopy using deep learning

(2019) Kevin de Haan et al.   Scientific Reports

A joint deep learning model to recover information and reduce artifacts in missing-wedge sinograms for electron tomography and beyond

(2019) Guanglei Ding et al.   Scientific Reports

Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels

(2019) Graham Roberts et al.   Scientific Reports

Decoding crystallography from high-resolution electron imaging and diffraction datasets with deep learning

(2019) J. A. Aguiar et al.   Science Advances

A sub-sampled approach to extremely low-dose STEM

(2018) A. Stevens et al.   APPLIED PHYSICS LETTERS

Generative Adversarial Networks: An Overview

(2018) Antonia Creswell et al.   IEEE SIGNAL PROCESSING MAGAZINE

Fully automated primary particle size analysis of agglomerates on transmission electron microscopy images via artificial neural networks

(2018) Max Frei et al.   POWDER TECHNOLOGY

Atomic-resolution transmission electron microscopy of electron beam–sensitive crystalline materials

(2018) Daliang Zhang et al.   SCIENCE

How should a fixed budget of dwell time be spent in scanning electron microscopy to optimize image quality?

(2018) Patrick Trampert et al.   ULTRAMICROSCOPY

A deep convolutional neural network to analyze position averaged convergent beam electron diffraction patterns

(2018) W. Xu et al.   ULTRAMICROSCOPY

Virus Particle Detection by Convolutional Neural Network in Transmission Electron Microscopy Images

(2018) Eisuke Ito et al.   Food and Environmental Virology

Advanced Steel Microstructural Classification by Deep Learning Methods

(2018) Seyed Majid Azimi et al.   Scientific Reports

Automated defect analysis in electron microscopic images

(2018) Wei Li et al.   npj Computational Materials

CDeep3M—Plug-and-Play cloud-based deep learning for image segmentation

(2018) Matthias G. Haberl et al.   NATURE METHODS

Single-particle cryo-EM—How did it get here and where will it go

(2018) Yifan Cheng   SCIENCE

Detection of herpesvirus capsids in transmission electron microscopy images using transfer learning

(2018) K. Shaga Devan et al.   HISTOCHEMISTRY AND CELL BIOLOGY

Real-time coherent diffraction inversion using deep generative networks

(2018) Mathew J. Cherukara et al.   Scientific Reports

Deep Consensus, a deep learning-based approach for particle pruning in cryo-electron microscopy

(2018) Ruben Sanchez-Garcia et al.   IUCrJ

Residual Deconvolutional Networks for Brain Electron Microscopy Image Segmentation

(2017) Ahmed Fakhry et al.   IEEE TRANSACTIONS ON MEDICAL IMAGING

Convolutional neural networks for automated annotation of cellular cryo-electron tomograms

(2017) Muyuan Chen et al.   NATURE METHODS

Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review

(2017) Waseem Rawat et al.   NEURAL COMPUTATION

Neural Network for Nanoscience Scanning Electron Microscope Image Recognition

(2017) Mohammad Hadi Modarres et al.   Scientific Reports

Image driven machine learning methods for microstructure recognition

(2016) Aritra Chowdhury et al.   COMPUTATIONAL MATERIALS SCIENCE

DeepPicker: A deep learning approach for fully automated particle picking in cryo-EM

(2016) Feng Wang et al.   JOURNAL OF STRUCTURAL BIOLOGY

Deep learning for visual understanding: A review

(2016) Yanming Guo et al.   NEUROCOMPUTING

Segmentation of carbon nanotube images through an artificial neural network

(2016) María Celeste Ramírez Trujillo et al.   SOFT COMPUTING

Cryo-EM single particle analysis with the Volta phase plate

(2016) Radostin Danev et al.   eLife

Single-Particle Cryo-EM at Crystallographic Resolution

(2015) Yifan Cheng   CELL

Molecular Mechanism of V(D)J Recombination from Synaptic RAG1-RAG2 Complex Structures

(2015) Heng Ru et al.   CELL

Saturated Reconstruction of a Volume of Neocortex

(2015) Narayanan Kasthuri et al.   CELL

Deep learning

(2015) Yann LeCun et al.   NATURE

Big–deep–smart data in imaging for guiding materials design

(2015) Sergei V. Kalinin et al.   NATURE MATERIALS

Crowdsourcing the creation of image segmentation algorithms for connectomics

(2015) Ignacio Arganda-Carreras et al.   Frontiers in Neuroanatomy

Principles of cryo-EM single-particle image processing

(2015) Fred J. Sigworth   Microscopy

Third-dimension information retrieval from a single convergent-beam transmission electron diffraction pattern using an artificial neural network

(2014) Robert S. Pennington et al.   PHYSICAL REVIEW B

Structure of  -galactosidase at 3.2-A resolution obtained by cryo-electron microscopy

(2014) A. Bartesaghi et al.   PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA

The Resolution Revolution

(2014) W. Kuhlbrandt   SCIENCE

Cryo-EM structure of the Plasmodium falciparum 80S ribosome bound to the anti-protozoan drug emetine

(2014) Wilson Wong et al.   eLife

Structure of the TRPV1 ion channel determined by electron cryo-microscopy

(2013) Maofu Liao et al.   NATURE

Connectomic reconstruction of the inner plexiform layer in the mouse retina

(2013) Moritz Helmstaedter et al.   NATURE

General framework for quantitative three-dimensional reconstruction from arbitrary detection geometries in TEM

(2013) Wouter Van den Broek et al.   PHYSICAL REVIEW B

The potential for Bayesian compressive sensing to significantly reduce electron dose in high-resolution STEM images

(2013) Andrew Stevens et al.   Microscopy

Method for Retrieval of the Three-Dimensional Object Potential by Inversion of Dynamical Electron Scattering

(2012) Wouter Van den Broek et al.   PHYSICAL REVIEW LETTERS

A clarification of the terms used in comparing semi-automated particle selection algorithms in Cryo-EM

(2011) Robert Langlois et al.   JOURNAL OF STRUCTURAL BIOLOGY

Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation

(2009) Srinivas C. Turaga et al.   NEURAL COMPUTATION

An improved ptychographical phase retrieval algorithm for diffractive imaging

(2009) Andrew M. Maiden et al.   ULTRAMICROSCOPY

Reconstruction of off-axis electron holograms using a neural net

(2003) Meyer et al.   JOURNAL OF MICROSCOPY