4.7 Review

At the intersection of optics and deep learning: statistical inference, computing, and inverse design

Journal

ADVANCES IN OPTICS AND PHOTONICS
Volume 14, Issue 2, Pages 209-290

Publisher

Optica Publishing Group
DOI: 10.1364/AOP.450345

Keywords

-

Categories

Funding

  1. Office of Naval Research
  2. Air Force Office of Scientific Research

Ask authors/readers for more resources

Deep learning is revolutionizing information processing in various fields, and optical computing may address the need for low-power, scalable, and fast computing hardware. Recent advancements in optics and photonics have enabled the use of light to accelerate machine learning tasks, and deep learning has also had a significant impact on inverse optical/photonics design.
Deep learning has been revolutionizing information processing in many fields of science and engineering owing to the massively growing amounts of data and the advances in deep neural network architectures. As these neural networks are expanding their capabilities toward achieving state-of-the-art solutions for demanding statistical inference tasks in various applications, there appears to be a global need for low-power, scalable, and fast computing hardware beyond what existing electronic systems can offer. Optical computing might potentially address some of these needs with its inherent parallelism, power efficiency, and high speed. Recent advances in optical materials, fabrication, and optimization techniques have significantly enriched the design capabilities in optics and photonics, leading to various successful demonstrations of guided-wave and free-space computing hardware for accelerating machine learning tasks using light. In addition to statistical inference and computing, deep learning has also fundamentally affected the field of inverse optical/photonic design. The approximation power of deep neural networks has been utilized to develop optics/photonics systems with unique capabilities, all the way from nanoantenna design to end-to-end optimization of computational imaging and sensing systems. In this review, we attempt to provide a broad overview of the current state of this emerging symbiotic relationship between deep learning and optics/photonics. (C) 2022 Optica Publishing Group

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Nanoscience & Nanotechnology

Diffractive interconnects: all-optical permutation operation using diffractive networks

Deniz Mengu, Yifan Zhao, Anika Tabassum, Mona Jarrahi, Aydogan Ozcan

Summary: This study presents the use of diffractive optical networks to perform permutation operations with a large number of input-output connections. By using deep learning techniques, the capacity of the diffractive optical network to approximate permutation operations increases with the number of diffractive layers and trainable transmission elements in the system. The authors also addressed challenges related to physical alignment and output efficiency by designing misalignment tolerant diffractive designs. The proposed diffractive permutation networks have potential applications in security, image encryption, and data processing, and can serve as channel routing and interconnection panels in wireless networks.

NANOPHOTONICS (2023)

Article Chemistry, Multidisciplinary

Data-Class-Specific All-Optical Transformations and Encryption

Bijie Bai, Heming Wei, Xilin Yang, Tianyi Gan, Deniz Mengu, Mona Jarrahi, Aydogan Ozcan

Summary: This paper presents a diffractive optical network that performs data-class-specific transformations between the input and output fields-of-view (FOVs) using optical methods. The visual information of objects is encoded into the amplitude, phase, or intensity of the optical field at the input and processed by a data-class-specific diffractive network. The output patterns are optically encrypted using preassigned transformation matrices, and the original input images can be recovered by applying the correct decryption key.

ADVANCED MATERIALS (2023)

Article Engineering, Electrical & Electronic

Analysis of Diffractive Neural Networks for Seeing Through Random Diffusers

Yuhang Li, Yi Luo, Bijie Bai, Aydogan Ozcan

Summary: Imaging through diffusive media is a challenging problem that typically requires digital computers for image reconstruction. In this study, we propose an alternative method using diffractive neural networks to see through random, unknown phase diffusers. Through detailed analysis, we observed a trade-off between image reconstruction fidelity and distortion reduction capability of the diffractive network. We also discovered that training the network with a variety of random diffusers and introducing misalignments improved its generalization performance.

IEEE JOURNAL OF SELECTED TOPICS IN QUANTUM ELECTRONICS (2023)

Article Multidisciplinary Sciences

Unidirectional imaging using deep learning-designed materials

Jingxi Li, Tianyi Gan, Yifan Zhao, Bijie Bai, Che-Yung Shen, Songyu Sun, Mona Jarrahi, Aydogan Ozcan

Summary: The first demonstration of unidirectional imagers is reported, presenting polarization-insensitive and broadband uni-directional imaging based on successive diffractive layers that are linear and isotropic. The diffractive unidirectional imager maintains its functionality over a large spectral band and works under broadband illumination. The diffractive unidirectional imaging using structured materials will have applications in security, defense, telecommunications, and privacy protection.

SCIENCE ADVANCES (2023)

Article Engineering, Biomedical

Rapid and stain-free quantification of viral plaque via lens-free holography and deep learning

Tairan Liu, Yuzhu Li, Hatice Ceylan Koydemir, Yijie Zhang, Ethan Yang, Merve Eryilmaz, Hongda Wang, Jingxi Li, Bijie Bai, Guangdong Ma, Aydogan Ozcan

Summary: An automated plaque assay leveraging lens-free holographic imaging and deep learning rapidly and accurately detects the cell-lysing events caused by viral replication.

NATURE BIOMEDICAL ENGINEERING (2023)

Article Chemistry, Multidisciplinary

Universal Polarization Transformations: Spatial Programming of Polarization Scattering Matrices Using a Deep Learning-Designed Diffractive Polarization Transformer

Yuhang Li, Jingxi Li, Yifan Zhao, Tianyi Gan, Jingtian Hu, Mona Jarrahi, Aydogan Ozcan

Summary: A universal polarization transformer is demonstrated that can synthesize various complex-valued polarization scattering matrices, providing a solution for controlled synthesis of optical fields with nonuniform polarization distributions.

ADVANCED MATERIALS (2023)

Article Engineering, Electrical & Electronic

eFIN: Enhanced Fourier Imager Network for Generalizable Autofocusing and Pixel Super-Resolution in Holographic Imaging

Hanlong Chen, Luzhe Huang, Tairan Liu, Aydogan Ozcan

Summary: The application of deep learning techniques has significantly improved holographic imaging capabilities, with enhanced phase recovery and image reconstruction. This study introduces eFIN, a deep neural network that utilizes pixel super-resolution and image autofocusing for hologram reconstruction. Experimental results demonstrate the superior image quality and external generalization of eFIN, which achieves a wide autofocusing range and accurately predicts hologram axial distances. The network also enables 3x pixel super-resolution and improves the space-bandwidth product of reconstructed images by 9-fold.

IEEE JOURNAL OF SELECTED TOPICS IN QUANTUM ELECTRONICS (2023)

Review Optics

High-throughput terahertz imaging: progress and challenges

Xurong Li, Jingxi Li, Yuhang Li, Aydogan Ozcan, Mona Jarrahi

Summary: This article reviews the development of terahertz imaging technologies and discusses different types of hardware and computational imaging algorithms. It explores opportunities for capturing various image data and briefly introduces the prospects and challenges for future high-throughput terahertz imaging systems.

LIGHT-SCIENCE & APPLICATIONS (2023)

Article Computer Science, Artificial Intelligence

Self-supervised learning of hologram reconstruction using physics consistency

Luzhe Huang, Hanlong Chen, Tairan Liu, Aydogan Ozcan

Summary: GedankenNet is a self-supervised learning model that achieves image reconstruction without the need for labelled or experimental training data, demonstrating superior generalization on hologram reconstruction tasks.

NATURE MACHINE INTELLIGENCE (2023)

Article Optics

Massively parallel universal linear transformations using a wavelength-multiplexed diffractive optical network

Jingxi Li, Tianyi Gan, Bijie Bai, Yi Luo, Mona Jarrahi, Aydogan Ozcan

Summary: Large-scale linear operations are crucial for complex computational tasks, and optical computing offers advantages in terms of speed, parallelism, and scalability. The deep-learning-based design of a broadband diffractive neural network enables the performance of a large group of complex-valued linear transformations. By assigning different illumination wavelengths to each transformation, a single diffractive network can execute multiple linear transformations simultaneously or sequentially. This technology allows for the approximation of unique linear transforms with negligible errors, and the spectral multiplexing capability can be increased by increasing the number of diffractive neurons.

ADVANCED PHOTONICS (2023)

Article Automation & Control Systems

Time-Lapse Image Classification Using a Diffractive Neural Network

Md Sadman Sakib Rahman, Aydogan Ozcan

Summary: This study demonstrates for the first time a time-lapse image classification scheme using a diffractive network, improving classification accuracy and generalization performance by leveraging the lateral movements of the input objects and/or the diffractive network. Numerical exploration reveals a blind testing accuracy of 62.03% on the optical classification of objects from the CIFAR-10 dataset using time-lapse diffractive networks, achieving the highest inference accuracy so far. Time-lapse diffractive networks will be widely beneficial for spatiotemporal analysis of input signals using all-optical processors.

ADVANCED INTELLIGENT SYSTEMS (2023)

Article Mathematical & Computational Biology

Digital staining facilitates biomedical microscopy

Michael John Fanous, Nir Pillar, Aydogan Ozcan

Summary: Traditional staining methods have drawbacks, while computational virtual staining using deep learning techniques has emerged as a powerful solution. Virtual staining can be combined with neural networks to correct microscopy aberrations and enhance resolution, significantly improving sample preparation and imaging in biomedical microscopy.

FRONTIERS IN BIOINFORMATICS (2023)

No Data Available