An integrated imaging sensor for aberration-corrected 3D photography

Planar digital image sensors facilitate broad applications in a wide range of areas(1-5), and the number of pixels has scaled up rapidly in recent years(2,6). However, the practical performance of imaging systems is fundamentally limited by spatially nonuniform optical aberrations originating from imperfect lenses or environmental disturbances(7,8). Here we propose an integrated scanning light-field imaging sensor, termed a meta-imaging sensor, to achieve high-speed aberration-corrected three-dimensional photography for universal applications without additional hardware modifications. Instead of directly detecting a two-dimensional intensity projection, the meta-imaging sensor captures extra-fine four-dimensional light-field distributions through a vibrating coded microlens array, enabling flexible and precise synthesis of complex-field-modulated images in post-processing. Using the sensor, we achieve high-performance photography up to a gigapixel with a single spherical lens without a data prior, leading to orders-of-magnitude reductions in system capacity and costs for optical imaging. Even in the presence of dynamic atmosphere turbulence, the meta-imaging sensor enables multisite aberration correction across 1,000 arcseconds on an 80-centimetre ground-based telescope without reducing the acquisition speed, paving the way for high-resolution synoptic sky surveys. Moreover, high-density accurate depth maps can be retrieved simultaneously, facilitating diverse applications from autonomous driving to industrial inspections.

Automated summary

The article introduces an integrated scanning light-field imaging sensor that enables high-speed aberration-corrected three-dimensional photography. The sensor captures extra-fine four-dimensional light-field distributions through a vibrating coded microlens array, allowing for flexible and precise synthesis of complex-field-modulated images. Experimental results demonstrate that the sensor achieves high-performance photography and accurate depth mapping in different environments.