A photosensor employing data-driven binning for ultrafast image recognition

Pixel binning is a technique, widely used in optical image acquisition and spectroscopy, in which adjacent detector elements of an image sensor are combined into larger pixels. This reduces the amount of data to be processed as well as the impact of noise, but comes at the cost of a loss of information. Here, we push the concept of binning to its limit by combining a large fraction of the sensor elements into a single superpixel that extends over the whole face of the chip. For a given pattern recognition task, its optimal shape is determined from training data using a machine learning algorithm. We demonstrate the classification of optically projected images from the MNIST dataset on a nanosecond timescale, with enhanced dynamic range and without loss of classification accuracy. Our concept is not limited to imaging alone but can also be applied in optical spectroscopy or other sensing applications.

Automated summary

Pixel binning is a widely used technique in optical image acquisition and spectroscopy that combines adjacent detector elements to reduce data processing and noise while sacrificing information. Researchers have pushed the concept further by combining a large fraction of sensor elements into a single superpixel, determined by machine learning algorithm, for pattern recognition tasks. The technique has been successfully applied to nanosecond-scale classification of optically projected images without losing accuracy. This concept is not limited to imaging and can be extended to optical spectroscopy and other sensing applications.