A Sparse and Spike-Timing-Based Adaptive Photoencoder for Augmenting Machine Vision for Spiking Neural Networks

The representation of external stimuli in the form of action potentials or spikes constitutes the basis of energy efficient neural computation that emerging spiking neural networks (SNNs) aspire to imitate. With recent evidence suggesting that information in the brain is more often represented by explicit firing times of the neurons rather than mean firing rates, it is imperative to develop novel hardware that can accelerate sparse and spike-timing-based encoding. Here a medium-scale integrated circuit composed of two cascaded three-stage inverters and one XOR logic gate fabricated using a total of 21 memtransistors based on photosensitive 2D monolayer MoS2 for spike-timing-based encoding of visual information, is introduced. It is shown that different illumination intensities can be encoded into sparse spiking with time-to-first-spike representing the illumination information, that is, higher intensities invoke earlier spikes and vice versa. In addition, non-volatile and analog programmability in the photoencoder is exploited for adaptive photoencoding that allows expedited spiking under scotopic (low-light) and deferred spiking under photopic (bright-light) conditions, respectively. Finally, low energy expenditure of less than 1 mu J by the 2D-memtransistor-based photoencoder highlights the benefits of in-sensor and bioinspired design that can be transformative for the acceleration of SNNs.

Automated summary

This article introduces a medium-scale integrated circuit based on photosensitive 2D material for spike-timing-based encoding of visual information. The study shows that different illumination intensities can be encoded into sparse spiking, with the time-to-first-spike representing the illumination information. In addition, non-volatile and analog programmability in the photoencoder enables adaptive photoencoding.