LSTM networks enabled nonlinear equalization in 50-Gb/s PAM-4 transmission links

This paper proposes a nonlinear equalization technique enabled by long short-term memory (LSTM) recurrent neural networks. The proposed technique is implemented at the end of offline digital signal processing. And two approaches utilizing the LSTM network are experimentally tested and demonstrated in transmission of a 50-Gb/s four-level pulse amplitude modulation intensity modulation direct detection link over 100-km standard single-mode fiber. The first approach uses the LSTM network-based equalizer to directly categorize the received signal into four amplitude levels, and the second approach uses the LSTM network to estimate signal noise for compensating the received signal. The experimental results show remarkable performance improvement of the proposed method over conventional linear equalizers, and significant enhancement at high launch power compared with Volterra filtering. Also, the proposed method reveals better short-time universality. (C) 2019 Optical Society of America