Temporal Fusion Transformers for interpretable multi-horizon time series forecasting

Multi-horizon forecasting often contains a complex mix of inputs - including static (i.e. time-invariant) covariates, known future inputs, and other exogenous time series that are only observed in the past - without any prior information on how they interact with the target. Several deep learning methods have been proposed, but they are typically `black-box' models that do not shed light on how they use the full range of inputs present in practical scenarios. In this paper, we introduce the Temporal Fusion Transformer (TFT) - a novel attention-based architecture that combines high-performance multi-horizon forecasting with interpretable insights into temporal dynamics. To learn temporal relationships at different scales, TFT uses recurrent layers for local processing and interpretable self-attention layers for long-term dependencies. TFT utilizes specialized components to select relevant features and a series of gating layers to suppress unnecessary components, enabling high performance in a wide range of scenarios. On a variety of real-world datasets, we demonstrate significant performance improvements over existing benchmarks, and highlight three practical interpretability use cases of TFT. (C) 2021 The Authors. Published by Elsevier B.V. on behalf of International Institute of Forecasters.

Automated summary

This paper introduces the Temporal Fusion Transformer (TFT), a novel attention-based architecture that combines high-performance multi-horizon forecasting with interpretable insights into temporal dynamics. TFT utilizes recurrent layers for local processing and interpretable self-attention layers for long-term dependencies, achieving high performance in a wide range of scenarios. By selecting relevant features and suppressing unnecessary components, TFT demonstrates significant performance improvements over existing benchmarks on various real-world datasets.