Off-the-shelf deep learning is not enough, and requires parsimony, Bayesianity, and causality

Deep neural networks ('deep learning') have emerged as a technology of choice to tackle problems in speech recognition, computer vision, finance, etc. However, adoption of deep learning in physical domains brings substantial challenges stemming from the correlative nature of deep learning methods compared to the causal, hypothesis driven nature of modern science. We argue that the broad adoption of Bayesian methods incorporating prior knowledge, development of solutions with incorporated physical constraints and parsimonious structural descriptors and generative models, and ultimately adoption of causal models, offers a path forward for fundamental and applied research.

Automated summary

Deep neural networks have become a preferred technology for solving problems in speech recognition, computer vision, finance, etc. However, applying deep learning in physical domains presents substantial challenges due to the correlative nature of deep learning methods compared to the hypothesis-driven nature of modern science. It is argued that adopting Bayesian methods, integrating physical constraints and parsimonious structural descriptors, and ultimately transitioning to causal models, can pave the way for foundational and applied research.