How multisensory neurons solve causal inference

Sitting in a static railway carriage can produce illusory self-motion if the train on an adjoining track moves off. While our visual system registers motion, vestibular signals indicate that we are stationary. The brain is faced with a difficult challenge: is there a single cause of sensations (I am moving) or two causes (I am static, another train is moving)? If a single cause, integrating signals produces a more precise estimate of self-motion, but if not, one cue should be ignored. In many cases, this process of causal inference works without error, but how does the brain achieve it? Electrophysiological recordings show that the macaque medial superior temporal area contains many neurons that encode combinations of vestibular and visual motion cues. Some respond best to vestibular and visual motion in the same direction (congruent neurons), while others prefer opposing directions (opposite neurons). Congruent neurons could underlie cue integration, but the function of opposite neurons remains a puzzle. Here, we seek to explain this computational arrangement by training a neural network model to solve causal inference for motion estimation. Like biological systems, the model develops congruent and opposite units and recapitulates known behavioral and neurophysiological observations. We show that all units (both congruent and opposite) contribute to motion estimation. Importantly, however, it is the balance between their activity that distinguishes whether visual and vestibular cues should be integrated or separated. This explains the computational purpose of puzzling neural representations and shows how a relatively simple feedforward network can solve causal inference.

Automated summary

The brain faces the challenge of determining whether sensations of motion have a single cause or multiple causes, integrating vestibular and visual cues for more precise estimation of self-motion. The macaque medial superior temporal area contains neurons that encode combinations of vestibular and visual motion cues, with some responding to congruent cues and others to opposite cues. A neural network model trained for causal inference in motion estimation exhibits both congruent and opposite units, showing the importance of the balance between their activities in determining whether cues should be integrated or separated.