Dopamine signals as temporal difference errors: recent advances

In the brain, dopamine is thought to drive reward-based learning by signaling temporal difference reward prediction errors (TD errors), a 'teaching signal' used to train computers. Recent studies using optogenetic manipulations have provided multiple pieces of evidence supporting that phasic dopamine signals function as TD errors. Furthermore, novel experimental results have indicated that when the current state of the environment is uncertain, dopamine neurons compute TD errors using 'belief states' or a probability distribution over potential states. It remains unclear how belief states are computed but emerging evidence suggests involvement of the prefrontal cortex and the hippocampus. These results refine our understanding of the role of dopamine in learning and the algorithms by which dopamine functions in the brain.

Automated summary

Dopamine in the brain is believed to facilitate reward-based learning by calculating temporal difference reward prediction errors (TD errors), especially in uncertain environments where dopamine neurons use belief states to compute TD errors. The prefrontal cortex and the hippocampus are suggested to be involved in computing belief states.