Delayed Rewards Facilitate Habit Formation

The transition from goal-directed to habitual forms of instrumental behavior is determined by variables such as the amount of training, schedules of reinforcement, the availability of choices, and exposure to drugs of abuse. Less is known about the control of goal-directed behavior when reinforcement is delayed rather than immediate. In these experiments, we investigated in rats the role of response-outcome contiguity on the control of goal-directed action, assessed through satiety-specific outcome devaluation tests. In Experiment 1 using a within-subjects design we observed goal-directed behavior after 6 days of FR1 training when the outcome was presented immediately following the lever press, but not when it was delayed for 20 s, revealing habit formation with delayed outcomes. Experiment 2 revealed that the habitual control observed with 20-s delays of reinforcement can be prevented if, immediately before each instrumental training session, the rats were exposed to the experimental context in the absence of both the lever and reinforcement. In summary, these experiments suggest that response-outcome contiguity plays an important role in the control of goal-directed actions and habits.