Learning Pt3 PBS5

  1. Define instrumental conditioning.
    • A change in behaviour (learning) caused by a causal relationship between the behaviour and a biologically important stimulus (reinforcer).
    • (Process whereby organisms learn to make responses in order to obtain or avoid important consequences)
  2. Outline the difference between instrumental and Pavlovian conditioning.
    • Pavlovian: response is a reflex produced in expectation of an outcome
    • Instrumental: response is instrumental in receiving/determining the outcome
  3. Who first demonstrated instrumental conditioning? How?
    • Thorndike (1911)
    • Cats could learn to press lever to get out of a cage
    • Over time, the time taken for them to escape became shorter and shorter
  4. From this experiment, what did Thorndike observe? What law?
    • Law of Effect
    • Of the several responses made to same situation (lying down, meowing, pulling lever) those which were accompanied or closely followed by satisfaction will be more firmly connected to the situation.
  5. [Principles of reinforcement] In Pavlovian conditioning, irrespective of whether reinforcer is appetitive or aversive, conditioning results in increase in vigour of response. What are the different types of outcomes in instrumental conditioning?
    • Positive reinforcer: response followed by pleasant outcome - probability of response increases
    • Negative reinforcer: response followed by removal of unpleasant stimulus - response increases
    • Punisher: respose is followed by unpleasant outcome (positive) or response is followed by removal of pleasant stimulus (negative). Responses decrease.
  6. What are the 2 different schedule types that arrange response-outcome relationships?
    • Ratio: presence of reinforcer depends on number of responses
    • Interval: presence of reinforcer depends on how much time has passed since the last reinforcer (irrespective of number of responses)
    • Both these can be fixed or variable. (VR = variable ratio; FI = fixed interval).
  7. Ratio schedule or Interval - which is more common in nature?
    • Interval ratio is more similar to what happens in nature
    • because in nature, reinforcers are often organic and organic resources deplete and only replete after a certain time has elapsed so that there is a limit on rate of reinforcement.
  8. Describe the study that compared the performance on two types of reinforcement schedule.
    • Matthews et al (1977)
    • Rewarded lever pressing with monetary reward on VR schedule
    • every time reward was delivered on VR schedule, a reward became availble for another participant- generating a yoked variable interval (VI) schedule with similar rate of reinforcement
    • VR ratio schedule yielded much higher rate of responding than yoked VI
    • (probably because of awareness that delivery depends on response - VI participants will only get really high response rates when enough time elapses that they expect another reward soon)
  9. In his '__ __ __', Thorndike proposed that reinforcers strengthen an association between __ present when response is performed and the ___ itself. __-__ (__), thereby establishing a __ whereby __ provoked ___.
    • law of effect
    • stimuli
    • response
    • stimulus-response (S-R)
    • habit
    • stimulus 
    • response
  10. Whose study showed how there is a differece in brain areas activated depending on Pavlovian or Instrumental conditioning (habit). Describe the study.
    • O'Doherty et al (2004)
    • Participants touch one of two simultaneously presented visual stimuli - one of them yields fruit juice reward
    • Other participants just got yoked pairings of stimuli with outcome without making choice response
    • fMRI: showed activity in dorsal striatum ONLY during instrumental learning (first group of participants), and not for Pavlovian
    • Both groups showed activation in ventral striatum.
  11. According to __-__ accounts, instrumental learning is supported by both a __-__ and a __ system.
    • dual-system
    • goal-directed
    • habitual
  12. What is the limitation/difference of S-R reinforcement compared to goal-directed learning/behaviour?
    • S-R reinfrcement establishes instrumental habits that are not mediated by knowledge of the outcome of response
    • So, it doesn't allow selection of instrumental action based on agent's current goal
    • Goal-directed - representation of a causal relationship between action and outcome + representation of current incentive value of outcome.
  13. Describe the study which showed how one can discriminate between instrumental action that is an S-R habit or goal-directed action.
    • Klossek et al (2008)
    • Outcome devaluation paradigm
    • Train children to touch one icon for one cartoon clip and another for another clip
    • After this, one of the cartoons devaluated by prolonged exposure inducing boredom
    • Then, test the choice between touching two icons in extinction without cartoons
    • If original training simply established stimulus-response habits with no knowledge of outcome, then children should perform both responses equally during extinction test
    • But, if they learned which cartoon was produced by each response, they should show devaluation effect and should touch the icon with the more valuable outcome during the test. 
    • Older children (>27months) showed outcome devaluation effect 
    • Younger children (<27 months) performed both responses equally
    • Importantly, absence of devaluation effect in younger children wasn't because of failure of cartoon devaluation - two other control groups (younger and older) and the younger children chose response producing valued outcome when outcomes were still available rather than extinction.
  14. Describe the study showing neural correlates of goal-directed and habit learning. For this slide, first just tell me how they managed to isolate one type of learning from another.
    • De Wit et al (2009)
    • Task: learn association between a stimulus printed on box and the outcome inside - after being presented with stimulus, participant must press either right or left button.
    • 3 types of discrimination: 
    • 1. Control - stimulus and outcome different 
    • 2. Congruent - stimulus and outcome are same
    • 3. Incongruent - stimulus of one problem is the same as outcome in the other
    • The incongruent discrimination relies solely on the habit system, thus allowing isolation of different parts of brain used for each type of learning.
    • Goal-directed learning cannot work in the incongruent task because it will activate S-O-R of one chain and O-R of another.
    • eg. cherry(S) -> pear (O) -> right (R) AND pear (S) -> cherry (O) -> left (R)
    • In this case, if participant is presented with cherry stimulus, it will activate S-O-R chain where they will press right button, but will also activate O-R (cherry->left) which will cause a left button response. 
    • Image Upload 1
  15. De Wit confirmed that the incongruent conditions engaged only the S-R habit response by doing what?
    • Outcome devaluation phase
    • 2 open boxes with outcomes inside presented
    • one box had cross, showing outcome no longer produced points (while other still had food)
    • Prediction: accuracy maintain on control and congruent conditions because goal-directed learning is affected by outcome devaluation but accuracy decrease on incongruent condition.
    • This is exactly what was found, confirming that the incongruent condition engaged S-R habit response.
  16. What did De Wit's scanning results show about the neural correlates for each of the learning types then?
    • Goal-directed: greater ventro-medial PFC activation
    • Habit: greater dorso-medial PFC activation
    • During devaluation phase: ventral medial PFC activation higher in goal-directed control trials. (Positive correlation between accuracy in outcome devaluation test and vmPFC activation).
  17. Describe the study which suggested that a particular brain region is sensitive to incentive value of reinforcers.
    • Valentin et al (2007) (fMRI study)
    • 1. Participants first learned association between visual stimulus and food reward (chocolate or tomato) with probability of .4.
    • Choosing the other visual stimulus (out of 2) would produce common outcome (orange juice) of .3
    • So, one action will produce food reward of .7 and the other .3
    • Also a control trial with tasteless liquid reward (neutral)
    • Result: as number of blocks/trials increase -> the more number of high probability choices for rewarding outcomes (but stayed same for neutral)
    • ALSO - medial orbitofrontal cortex (part of vmPFC) activation in rearding outcome.
    • 2. After training in scanner, participants consumed one outcome to satiation (to the full) decreasing the incentive value of the outcome
    • Then they did the same instrumental choice procedure
    • Result: after devaluation by satiation, greater medial & central PFC activity in valued condition (compared to devalued)
    • These two scanning results suggest that this region (vmPFC area) is sensitive to the incentive value of reinforcers.
Card Set
Learning Pt3 PBS5
Lec3 - Instrumental conditioning