- Open Access
Discrimination learning with variable stimulus 'salience'
International Archives of Medicinevolume 4, Article number: 26 (2011)
In nature, sensory stimuli are organized in heterogeneous combinations. Salient items from these combinations 'stand-out' from their surroundings and determine what and how we learn. Yet, the relationship between varying stimulus salience and discrimination learning remains unclear.
Presentation of the hypothesis
A rigorous formulation of the problem of discrimination learning should account for varying salience effects. We hypothesize that structural variations in the environment where the conditioned stimulus (CS) is embedded will be a significant determinant of learning rate and retention level.
Testing the hypothesis
Using numerical simulations, we show how a modified version of the Rescorla-Wagner model, an influential theory of associative learning, predicts relevant interactions between varying salience and discrimination learning.
Implications of the hypothesis
If supported by empirical data, our model will help to interpret critical experiments addressing the relations between attention, discrimination and learning.
In nature, sensory stimuli are organized in heterogeneous combinations. Salient items from these combinations 'stand-out' from their surroundings and influence what and how we learn. The salience of these items arises from the joint action of the items' intrinsic physical properties and the motivational state of the subject that learns about them; ultimately, it determines the discriminative-incentive value of such items [1–3]. In psychophysics, perceptual thresholds of detection and discrimination are estimated by means of linear variations of stimulus properties from a level of 'no detection', to a level of 'robust detection', and vice versa . The sign and slope of these variations are not expected to interfere with the decoding capabilities that serve the setting of perceptual detection . Yet, stimulus salience is subject to variation as learning occurs, and multiple items compete for attention. From the point of view of discrimination learning, the relationship between varying salience and learning remains unclear.
For the past four decades, the Rescorla-Wagner (RW) model  has been a very influential theory of associative learning. It explains how the associative status of a conditioned stimulus (CS) varies when it is trained, i.e., repeatedly paired with an unconditioned stimulus (US) [6, 7]. Equation 1 shows the model as proposed by the authors:
Where V(t) is the strength of the CS-US association or the cumulative amount of learning, is the CS salience (0 ≤ α ≤ 1), β corresponds to US salience (0 ≤ β ≤ 1) and λ is the asymptote of learning, i.e., maximum retention level at infinite training repetitions. This model predicts that the development of a conditioned response will depend upon sustained changes in the strength of the CS-US association. In each learning trial, the change in V(t) will be proportional to the product between α, β and the difference between λ (set by specific attributes of the US) and the sum of V(t) for all the stimuli present in the trial. Thus, the strength of the CS-US association and the degree of learning towards the CS will increase throughout successive learning trials in a negatively accelerated fashion, as V(t) approaches λ.
The RW-model has been influential because it is simple and allows predictions in situations where multiple cues are reinforced simultaneously, accounting for learning phenomenah as blocking and overshadowing . Yet, while the RW-model assumes a constant processing of CS information, in nature, CS (and US) salience is subject to variation. Indeed, there is general agreement that the salience of any given CS (or conditioned situation) will depend on: (i) the physical properties of the environment that determine how discriminativeis the CS (as it stands against a background), as well as on (ii) subject- and motivation-dependent perceptual features that influence learning [8, 9]. In other words, α depends on constellations of sensory inputs and the subject's information capabilities, but it also varies with experience and motivation. Ultimately, the joint action of these external and internal elements will determine whether and how the CS is assigned with a particular predictive value.
Presentation of the hypothesis
In the laboratory, learning is easier to predict when training stimuli and motivational states are kept as constant as possible, a most unlikely situation in real life. In nature, open environments vary and afford locomotion, changing the structure of sensory arrays . Salience is strongly influenced by the interplay between locomotion, perception, past experience and acquired knowledge. Irrespective of the physical properties of the stimulus in question, CS associability is not immutable because reinforcement modifies incentive values and leads to complex interactions between sensory inputs and conditioned responses . Thus, a rigorous formulation of the problem of discrimination learning should account for varying CS salience and perceptibility. We hypothesize that controlled variations of the environment will modify CS salience and determine learning rates and retention values in a predictable manner. As the subject learns at different rates, this may lead to different computational strategies to discriminate objects from the sensory stream. We subscribe to the idea that theoretical models of learning can guide experimental design. We here explore the validity of our hypothesis by means of a modified version of the RW-model accounting for varying CS salience effects.
Testing the hypothesis
Let us modify the RW-model to account for varying CS salience, as well as to include a putative discrimination threshold in the following equation:
Where α(t) represents variable salience over time and α min is the salience threshold for learning to occur. For simplicity, we represent λ as a sliding logistic function of α , because the quality of sensory representation should degrade gradually as salience reaches α min, compromising discrimination  and learning. We assume that discrimination performance is constrained by a perceptual grid that filters out relevant information for the discrimination task, as represented by λ(α) at low α values.
Regarding varying salience: if stimulus 'i' is reinforced, then α i (t) should increase, and if stimulus 'j' is not reinforced, then α j (t) should decrease. In a situation where the stimuli, 'i', 'j', and 'k' are sequentially reinforced, then an increase in a α i (t) should affect α j (t) and α k (t) according to the degree of similarity between the stimuli. Therefore, the varying salience over time may adopt the following form:
where Si,j represents the degree of similarity between the ith (reference) and jth stimuli (0 ≤ S ≤ 1), and α i (t) is the dynamic representation of salience with respect to item 'i', as the probability of attention will vary together with salience and learning [16, 17]. Thus, α (t) should increase or decrease depending on both, reinforcement levels and the temporal arrangement of stimuli similarity during training. Evidently, we do not know how salience evolves with learning. Let us consider a simple steady-state scenario, where α (t) equals Si,j. What would be the effect of varying stimuli similarity during learning? To explore this idea, we first generated a set ofstimuli with different degrees of similarity by using random numbers from normal distributions with fixed meanand variable standard deviations (Figure 1A). These numbers represent training stimuli with different salience. To investigate whether variable salience has a relevant effect in learning, we sorted the stimuli using other decreasing (black line) or increasing (gray line) similarity (Figure 1B). These arrangements maximize the relative difference in salience between training programs but consist of exactly the same stimuli. Next, we calculated λ(α), applying either no salience threshold (i.e. α min = 0) or a putative threshold of 0.3 (α min = 0.3; Figure 1C). Panels D-E show the predicted learning curves, as given by Eq.2. In all cases of identical mean salience of 0.5, the temporal arrangement of training stimuli determined the shape of the learning curves.
Moreover, when discriminative training involved stimuli below the salience threshold for learning (Figure 1E), stimuli with salience below α min were undetectable, V(t) did not increase (for V(t) = 0), and the curves decayed in a mono-exponential manner due to the lack of reinforcement (0 ≤ V(t) ≤ 1). When similarity was held constant (thick dotted lines), the learning curves were identical to those predicted by the standard model.
Implications of the hypothesis
In order to survive, organisms must learn to discriminate items with predictive values. Some models of associative learning assume a processing of conditioned stimuli with constant salience , but in nature salience is variable as environments and experience change dynamically. Some theories emphasize that multiple CSs must compete for internal representations of limited capacity, forcing learning about some stimuli to be at the expense of learning about other stimuli . A realistic formulation of the problem of learning must consider varying CS salience, not only because learning exerts a direct influence on it (via attention and contiguity), but also because discriminative stimuli exchange and compete for attention. Using numerical simulations of discriminative training, we here show that a modified version of the Rescorla-Wagner model predicts how varying CS salience influences discrimination learning. This interaction may become evident in conditions where discrimination learning is slow and multiple arrangements of training stimuli are compared, as we did here. If true, such a mathematical variant may become useful to explain the co-varying interactions between attention, discrimination and learning. A general learning theory must address the internal and external factors that influence how the brain allocates attention and apprehends the environment to select, store and retrieve information for generating adaptive behavior.
Mackintosh NJ: A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement. Psychological Review 1975, 82:276–298.
Koch C, Ullman S: Shifts in selective visual attention: towards the underlying neural circuitry. Hum Neurobiol 1985, 4:219–227.
Bindra D: A motivational view of learning, performance, and behavior modification. Psychol Rev 1974, 81:199–213.
Gescheider G: Psychophysics: the fundamentals. Lawrence Erlbaum Associates; 1997.
Romo R, Hernandez A, Zainos A, Salinas E: Correlated neuronal discharges that increase coding efficiency during perceptual discrimination. Neuron 2003, 38:649–657.
Rescorla RA, Wagner AR: A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non reinforcement. In Classical conditioning II: current research and theory. Edited by: Black AH, Prokasy WF. New York: Appleton-Century-Crofts; 1972:64–99.
Miller RR, Barnet RC, Grahame NJ: Assessment of the Rescorla-Wagner model. Psychol Bull 1995, 117:363–386.
McFarland DJ: Feedback Mechanisms in Animal Behaviour. Academic Press. London; 1971.
Moran J, Desimone R: Selective attention gates visual processing in the extrastriate cortex. Science 1985, 229:782–784.
Gibson JJ: The ecological approach to visual perception. Psychology Press; 1986.
Britten KH, Shadlen MN, Newsome WT, Movshon JA: The analysis of visual motion: a comparison of neuronal and psychophysical performance. J Neurosci 1992, 12:4745–4765.
Mountcastle VB, Steinmetz MA, Romo R: Frequency discrimination in the sense of flutter: psychophysical measurements correlated with postcentral events in behaving monkeys. J Neurosci 1990, 10:3032–3044.
Croner LJ, Kaplan E: Receptive fields of P and M ganglion cells across the primate retina. Vision Res 1995, 35:7–24.
Boynton GM, Demb JB, Glover GH, Heeger DJ: Neuronal basis of contrast discrimination. Vision Res 1999, 39:257–269.
Pearce JM, Bouton ME: Theories of associative learning in animals. Annu Rev Psychol 2001, 52:111–139.
Lawrence DH: Acquired distinctiveness of cues; transfer between discrimination on the basis of familiarity with the stimulus. J Exp Psychol 1949, 39:770–784.
Lawrence DH: Acquired distinctiveness of cues: selective association in a constant stimulus situation. J Exp Psychol 1950, 40:175–188.
We thank T. Keller for fruitful talks. We thank Prof. P.H. Seeburg for constant support. M.T. was supported by a Max-Planck-Fellowship.
The authors have no financial competing interests.
MT and RM: conceived ideas. MT and EA: developed the equations and simulations in MATLAB 7.8 (MathWorks, Inc.; Natick, USA). All authors contributed writing and revising the manuscript. All authors read and approved the final version of the manuscript.