Feature detection is a process by which the nervous system sorts or filters complex natural stimuli in order to extract behaviorally relevant cues that have a high probability of being associated with important objects or organisms in their environment, as opposed to irrelevant background or noise.
Feature detectors are individual neurons—or groups of neurons—in the brain which code for perceptually significant stimuli. Early in the sensory pathway feature detectors tend to have simple properties; later they become more and more complex as the features to which they respond become more and more specific.
For example, simple cells in the visual cortex of the domestic cat (Felis catus), respond to edges—a feature which is more likely to occur in objects and organisms in the environment. By contrast, the background of a natural visual environment tends to be noisy—emphasizing high spatial frequencies but lacking in extended edges. Responding selectively to an extended edge—either a bright line on a dark background, or the reverse—highlights objects that are near or very large. Edge detectors are useful to a cat, because edges do not occur often in the background "noise" of the visual environment, which is of little importance to the cat.
History Early in the history of sensory neurobiology, physiologists favored the idea that the nervous system detected specific features of stimuli, rather than faithfully copying the sensory world onto a sensory map in the brain. For example, they favored the idea that the visual system detects specific features of the visual world. This view contrasted with the metaphor that the retina acts like a camera and the brain acts like film that preserves all elements without making assumptions about what is important in the environment. It wasn't until the late 1950s that the feature detector hypothesis fully developed, and over the last fifty years, it has been the driving force behind most work on sensory systems.
Horace B. Barlow was one of the first investigators to use the concept of the feature detector to relate the receptive field of a neuron to a specific animal behavior. In 1953, H.B. Barlow's electrophysiological recordings from excised retina of the frog provided the first evidence for the presence of an inhibitory surround in the receptive field of a frog's retinal ganglion cell. In reference to "on-off" ganglion cells—which respond to both the transition from light to dark and the transition from dark to light—and also had very restricted receptive fields of visual angle (about the size of a fly at the distance that the frog could strike), Barlow stated, "It is difficult to avoid the conclusion that the 'on-off' units are matched to the stimulus and act as fly detectors". In the same year, Stephen Kuffler published in vivo evidence for an excitatory center, inhibitory surround architecture in the ganglion cells of the mammalian retina which further supported Barlow's suggestion that on-off units can code for behaviorally relevant events.
Barlow's idea that certain cells in the retina could act as "feature detectors" was influenced by E.D. Adrian and Nikolaas Tinbergen. E.D. Adrian, Barlow's advisor, was the discoverer of the frequency code—the observation that sensory nerves convey signal intensity though the frequency of their firing. On the other hand, during Barlow's career, Nikolaas Tinbergen was introducing the concept of the innate release mechanism (IRM) and sign stimulus. IRMs are hard wired mechanisms that give an animal the innate ability to recognize complex stimuli. The sign stimulus is a simple, reduced stimulus including only the necessary features of the stimulus capable of evoking a behavioral response. Tinbergen's examination of the pecking behavior in herring gull chicks illustrated that the pecking response could be evoked by any bill-shaped long rod with a red spot near the end. In his own paper, Barlow later compared a sign stimulus to a password which was either accepted or rejected by a feature detector. Accepted passwords would contain the features necessary to trigger specific behavioral responses in an animal.
In the late 1950s, Jerome Lettvin and his colleagues began to expand the feature detection hypothesis and clarify the relationship between single neurons and sensory perception. In their paper "What the Frog's Eye Tells the Frog's Brain", Lettvin et al. (1959) looked beyond the mechanisms for signal-noise discrimination in the frog's retina and were able to identify four classes of ganglion cells in the frog retina: sustained contrast detectors, net convexity detectors (or bug detectors), moving edge detectors, and net dimming detectors.
In the same year, David Hubel and Torsten Wiesel began investigating properties of neurons in the visual cortex of cats, processing in the mammalian visual system. In their first paper in 1959, Hubel and Wiesel took recordings from single cells in the striate cortex of lightly anesthetized cats. The retinas of the cats were stimulated either individually or simultaneously with spots of light of various sizes and shapes. From the analysis of these recordings, Hubel and Wiesel identified orientation-selective cells in the cat's visual cortex and generated a map of the receptive field of cortical cells. At the time, circular spots of light were used as stimuli in studies of the visual cortex. However, Hubel and Wiesel noticed that rectangular bars of light were more effective stimuli (i.e. more natural stimuli) than circular spots of light, as long as the orientation was adjusted to the correct angle appropriate for each ganglion cell. These so-called simple cells were later called bar detectors or edge detectors. While comparing the receptive fields of neurons in the cat striate cortex with the concentric "on" and "off" receptive fields identified in cat ganglion cells by Kuffler et al., Hubel and Wiesel noticed that, although "on" and "off" regions were present in the striate cortex, they were not arranged in concentric circles. From their discovery of these uniquely orienting receptive fields, Hubel and Wiesel concluded that orientation-selective cells exist within the cat's visual cortex.
In their second major paper, Hubel and Wiesel extended their technique to more complex regions in the visual cortex in an effort to understand the difference between cortical receptive fields and lateral geniculate fields. They observed that the cat striate cortex contained more cells than the lateral geniculate, and they reasoned that the cortex needs a large number of neurons to digest the large amount of information it receives. Through experimentation, they found that each neuron in the cortex is responsible for a small region of the visual field and also has its own orientation specificity. From the results of these single cell readings in the striate cortex and lateral geniculate, Hubel and Wiesel postulated that simple cortical receptive fields gain complexity and an intricate spatial arrangement through the patterned convergence of multiple "on" or "off" projections from lateral geniculate cells onto single cortical cells.
Hubel and Wiesel's investigation of the cat visual cortex sparked interest in the feature detection hypothesis and its relevance to other sensory systems. In fact, T.H. Bullock contended in 1961 that the vestibular system was being ignored by most of the contemporary sensory system research, and he suggested that the equivalent stimulation of vestibular organs may yield similarly intriguing results. Hubel and Wiesel's work also raised the question: How far does the hierarchy of visual processing go? In one answer to this question, Lettvin coined the term grandmother cells in 1969 to describe hypothetical cells that are so specific that they only fire when your grandmother's face is viewed.
Examples
In toad vision
Jörg-Peter Ewert pioneered the study of the neuronal correlates of feature detection in toad vision.Toad Video: Image Processing in the Visual System of the Common Toad — Behaviour, Brain Function, Artificial Neuronal Net. https://doi.org/10.3203/IWF/C-1805eng He made significant progress by taking advantage of the common toad's natural prey catching behavior.J.-P. Ewert (1968): "Der Einfluss von Zwischenhirndefekten auf die Visuomotorik im Beute- und Fluchtverhalten der Erdkröte (Bufo bufo L.)" In: Z. verg. Physiol. 61, 41-70. J.-P. Ewert (1984) Tectal mechanisms that underlie prey-catching and avoidance behavior in toads, Chapter 11 In: H. Vanegas (ed.) Comparative Neurology of the Optic Tectum. Plenum, pp.247-416
Toads (Bufonidae) prefer to feed on arthropods such as earthworms, carabid beetles, and millipedes, whose bodies are oriented parallel to the direction of their movement. Toads avoid large objects such as the shadows of birds of prey. Snakes—the 'arch enemies' of toads—send out threatening visual signals with their body coils that are typically aligned across the snake's movement direction. These behaviorally relevant features are referred to in laboratory jargon as 'worm' and 'anti-worm.' An evolutionarily acquired anti-worm label that marks worm-like objects can protect them from prey motivated toads by 'snake mimicry,' as quantitative experiments with dummies suggest Ewert J.-P., Speckhardt I., Amelang W. (1970)Visuelle Inhibition und Excitation im Beutefangverhalten der Erdkröte (Bufo bufo L.) (Visual inhibition and excitation in the prey-catching behavior of the common toad (Bufo bufo L.) In: Z. vgl. Physiol. 68, 84-110. (see Vision in toads: a worm-like object that moves with a transversely arranged spot is ignored as prey).
To study toad behavior, Ewert and coworkers placed the toad in a cylindrical glass container at a fixed distance from the stimulus. They then moved a rectangular bar around the container in an effort to mimic a worm-like prey object. The toad's rate of turning toward the object was used to quantify the toad's orienting behavior. The team showed, by using spots, bars, and square stimuli of different sizes, that toads snapped at a bar which was moving in a direction parallel to its long axis, whereas the same bar oriented across the direction of movement (anti-worm configuration) was ignored as prey. Another experimental setup allowed worm or anti-worm stimuli to traverse the toad's visual field in different direction in the x-y co-ordinates, demonstrating that the worm versus anti-worm discrimination is invariant under changes in the direction of movement.
The characteristics of the moving worm (prey-like) and the moving anti-worm (threatening) are not explicitly determined by factors such as a horizontal or vertical alignment. Rather, following the conceptual framework by Nikolaas Tinbergen, these characteristics are determined by the proportion of two distinct features in their dynamic gestalt Tinbergen, N. (1989) The Study of Instinct. Oxford University Press, New York, pp. 228 Burghagen H., Ewert J.-P. (2022) Stimulus perception. In: Bolhuis J.J., Giraldeau L.-A., J.A.Hogan (eds.) The Behavior of Animals: Mechanisms, Function, and Evolution. 2nd Edn. John Wileys & Sons, Inc, pp.12-47. https://doi.org/10.1002/9781119109556.ch2 (configuration), namely the spatial extents of area parallel to vs. across the direction of movement: configurational form vision.
More precisely, configuration in this context refers to the relationship between two visual features of a moving bar: the area ep that is aligned parallel to the direction of movement (and signals prey), and the area ea (or et) that is aligned across (transverse) the direction of movement (and signals threat). The ea variation concerns spatial aspects of the visual stimulus, while the ep variation addresses both spatial and temporal aspects.