Comprehensive Physiology Wiley Online Library

Neural Mechanisms of High‐Level Vision

Full Article on Wiley Online Library



ABSTRACT

The last three decades have seen major strides in our understanding of neural mechanisms of high‐level vision, or visual cognition of the world around us. Vision has also served as a model system for the study of brain function. Several broad insights, as yet incomplete, have recently emerged. First, visual perception is best understood not as an end unto itself, but as a sensory process that subserves the animal's behavioral goal at hand. Visual perception is likely to be simply a side effect that reflects the readout of visual information processing that leads to behavior. Second, the brain is essentially a probabilistic computational system that produces behaviors by collectively evaluating, not necessarily consciously or always optimally, the available information about the outside world received from the senses, the behavioral goals, prior knowledge about the world, and possible risks and benefits of a given behavior. Vision plays a prominent role in the overall functioning of the brain providing the lion's share of information about the outside world. Third, the visual system does not function in isolation, but rather interacts actively and reciprocally with other brain systems, including other sensory faculties. Finally, various regions of the visual system process information not in a strict hierarchical manner, but as parts of various dynamic brain‐wide networks, collectively referred to as the “connectome.” Thus, a full understanding of vision will ultimately entail understanding, in granular, quantitative detail, various aspects of dynamic brain networks that use visual sensory information to produce behavior under real‐world conditions. © 2017 American Physiological Society. Compr Physiol 8:903‐953, 2018.

Figure 1. Figure 1. The abstract painting Interchanged (1955) by Willem de Kooning. At a reported purchase price of US $300 million, it is one of the most expensive paintings in the world. See text for additional details.
Figure 2. Figure 2. The importance of recurrent processing in visual perception. When viewed for the first time, this two‐tone “Mooney” image appears to be an unrecognizable pattern of black and white blobs. That is, it is hard to interpret this image based on sensory information (i.e., bottom‐up or feed‐forward processing) alone. However, after viewing a full grayscale or color counterpart (for which see Figure 3A—which, being smaller in size, has no pixel‐to‐pixel correspondence with the above image), the Mooney image becomes easy to interpret. Note that viewing the disambiguating image rapidly, drastically, and enduringly alters our perception of the Mooney image, although the Mooney image itself remains physically unchanged. Feed‐forward theories of vision cannot help explain such phenomena. Recurrent (or “reentrant”) neural signals bring to bear such top‐down influences as prior knowledge and the behavioral context to help constrain the interpretation of the visual image. Learning to interpret Mooney images can be understood as an extreme case of knowledge‐mediated disambiguation that is part and parcel of normal visual perception ().
Figure 3. Figure 3. Some complexities of natural images and the information processing required to making sense of them. Panels A through D show real‐world scenes that help illustrate some of the complexities of such scenes. For instance, there are multiple objects of the same type in each picture. They all vary greatly in image size, illumination, shadows, occlusion, viewpoint, etc. But the visual system recognizes them as objects of the same kind. In other words, the brain must be able to discard a variety of image features as irrelevant to recognizing the images. Note that, in case of each scene in this figure, our understanding of the scene evolves over time. Note also that the objects, their spatial relationships, and even the semantic similarities and differences among them are such that the percept that each scene elicits is more than just the sum of the parts. In addition, each picture contains implicit cues to motion and/or depth that static, 2D pictures such as these do not do justice to. In fact, natural scenes differ from each other and from the relatively simple stimuli used in many a vision study, such as a sinusoidal grating on a neutral gray background, in myriad ways. Studies have shown that the visual system is adapted to contend with the statistics of natural scenes (), so that neural responses to “artificial” stimuli may provide a substantially different and potentially misleading picture of how the brain works. On the other hand, note that our visual system performs very well in recognizing scenes with “unnatural” statistics, such as the street scene in panel C and the indoor scene in panel D, even though it evidently did not encounter the “unnatural” statistics of human‐made objects until quite recently on the evolutionary time scale. This is because the brain is quite good at adapting to a variety of nonoptimal inputs.
Figure 4. Figure 4. Visual system can often recognize objects with great precision with very little information. A veridical description of the object, even if it were possible, is not necessary. (A) In the picture on the left, very little of the dog is visible. But we have little trouble recognizing the dog. Many probably can even readily name the breed of the dog. (B) Few would have trouble recognizing the person from the picture. (C) The tell‐tale hat. For most readers, this picture simply shows a hat hung on some plumbing. But this famous picture, on the cover of the very first issue of Physics Today in May 1948, showed the famous “pork pie” hat of J. Robert Oppenheimer, the father of the atomic bomb. Therefore, the picture was highly topical and was readily understood at the time. But for most of us living today, this picture requires either some historical knowledge or explanation. Thus, our perception of a visual image is influenced by a great many nonvisual factors. Neurophysiological understanding of such influences on cognition represents a major challenge.
Figure 5. Figure 5. There is more to vision than feed‐forward processing. This picture of an unknown little girl was taken by South African photojournalist Kevin Carter in the famine‐stricken South Sudan in March 1993. The girl had reportedly collapsed from weakness on her way to a United Nations food center. (Reproduced, with permission, from The Vulture and the Little Girl by Kevin Carter. Pulitzer Prize for Feature Photography, 1994.) Note that feed‐forward processing by itself would completely miss the import of the picture.
Figure 6. Figure 6. (A) A self‐driving car. (B) Two instances of the CAPTCHA (or Completely Automated Public Turing test to tell Computers and Humans Apart) internet device designed to prevent automated logins (). In the example on the left, the website asks the user to perform a straightforward object categorization task, namely distinguish pictures of people wearing glasses from pictures of people without glasses. This tends to be quite successful in preventing machines from logging on to the website. However, this success is not so much because it would be all that difficult nowadays to train a suitably designed computer program to distinguish the two categories of human faces (see, e.g., ()), but essentially because, at present, such programs tend to be hyperspecialized, and tend not to generalize beyond their training sets to other visual objects or tasks (). That is, such an “intelligent” program, once trained to tell aforementioned types of faces apart, can be readily stumped at present by rather slight changes in the underlying categories (panel B, right), or the task (not shown) (). However, recent research suggests that the problem of overspecialization of intelligent machines is likely to straightforwardly surmountable, so that CAPTCHA strategies such as the one illustrated in this panel are unlikely to be effective for long. That is, machine vision is getting ever better at mimicking human high‐level vision ().
Figure 7. Figure 7. An outline of Marr's theory of visual processing. See text for details.
Figure 8. Figure 8. Visual anatomical hierarchy in the macaque monkey described by Felleman and Van Essen, 1991 (). Each colored rectangle represents a distinct cortical visual area. The open rectangles at top denote higher cortical areas that are not considered primarily visual area. The two gray rectangles at bottom denote the retinal ganglion cells (RGC) and the lateral geniculate nucleus (LGN). Lines connecting the cortical areas denote interconnections, usually reciprocal, between a given pairs of areas. The various brain regions are represented in a tiered, or hierarchical, fashion based on objective anatomical criteria, most important of which are the laminar patterns of feed‐forward and feedback connections (also see ()). For additional details and abbreviations, see Felleman and Van Essen, 1991 (). An alternative formulation of the hierarchy, originally formulated by Mishkin, Gross, and their colleagues () (also see ()) is largely similar, but it parcels the cortex into many fewer areas and recognizes fewer interconnections. Also, some the visual area names are different in this scheme. For instance, areas AIT and CIT (anterior and central inferotemporal areas, respectively) in the Felleman and Van Essen scheme are equivalent to area TE (temporal area) in the Gross et al. scheme, and area PIT (posterior inferotemporal area) in the Van Essen scheme is equivalent to area TEO (temporooccipital area) in the Gross et al. scheme. Both schemes are used in this review, based on the scheme used by the study in question. Reproduced, with permission, from ().
Figure 9. Figure 9. Our evolving understanding of the functional organization of the primate visual system. Panels A and B depict our view of the two main visual processing streams in the macaque brain in the early 1980s. (A) The dorsal and ventral visual pathways as originally formulated by Mishkin and colleagues in 1993 is denoted by solid arrows (OB → OA → PG pathway being the anatomically dorsal pathway, and OB → OA → TEO → TE being the ventral pathway) (). The continuation of these pathways to FDΔ and FDv, respectively (dashed arrows) represented the collective outcome of many additional studies. (B) Anatomical locations of the various regions, some renamed according to a more modern naming convention (). (C) Key changes in the receptive field properties of neurons along the ventral pathway. Areas are color‐coded as in panel B. Panel D summarizes our understanding of the same pathways some 30 years later, as summarized by Kravitz and colleagues in 2013 (). Note that it is clear that the two pathways have turned out to be far more interconnected than previously envisioned. What is lost in terms of pedagogical simplicity is more than made up for by the nuance and granularity of this network picture, presaged decades ago by Kerrigan and Maunsell (). Self‐evidently, far more remains to be learned, including how these networks interact with other networks in the brain and subserve behavior. Adapted, with permission, from (). Human brain (not shown) is also purported to have two evolutionarily homologous processing pathways, although there is even less empirical information to support this notion.
Figure 10. Figure 10. What's to infer? Is not it all there in the image? The answer is no. Retinal image is simply a 2D pattern of image intensities. Image intensities of a particular image are represented in panel A as a color‐coded surface plot, where the height and color of a point denotes the image intensity at that point. Note that when the image is represented in this fashion, it makes no sense to us. But when the same image is represented as corresponding variations in image intensities (panel B), we readily recognize it as an image of a brook in the woods. However, the information in the two representations is exactly the same. The difference is that, in panel B, the image is in an input “format” that our eyes can process. Beyond that, the inferential processes to “make sense” of the pattern of intensities are exactly the same.
Figure 11. Figure 11. Visual illusions help illustrate the inferential nature of visual perception. They also demonstrate that the inference need not be a deliberate, volitional, or conscious process. (A) Ames room. Two people stand in opposite corners of the room. One appears to be much taller than the other, even though they are roughly the same height. Instead, it is the room that is distorted to produce this illusion. Picture courtesy of Ian Stannard, Flickr. Reproduced with permission. (B) Hollow‐mask illusion. This picture shows the front of the mask (right) and the hollow back of the mask (left). Nonetheless, both look like normal, convex faces. For a video demonstration of the hollow mask illusion, see https://www.youtube.com/watch?v=sKa0eaKsdA0. Note that, in this video, the mask appears to flip its direction of rotation at the same time the other side of the mask begins appearing. Theoretical studies show that visual illusions are often perfectly rational inferences given the evidence (see, e.g., ()).
Figure 12. Figure 12. The spatiotemporal domain of the methods available for the study of the functional organization of nervous system in 2014, compared to the methods available in 1988 (inset). Each colored region represents a range of spatial and temporal resolutions for a given method. Open regions represent measurement techniques; filled regions, perturbation techniques; EEG, electroencephalography; MEG, magnetoencephalography; PET, positron emission tomography; VSD, voltage‐sensitive dye; TMS, transcranial magnetic stimulation; and 2‐DG, 2‐deoxyglucose. Redrawn, with permission, from ().
Figure 13. Figure 13. Neuronal responses in monkey visual area V1 during binocular rivalry. See text for details. Adapted, with permission, from ().
Figure 14. Figure 14. Figure‐ground segregation and perceptual organization. (A) How many circles can you see in this image? In this image, referred to as the Coffer Illusion, you should be able to see 16 circles. Figure courtesy of Dr. Anthony Norcia, Stanford University. Reproduced with permission. (B) Camouflage is an extreme case of figure‐ground segregation, where the object of interest is hard to recognize even when “in plain view.” This image shows two variants of the pepper moth Biston betularia, one black and the other with light peppered coloring. The black variant is effectively camouflaged against colored tree bark whereas the light variant is easy to recognize (i.e., it “pops out”). The opposite is true when the same two variants are seen against a background of light bark and lichens. The black variant emerged for the first time in the industrial midlands of Britain in the 19th century, where tree barks were turning black with industrial soot. Soon the black variants became the more common variant, because the predators of the moths had much greater success breaking the camouflage of the lighter variants because it was harder for the lighter variants to find light backgrounds to camouflage themselves against. This was an instance of the prey “gaming” the predators’ high‐level visual faculties to enhance its own survival (). Figure from Ford, E.G. (1977) Ecological Genetics. Springer. Used with permission.
Figure 15. Figure 15. Selectivity for faces in the macaque superior temporal polysensory area (STP). (A) Anatomical location of area STP (yellow highlight). (B) Responses of a single STP neuron to various visual objects, including variations in face stimuli. Note that, among the stimuli tested, the neuron responds best to a face with all the key facial features (left column, second stimulus from top). Cutting this stimulus into 16 pieces and showing the pieces in a shuffled order essentially eliminated the response (right column, third stimulus form top). The icon at bottom right denotes the size and visual field location of the receptive field. C, contralateral visual field (i.e., contralateral to the recording location). I, ipsilateral. Such neurons with large receptive fields that span both the visual hemifields are common in the visually responsive areas of the central and anterior temporal cortex. Adapted, with permission, from ().
Figure 16. Figure 16. Coarse‐to‐fine tuning of shape categories in the macaque IT. The stimulus set consisted of 38 stimuli, a subset of which is shown in the inset. The global shape categories (inset, vertical axis) consisted of human faces, monkey faces, and geometric shapes. The fine categories (inset, horizontal axis) consisted of the various facial identities and expressions. The plots show the cumulative information transmission rate of a sample of IT neurons about both the global and fine categories (red and blue lines, respectively). The thick horizontal line along the x axis denotes the stimulus duration. Adapted, with permission, from ().
Figure 17. Figure 17. Grid cells help represent the external visual space. Grid cells were originally reported in rats, but have since been found in many species, including monkeys. (A) Responses of a single grid cell in the entorhinal cortex of the rat. Left, black lines denote the trajectory of a rat freely moving in a box. The cell fired spikes (red dots) when the rat was at specific locations within the box. These locations were organized in a grid‐like fashion that spanned the box. Right, the firing rates of the cell represented as a heatmap, where “warmer” colors denote higher firing rates. Adapted, with permission, from (). (B) Responses of a single grid cell in the entorhinal cortex of the macaque monkey. Unlike the rat referred to in panel A, the monkey was stationary. It sat in a primate chair with its head held steady, but freely moved its eyes as it looked at real‐world pictures (not shown). Left, red dots denote the locations in a picture (not shown) that the monkey fixated, or gazed steadily at for a brief period. Center, firing rate of the grid cell shown in heatmap format. The scale bars at bottom each denote 6° of visual angle. Adapted, with permission, from ().
Figure 18. Figure 18. Coding of visual context. The classical receptive field (CRF) of a given neuron is the portion of the visual field in which the neuron is most responsive to visual stimuli. The surrounding region in which visual stimulation modulates the responses to CRF stimulation is referred to as the nonclassical receptive field (nCRF) or nonclassical surround. This figure shows one of the earliest demonstrations of the modulatory effect of nCRF on CRF, in this case the response of a single neuron in macaque MT. Individual neurons in MT often response best when the stimulus moves in a particular direction, often referred to as its preferred or optimal direction. The neuron shown responded best when the dots in the CRF moved horizontally from left to right. CRF is denoted by the dashed rectangle in the icons at top. (Left panel) Modulatory effect of stationary surround on motion stimuli in the CRF. The direction of the movement of dots in the CRF was systematically varied, while the dots in the nCRF were held stationary. (Right panel) Modulatory effect of surround motion on motion stimuli in the CRF. The responses of the same neuron when the dots moved in its optimal direction, while the direction of the dots in the nCRF was systematically varied. Note that the neuron's responses vary systematically (i.e., the responses are “tuned”) with respect to both CRF motion, and motion in both the center and surround. Thus, neuron can convey information of the motion “context,” or motion of a given moving object relative to nearby stationary or moving objects. Adapted, with permission, from ().
Figure 19. Figure 19. Corticostriatal loops in human brain. (A) Four distinguishable but mutually overlapping loops are usually recognized (colored labeled arrows), based primarily on the types of tasks in which they play prominent roles. Cortical inputs arrive largely via the striatum and ultimately are directed back into the cortex via the thalamus. The (ultimate) cortical output of the basal ganglia reaches largely to the same cortical areas that give rise to the initial inputs to the basal ganglia. The visual loop is known to play a prominent role in the learning of visual object categories, but during object categorization tasks using learned categories, the executive loop also plays a prominent role. Corticostriatal loops in the nonhuman primate brain are largely similar (not shown). Adapted, with permission, from (); also see (). (B) A more detailed circuit map of the visual loop shows the flow of information within the loop. GPe: Globus pallidus, external portion. GPi: Globus pallidus, internal portion. SNr: Substantia nigra pars reticulata. SNc: Substantia nigra pars compacta. STN: Subthalamic nucleus. VTA: Ventral tegmental area. Adapted, with permission, from ().
Figure 20. Figure 20. An illustration of Granger causality. This figure shows cause and effect relationship between two hypothetical neural responses (top and bottom rows, respectively). The cause‐and‐effect relationship exists throughout the responses, but is most readily apparent by visual inspection of those portions of the responses where the response is prominently modulated (arrows). Granger causality uses the entire length of both responses to quantitatively measure the cause‐and‐effect relationship even when the relationship may be too subtle or complex to be visually evident. In the present case, the response shown in the top row is said to “Granger‐cause” the response in the bottom row. Note that the term “Granger causation” denotes an inferred cause‐and‐effect relationship, which may or not include direct causation. Thus, the concept of Granger causality is in some respects narrower, and in some other respects broader, than the concept of direct causation (). But in either case, the cause must necessarily precede the effect.
Figure 21. Figure 21. Resting state brain networks are very similar to brain networks active during tasks. (A) Smith and colleagues () extracted 20 mutually independent patterns of activation in the resting state networks (RSNs) from a database of 36 adult human subjects using the independent components analysis (ICA) (). Brain regions identified by each of these ICAs can be thought of as an independent network. Smith and colleagues then compared RSN ICAs with ICAs of task‐activated networks from nearly 30,000 subjects from the BrainMap (BM) database (brainmap.org). Comparisons for ten most informative ICAs are shown side by side in this panel. For each ICA, activation is shown in a color‐coded format for a coronal, sagittal, and horizontal section (top, middle, bottom row, respectively). (B) Smith and colleagues () then analyzed the extent to which the top ten of the RSN ICAs play a role in various types of behavioral tasks (or “behavioral task domains” defined by the BrainMap database). Higher color values denote a correspondingly larger role by the given network in a given task. Note that each given type of task recruits different RSNs to different extents. Conversely, each RSN is active during multiple, different task paradigms, with the degree of participation varying according to the task. Thus, the RSNs represent a repertoire that the brain recruits and employs to various degrees depending on the task at hand. For details and some important caveats, see (). Adapted, with permission, from ().
Figure 22. Figure 22. Networks in human brain at rest. Nodes denote the center of mass of the corresponding brain regions, and the edges (i.e., colored lines) represent intrinsic connections between a pair of regions identifiable in the resting brain. Blue nodes represent the “rich club” brain regions, which are well‐connected brain regions that are well‐connected with other well‐connected brain regions. Gray dots denote nonrich club regions. Red lines denote connections between rich club regions. Orange lines denote “feeder” connections that connect a rich club region with a nonrich club region. Yellow lines denote “feeder” connections that connect a nonrich club region with another nonrich club region. Adapted, with permission, from (). Note that this figure does not show subcortical or cerebellar networks, which decidedly play crucial roles in brain function.
Figure 23. Figure 23. Neuronal responses in monkey middle temporal area (MT) reflect perceptual decisions in awake, behaving monkeys. (A) Stimuli and task paradigm. The studies by Newsome and colleagues () () used dots moving either in the preferred direction of the cell under study, or in the opposite (“null”) direction, depending on the trial. The investigators “tuned” the strength of the motion information by changing the proportion of dots that moved in the same direction (i.e., percentage correlation of motion). (B) The “neurometric function” (solid dots and solid fitted curve) an individual neuron that closely paralleled the “psychometric function” of the animal's behavioral responses (open dots and dashed fitted curve). (C) The close parallels between the neurometric versus psychometric functions held across different animal subjects. The dashed line notwithstanding indicates the best fitting linear trend. (D) Demonstration of a causal relationship between the responses of individual neurons and the animal's percepts using microstimulation. The psychometric function of an animal with or without microstimulation (solid dots and fitted line and open and dashed fitted line, respectively). See text and () for details. Adapted, with permission, from ().
Figure 24. Figure 24. A schematic illustration of correlation, decorrelation, and sparsening of the responses at the population level during the initial rapid transient responses (panel A), or at later stages (panels B‐E). Each panel shows a highly idealized “population” consisting of four neurons (circles). Each quadrant of a given circle denotes the response of the neuron to a given stimulus, color‐coded according to the color scale at bottom left. See text for details. Adapted, with permission, from ().
Figure 25. Figure 25. Visual cliff demonstrates development of depth perception. (A) The visual cliff is a laboratory apparatus that helps test depth perception in human infants and animals. It consists of an actual cliff covered with a sturdy but transparent plexiglass (). The cliff is textured with a high‐contrast checkerboard pattern, so that the cliff is clearly visible through the plexiglass. An infant called by his mother from the opaque side of the apparatus readily crawls to her (). On the other hand, he is reluctant to venture over the perceived cliff (panel B). Even when the infants know by patting the glass that it is solid, they still tend to be reluctant to cross. Infants’ decision as to whether or not to cross the visual cliff are also influenced by whether the gestures of the parent are encouraging, neutral, or discouraging (). Such behaviors show sophisticated inferences based on a joint evaluation of various depth cues, risks, and rewards. Studies show that healthy human infants have such depth perception even before they are able to crawl (). The visual cliff effect has been reported in many mammalian species (). For a video of visual cliff effect, see https://www.youtube.com/watch?v=p6cqNhHrMJA.
Figure 26. Figure 26. fMRI in young human infants. (A) Renderings of what infants at various ages are likely to see when they view a teddy bear. (B) Visual responses in the neonate. Visually responsive regions are located in the anterior aspect of the calcarine sulcus in either hemisphere. Moreover, the visually evoked responses are lower compared to the periods of rest. (C) Visual responses in a different 5‐month old infant. Visual stimulation activates a much posterior aspect of the calcarine sulcus. Also, the visually evoked responses are higher than the responses during rest. Panels B and C are courtesy of Dr. Ernst Martin () and reproduced with permission.
Figure 27. Figure 27. Preferential responses in the human cerebellum during high‐level cognitive task. Panel A shows the differential PET responses in a heatmap format, where brighter colors represent greater response. Panel B schematically summarizes the regions (red squares) that showed the task‐dependent preferential response. Note that the responses are highly lateralized. Adapted, with permission, from ().
Figure 28. Figure 28. Hemineglect. This figure shows the results from a drawing test () from a single patient with left hemineglect, resulting from a localized lesion in the right temporal lobe. The patient was asked the draw the dial of a clock. In most clinical cases, lesions tend to be less circumscribed and more widespread than in the patient whose drawing is shown here. Hence, drawings by most hemineglect patients tend to be much more complex, and less clear‐cut, than the “text book” case shown in this figure (see () for reviews). Figure courtesy of Scholarpedia.
Figure 29. Figure 29. Our evolving understanding of multimodal anatomical connections with the visual system. (A) Traditional view of the cortical anatomy of the primate brain recognized very few areas with multimodal anatomical connections (colored areas). (B) A more modern scheme of the cortical anatomy of multisensory areas. Colored areas represent regions where anatomical and/or electrophysiological studies have demonstrated multisensory interactions. Dashed gray outlines represent opened sulci. See () for details, including the criteria used for determining multimodal connectivity at the anatomical level. Adapted, with permission, from ().
Figure 30. Figure 30. Visual‐haptic object processing activates lateral occipital complex (LOC) in the occipitotemporal pathway. Ahmedi and colleagues () compared BOLD responses to four conditions: visual objects, somatosensory (or haptic) objects, visual textures, and haptic textures. Statistical map of the contralateral hemisphere from a single subject are shown in panel A (3D folded view), panel B (inflated view of the same hemisphere), and panel C (flattened view of the same hemisphere). Bottom, BOLD responses to the four conditions are shown in the somatosensory cortex (bottom left) and LOC in the occipitotemporal junction (bottom right). Col S, collateral sulcus; Cal S, calcarine sulcus; CS, central sulcus; IPS, intraparietal sulcus; lateral S, lateral sulcus; STS, superior temporal sulcus. Adapted, with permission, from ().
Figure 31. Figure 31. Color‐graphemic synesthesia. (A) (left) A stimulus that can elicit synesthesia in color‐graphemic synesthetes. (A) (right) A rendition of what the synesthete likely to have perceived. Note that, since color‐graphemic synesthetes tend to perceive different numbers as different colors, the triangle made up of 2’s stands out, or “pops out,” perceptually for them. By contrast, nonsynesthetes perceive all the numbers to be of the same color, so that for them, the triangle is not readily distinguishable from the background. (B) Neural responses during color‐graphemic synesthesia as measured by fMRI. BOLD responses to graphemic stimuli were contrasted against the responses to nongraphemic stimuli in synesthetes (left) and control subjects and results are rendered on inflated, bottom‐up views of brains of representative subjects. Both control subjects and synesthetes showed common activation of the “grapheme region” (Gr). In addition to this common activation, graphemes activated the color selective areas of the retinotopic region V4 (hV4) in synesthetes but not in nonsynesthetes. See () for details. Adapted, with permission, from ().


Figure 1. The abstract painting Interchanged (1955) by Willem de Kooning. At a reported purchase price of US $300 million, it is one of the most expensive paintings in the world. See text for additional details.


Figure 2. The importance of recurrent processing in visual perception. When viewed for the first time, this two‐tone “Mooney” image appears to be an unrecognizable pattern of black and white blobs. That is, it is hard to interpret this image based on sensory information (i.e., bottom‐up or feed‐forward processing) alone. However, after viewing a full grayscale or color counterpart (for which see Figure 3A—which, being smaller in size, has no pixel‐to‐pixel correspondence with the above image), the Mooney image becomes easy to interpret. Note that viewing the disambiguating image rapidly, drastically, and enduringly alters our perception of the Mooney image, although the Mooney image itself remains physically unchanged. Feed‐forward theories of vision cannot help explain such phenomena. Recurrent (or “reentrant”) neural signals bring to bear such top‐down influences as prior knowledge and the behavioral context to help constrain the interpretation of the visual image. Learning to interpret Mooney images can be understood as an extreme case of knowledge‐mediated disambiguation that is part and parcel of normal visual perception ().


Figure 3. Some complexities of natural images and the information processing required to making sense of them. Panels A through D show real‐world scenes that help illustrate some of the complexities of such scenes. For instance, there are multiple objects of the same type in each picture. They all vary greatly in image size, illumination, shadows, occlusion, viewpoint, etc. But the visual system recognizes them as objects of the same kind. In other words, the brain must be able to discard a variety of image features as irrelevant to recognizing the images. Note that, in case of each scene in this figure, our understanding of the scene evolves over time. Note also that the objects, their spatial relationships, and even the semantic similarities and differences among them are such that the percept that each scene elicits is more than just the sum of the parts. In addition, each picture contains implicit cues to motion and/or depth that static, 2D pictures such as these do not do justice to. In fact, natural scenes differ from each other and from the relatively simple stimuli used in many a vision study, such as a sinusoidal grating on a neutral gray background, in myriad ways. Studies have shown that the visual system is adapted to contend with the statistics of natural scenes (), so that neural responses to “artificial” stimuli may provide a substantially different and potentially misleading picture of how the brain works. On the other hand, note that our visual system performs very well in recognizing scenes with “unnatural” statistics, such as the street scene in panel C and the indoor scene in panel D, even though it evidently did not encounter the “unnatural” statistics of human‐made objects until quite recently on the evolutionary time scale. This is because the brain is quite good at adapting to a variety of nonoptimal inputs.


Figure 4. Visual system can often recognize objects with great precision with very little information. A veridical description of the object, even if it were possible, is not necessary. (A) In the picture on the left, very little of the dog is visible. But we have little trouble recognizing the dog. Many probably can even readily name the breed of the dog. (B) Few would have trouble recognizing the person from the picture. (C) The tell‐tale hat. For most readers, this picture simply shows a hat hung on some plumbing. But this famous picture, on the cover of the very first issue of Physics Today in May 1948, showed the famous “pork pie” hat of J. Robert Oppenheimer, the father of the atomic bomb. Therefore, the picture was highly topical and was readily understood at the time. But for most of us living today, this picture requires either some historical knowledge or explanation. Thus, our perception of a visual image is influenced by a great many nonvisual factors. Neurophysiological understanding of such influences on cognition represents a major challenge.


Figure 5. There is more to vision than feed‐forward processing. This picture of an unknown little girl was taken by South African photojournalist Kevin Carter in the famine‐stricken South Sudan in March 1993. The girl had reportedly collapsed from weakness on her way to a United Nations food center. (Reproduced, with permission, from The Vulture and the Little Girl by Kevin Carter. Pulitzer Prize for Feature Photography, 1994.) Note that feed‐forward processing by itself would completely miss the import of the picture.


Figure 6. (A) A self‐driving car. (B) Two instances of the CAPTCHA (or Completely Automated Public Turing test to tell Computers and Humans Apart) internet device designed to prevent automated logins (). In the example on the left, the website asks the user to perform a straightforward object categorization task, namely distinguish pictures of people wearing glasses from pictures of people without glasses. This tends to be quite successful in preventing machines from logging on to the website. However, this success is not so much because it would be all that difficult nowadays to train a suitably designed computer program to distinguish the two categories of human faces (see, e.g., ()), but essentially because, at present, such programs tend to be hyperspecialized, and tend not to generalize beyond their training sets to other visual objects or tasks (). That is, such an “intelligent” program, once trained to tell aforementioned types of faces apart, can be readily stumped at present by rather slight changes in the underlying categories (panel B, right), or the task (not shown) (). However, recent research suggests that the problem of overspecialization of intelligent machines is likely to straightforwardly surmountable, so that CAPTCHA strategies such as the one illustrated in this panel are unlikely to be effective for long. That is, machine vision is getting ever better at mimicking human high‐level vision ().


Figure 7. An outline of Marr's theory of visual processing. See text for details.


Figure 8. Visual anatomical hierarchy in the macaque monkey described by Felleman and Van Essen, 1991 (). Each colored rectangle represents a distinct cortical visual area. The open rectangles at top denote higher cortical areas that are not considered primarily visual area. The two gray rectangles at bottom denote the retinal ganglion cells (RGC) and the lateral geniculate nucleus (LGN). Lines connecting the cortical areas denote interconnections, usually reciprocal, between a given pairs of areas. The various brain regions are represented in a tiered, or hierarchical, fashion based on objective anatomical criteria, most important of which are the laminar patterns of feed‐forward and feedback connections (also see ()). For additional details and abbreviations, see Felleman and Van Essen, 1991 (). An alternative formulation of the hierarchy, originally formulated by Mishkin, Gross, and their colleagues () (also see ()) is largely similar, but it parcels the cortex into many fewer areas and recognizes fewer interconnections. Also, some the visual area names are different in this scheme. For instance, areas AIT and CIT (anterior and central inferotemporal areas, respectively) in the Felleman and Van Essen scheme are equivalent to area TE (temporal area) in the Gross et al. scheme, and area PIT (posterior inferotemporal area) in the Van Essen scheme is equivalent to area TEO (temporooccipital area) in the Gross et al. scheme. Both schemes are used in this review, based on the scheme used by the study in question. Reproduced, with permission, from ().


Figure 9. Our evolving understanding of the functional organization of the primate visual system. Panels A and B depict our view of the two main visual processing streams in the macaque brain in the early 1980s. (A) The dorsal and ventral visual pathways as originally formulated by Mishkin and colleagues in 1993 is denoted by solid arrows (OB → OA → PG pathway being the anatomically dorsal pathway, and OB → OA → TEO → TE being the ventral pathway) (). The continuation of these pathways to FDΔ and FDv, respectively (dashed arrows) represented the collective outcome of many additional studies. (B) Anatomical locations of the various regions, some renamed according to a more modern naming convention (). (C) Key changes in the receptive field properties of neurons along the ventral pathway. Areas are color‐coded as in panel B. Panel D summarizes our understanding of the same pathways some 30 years later, as summarized by Kravitz and colleagues in 2013 (). Note that it is clear that the two pathways have turned out to be far more interconnected than previously envisioned. What is lost in terms of pedagogical simplicity is more than made up for by the nuance and granularity of this network picture, presaged decades ago by Kerrigan and Maunsell (). Self‐evidently, far more remains to be learned, including how these networks interact with other networks in the brain and subserve behavior. Adapted, with permission, from (). Human brain (not shown) is also purported to have two evolutionarily homologous processing pathways, although there is even less empirical information to support this notion.


Figure 10. What's to infer? Is not it all there in the image? The answer is no. Retinal image is simply a 2D pattern of image intensities. Image intensities of a particular image are represented in panel A as a color‐coded surface plot, where the height and color of a point denotes the image intensity at that point. Note that when the image is represented in this fashion, it makes no sense to us. But when the same image is represented as corresponding variations in image intensities (panel B), we readily recognize it as an image of a brook in the woods. However, the information in the two representations is exactly the same. The difference is that, in panel B, the image is in an input “format” that our eyes can process. Beyond that, the inferential processes to “make sense” of the pattern of intensities are exactly the same.


Figure 11. Visual illusions help illustrate the inferential nature of visual perception. They also demonstrate that the inference need not be a deliberate, volitional, or conscious process. (A) Ames room. Two people stand in opposite corners of the room. One appears to be much taller than the other, even though they are roughly the same height. Instead, it is the room that is distorted to produce this illusion. Picture courtesy of Ian Stannard, Flickr. Reproduced with permission. (B) Hollow‐mask illusion. This picture shows the front of the mask (right) and the hollow back of the mask (left). Nonetheless, both look like normal, convex faces. For a video demonstration of the hollow mask illusion, see https://www.youtube.com/watch?v=sKa0eaKsdA0. Note that, in this video, the mask appears to flip its direction of rotation at the same time the other side of the mask begins appearing. Theoretical studies show that visual illusions are often perfectly rational inferences given the evidence (see, e.g., ()).


Figure 12. The spatiotemporal domain of the methods available for the study of the functional organization of nervous system in 2014, compared to the methods available in 1988 (inset). Each colored region represents a range of spatial and temporal resolutions for a given method. Open regions represent measurement techniques; filled regions, perturbation techniques; EEG, electroencephalography; MEG, magnetoencephalography; PET, positron emission tomography; VSD, voltage‐sensitive dye; TMS, transcranial magnetic stimulation; and 2‐DG, 2‐deoxyglucose. Redrawn, with permission, from ().


Figure 13. Neuronal responses in monkey visual area V1 during binocular rivalry. See text for details. Adapted, with permission, from ().


Figure 14. Figure‐ground segregation and perceptual organization. (A) How many circles can you see in this image? In this image, referred to as the Coffer Illusion, you should be able to see 16 circles. Figure courtesy of Dr. Anthony Norcia, Stanford University. Reproduced with permission. (B) Camouflage is an extreme case of figure‐ground segregation, where the object of interest is hard to recognize even when “in plain view.” This image shows two variants of the pepper moth Biston betularia, one black and the other with light peppered coloring. The black variant is effectively camouflaged against colored tree bark whereas the light variant is easy to recognize (i.e., it “pops out”). The opposite is true when the same two variants are seen against a background of light bark and lichens. The black variant emerged for the first time in the industrial midlands of Britain in the 19th century, where tree barks were turning black with industrial soot. Soon the black variants became the more common variant, because the predators of the moths had much greater success breaking the camouflage of the lighter variants because it was harder for the lighter variants to find light backgrounds to camouflage themselves against. This was an instance of the prey “gaming” the predators’ high‐level visual faculties to enhance its own survival (). Figure from Ford, E.G. (1977) Ecological Genetics. Springer. Used with permission.


Figure 15. Selectivity for faces in the macaque superior temporal polysensory area (STP). (A) Anatomical location of area STP (yellow highlight). (B) Responses of a single STP neuron to various visual objects, including variations in face stimuli. Note that, among the stimuli tested, the neuron responds best to a face with all the key facial features (left column, second stimulus from top). Cutting this stimulus into 16 pieces and showing the pieces in a shuffled order essentially eliminated the response (right column, third stimulus form top). The icon at bottom right denotes the size and visual field location of the receptive field. C, contralateral visual field (i.e., contralateral to the recording location). I, ipsilateral. Such neurons with large receptive fields that span both the visual hemifields are common in the visually responsive areas of the central and anterior temporal cortex. Adapted, with permission, from ().


Figure 16. Coarse‐to‐fine tuning of shape categories in the macaque IT. The stimulus set consisted of 38 stimuli, a subset of which is shown in the inset. The global shape categories (inset, vertical axis) consisted of human faces, monkey faces, and geometric shapes. The fine categories (inset, horizontal axis) consisted of the various facial identities and expressions. The plots show the cumulative information transmission rate of a sample of IT neurons about both the global and fine categories (red and blue lines, respectively). The thick horizontal line along the x axis denotes the stimulus duration. Adapted, with permission, from ().


Figure 17. Grid cells help represent the external visual space. Grid cells were originally reported in rats, but have since been found in many species, including monkeys. (A) Responses of a single grid cell in the entorhinal cortex of the rat. Left, black lines denote the trajectory of a rat freely moving in a box. The cell fired spikes (red dots) when the rat was at specific locations within the box. These locations were organized in a grid‐like fashion that spanned the box. Right, the firing rates of the cell represented as a heatmap, where “warmer” colors denote higher firing rates. Adapted, with permission, from (). (B) Responses of a single grid cell in the entorhinal cortex of the macaque monkey. Unlike the rat referred to in panel A, the monkey was stationary. It sat in a primate chair with its head held steady, but freely moved its eyes as it looked at real‐world pictures (not shown). Left, red dots denote the locations in a picture (not shown) that the monkey fixated, or gazed steadily at for a brief period. Center, firing rate of the grid cell shown in heatmap format. The scale bars at bottom each denote 6° of visual angle. Adapted, with permission, from ().


Figure 18. Coding of visual context. The classical receptive field (CRF) of a given neuron is the portion of the visual field in which the neuron is most responsive to visual stimuli. The surrounding region in which visual stimulation modulates the responses to CRF stimulation is referred to as the nonclassical receptive field (nCRF) or nonclassical surround. This figure shows one of the earliest demonstrations of the modulatory effect of nCRF on CRF, in this case the response of a single neuron in macaque MT. Individual neurons in MT often response best when the stimulus moves in a particular direction, often referred to as its preferred or optimal direction. The neuron shown responded best when the dots in the CRF moved horizontally from left to right. CRF is denoted by the dashed rectangle in the icons at top. (Left panel) Modulatory effect of stationary surround on motion stimuli in the CRF. The direction of the movement of dots in the CRF was systematically varied, while the dots in the nCRF were held stationary. (Right panel) Modulatory effect of surround motion on motion stimuli in the CRF. The responses of the same neuron when the dots moved in its optimal direction, while the direction of the dots in the nCRF was systematically varied. Note that the neuron's responses vary systematically (i.e., the responses are “tuned”) with respect to both CRF motion, and motion in both the center and surround. Thus, neuron can convey information of the motion “context,” or motion of a given moving object relative to nearby stationary or moving objects. Adapted, with permission, from ().


Figure 19. Corticostriatal loops in human brain. (A) Four distinguishable but mutually overlapping loops are usually recognized (colored labeled arrows), based primarily on the types of tasks in which they play prominent roles. Cortical inputs arrive largely via the striatum and ultimately are directed back into the cortex via the thalamus. The (ultimate) cortical output of the basal ganglia reaches largely to the same cortical areas that give rise to the initial inputs to the basal ganglia. The visual loop is known to play a prominent role in the learning of visual object categories, but during object categorization tasks using learned categories, the executive loop also plays a prominent role. Corticostriatal loops in the nonhuman primate brain are largely similar (not shown). Adapted, with permission, from (); also see (). (B) A more detailed circuit map of the visual loop shows the flow of information within the loop. GPe: Globus pallidus, external portion. GPi: Globus pallidus, internal portion. SNr: Substantia nigra pars reticulata. SNc: Substantia nigra pars compacta. STN: Subthalamic nucleus. VTA: Ventral tegmental area. Adapted, with permission, from ().


Figure 20. An illustration of Granger causality. This figure shows cause and effect relationship between two hypothetical neural responses (top and bottom rows, respectively). The cause‐and‐effect relationship exists throughout the responses, but is most readily apparent by visual inspection of those portions of the responses where the response is prominently modulated (arrows). Granger causality uses the entire length of both responses to quantitatively measure the cause‐and‐effect relationship even when the relationship may be too subtle or complex to be visually evident. In the present case, the response shown in the top row is said to “Granger‐cause” the response in the bottom row. Note that the term “Granger causation” denotes an inferred cause‐and‐effect relationship, which may or not include direct causation. Thus, the concept of Granger causality is in some respects narrower, and in some other respects broader, than the concept of direct causation (). But in either case, the cause must necessarily precede the effect.


Figure 21. Resting state brain networks are very similar to brain networks active during tasks. (A) Smith and colleagues () extracted 20 mutually independent patterns of activation in the resting state networks (RSNs) from a database of 36 adult human subjects using the independent components analysis (ICA) (). Brain regions identified by each of these ICAs can be thought of as an independent network. Smith and colleagues then compared RSN ICAs with ICAs of task‐activated networks from nearly 30,000 subjects from the BrainMap (BM) database (brainmap.org). Comparisons for ten most informative ICAs are shown side by side in this panel. For each ICA, activation is shown in a color‐coded format for a coronal, sagittal, and horizontal section (top, middle, bottom row, respectively). (B) Smith and colleagues () then analyzed the extent to which the top ten of the RSN ICAs play a role in various types of behavioral tasks (or “behavioral task domains” defined by the BrainMap database). Higher color values denote a correspondingly larger role by the given network in a given task. Note that each given type of task recruits different RSNs to different extents. Conversely, each RSN is active during multiple, different task paradigms, with the degree of participation varying according to the task. Thus, the RSNs represent a repertoire that the brain recruits and employs to various degrees depending on the task at hand. For details and some important caveats, see (). Adapted, with permission, from ().


Figure 22. Networks in human brain at rest. Nodes denote the center of mass of the corresponding brain regions, and the edges (i.e., colored lines) represent intrinsic connections between a pair of regions identifiable in the resting brain. Blue nodes represent the “rich club” brain regions, which are well‐connected brain regions that are well‐connected with other well‐connected brain regions. Gray dots denote nonrich club regions. Red lines denote connections between rich club regions. Orange lines denote “feeder” connections that connect a rich club region with a nonrich club region. Yellow lines denote “feeder” connections that connect a nonrich club region with another nonrich club region. Adapted, with permission, from (). Note that this figure does not show subcortical or cerebellar networks, which decidedly play crucial roles in brain function.


Figure 23. Neuronal responses in monkey middle temporal area (MT) reflect perceptual decisions in awake, behaving monkeys. (A) Stimuli and task paradigm. The studies by Newsome and colleagues () () used dots moving either in the preferred direction of the cell under study, or in the opposite (“null”) direction, depending on the trial. The investigators “tuned” the strength of the motion information by changing the proportion of dots that moved in the same direction (i.e., percentage correlation of motion). (B) The “neurometric function” (solid dots and solid fitted curve) an individual neuron that closely paralleled the “psychometric function” of the animal's behavioral responses (open dots and dashed fitted curve). (C) The close parallels between the neurometric versus psychometric functions held across different animal subjects. The dashed line notwithstanding indicates the best fitting linear trend. (D) Demonstration of a causal relationship between the responses of individual neurons and the animal's percepts using microstimulation. The psychometric function of an animal with or without microstimulation (solid dots and fitted line and open and dashed fitted line, respectively). See text and () for details. Adapted, with permission, from ().


Figure 24. A schematic illustration of correlation, decorrelation, and sparsening of the responses at the population level during the initial rapid transient responses (panel A), or at later stages (panels B‐E). Each panel shows a highly idealized “population” consisting of four neurons (circles). Each quadrant of a given circle denotes the response of the neuron to a given stimulus, color‐coded according to the color scale at bottom left. See text for details. Adapted, with permission, from ().


Figure 25. Visual cliff demonstrates development of depth perception. (A) The visual cliff is a laboratory apparatus that helps test depth perception in human infants and animals. It consists of an actual cliff covered with a sturdy but transparent plexiglass (). The cliff is textured with a high‐contrast checkerboard pattern, so that the cliff is clearly visible through the plexiglass. An infant called by his mother from the opaque side of the apparatus readily crawls to her (). On the other hand, he is reluctant to venture over the perceived cliff (panel B). Even when the infants know by patting the glass that it is solid, they still tend to be reluctant to cross. Infants’ decision as to whether or not to cross the visual cliff are also influenced by whether the gestures of the parent are encouraging, neutral, or discouraging (). Such behaviors show sophisticated inferences based on a joint evaluation of various depth cues, risks, and rewards. Studies show that healthy human infants have such depth perception even before they are able to crawl (). The visual cliff effect has been reported in many mammalian species (). For a video of visual cliff effect, see https://www.youtube.com/watch?v=p6cqNhHrMJA.


Figure 26. fMRI in young human infants. (A) Renderings of what infants at various ages are likely to see when they view a teddy bear. (B) Visual responses in the neonate. Visually responsive regions are located in the anterior aspect of the calcarine sulcus in either hemisphere. Moreover, the visually evoked responses are lower compared to the periods of rest. (C) Visual responses in a different 5‐month old infant. Visual stimulation activates a much posterior aspect of the calcarine sulcus. Also, the visually evoked responses are higher than the responses during rest. Panels B and C are courtesy of Dr. Ernst Martin () and reproduced with permission.


Figure 27. Preferential responses in the human cerebellum during high‐level cognitive task. Panel A shows the differential PET responses in a heatmap format, where brighter colors represent greater response. Panel B schematically summarizes the regions (red squares) that showed the task‐dependent preferential response. Note that the responses are highly lateralized. Adapted, with permission, from ().


Figure 28. Hemineglect. This figure shows the results from a drawing test () from a single patient with left hemineglect, resulting from a localized lesion in the right temporal lobe. The patient was asked the draw the dial of a clock. In most clinical cases, lesions tend to be less circumscribed and more widespread than in the patient whose drawing is shown here. Hence, drawings by most hemineglect patients tend to be much more complex, and less clear‐cut, than the “text book” case shown in this figure (see () for reviews). Figure courtesy of Scholarpedia.


Figure 29. Our evolving understanding of multimodal anatomical connections with the visual system. (A) Traditional view of the cortical anatomy of the primate brain recognized very few areas with multimodal anatomical connections (colored areas). (B) A more modern scheme of the cortical anatomy of multisensory areas. Colored areas represent regions where anatomical and/or electrophysiological studies have demonstrated multisensory interactions. Dashed gray outlines represent opened sulci. See () for details, including the criteria used for determining multimodal connectivity at the anatomical level. Adapted, with permission, from ().


Figure 30. Visual‐haptic object processing activates lateral occipital complex (LOC) in the occipitotemporal pathway. Ahmedi and colleagues () compared BOLD responses to four conditions: visual objects, somatosensory (or haptic) objects, visual textures, and haptic textures. Statistical map of the contralateral hemisphere from a single subject are shown in panel A (3D folded view), panel B (inflated view of the same hemisphere), and panel C (flattened view of the same hemisphere). Bottom, BOLD responses to the four conditions are shown in the somatosensory cortex (bottom left) and LOC in the occipitotemporal junction (bottom right). Col S, collateral sulcus; Cal S, calcarine sulcus; CS, central sulcus; IPS, intraparietal sulcus; lateral S, lateral sulcus; STS, superior temporal sulcus. Adapted, with permission, from ().


Figure 31. Color‐graphemic synesthesia. (A) (left) A stimulus that can elicit synesthesia in color‐graphemic synesthetes. (A) (right) A rendition of what the synesthete likely to have perceived. Note that, since color‐graphemic synesthetes tend to perceive different numbers as different colors, the triangle made up of 2’s stands out, or “pops out,” perceptually for them. By contrast, nonsynesthetes perceive all the numbers to be of the same color, so that for them, the triangle is not readily distinguishable from the background. (B) Neural responses during color‐graphemic synesthesia as measured by fMRI. BOLD responses to graphemic stimuli were contrasted against the responses to nongraphemic stimuli in synesthetes (left) and control subjects and results are rendered on inflated, bottom‐up views of brains of representative subjects. Both control subjects and synesthetes showed common activation of the “grapheme region” (Gr). In addition to this common activation, graphemes activated the color selective areas of the retinotopic region V4 (hV4) in synesthetes but not in nonsynesthetes. See () for details. Adapted, with permission, from ().
References
 1.Aertsen A, Preißl H. Dynamics of activity and connectivity in physiological neuronal networks. In: HG S, editor. Nonlinear Dynamics and Neuronal Networks. New York: Wiley‐VCH, 1991, pp. 281‐302.
 2.Aghdam HH, Heravi EJ. Guide to Convolutional Neural Networks. New York: Springer, Berlin, Heidelberg, 2017.
 3.Ahissar M, Hochstein S. The reverse hierarchy theory of visual perceptual learning. Trends Cogn Sci 8: 457‐464, 2004.
 4.Ajina S, Bridge H. Blindsight and unconscious vision: What they teach us about the human visual system. Neuroscientist 23: 529‐541, 2016.
 5.Allman J, Miezin F, McGuinness E. Stimulus specific responses from beyond the classical receptive field: Neurophysiological mechanisms for local‐global comparisons in visual neurons. Annu Rev Neurosci 8: 407‐430, 1985.
 6.Allman JM, Kaas JH. A representation of the visual field in the caudal third of the middle tempral gyrus of the owl monkey (Aotus trivirgatus). Brain Res 31: 85‐105, 1971.
 7.Alvarado MC, Bachevalier J. Revisiting the maturation of medial temporal lobe memory functions in primates. Learn Mem 7: 244‐256, 2000.
 8.Amedi A, Malach R, Hendler T, Peled S, Zohary E. Visuo‐haptic object‐related activation in the ventral visual pathway. Nat Neurosci 4: 324‐330, 2001.
 9.Andersen RA, Cui H. Intention, action planning, and decision making in parietal‐frontal circuits. Neuron 63: 568‐583, 2009.
 10.Andersen RA, Snyder LH, Bradley DC, Xing J. Multimodal representation of space in the posterior parietal cortex and its use in planning movements. Annu Rev Neurosci 20: 303‐330, 1997.
 11.Angelucci A, Bressloff PC. Contribution of feedforward, lateral and feedback connections to the classical receptive field center and extra‐classical receptive field surround of primate V1 neurons. Prog Brain Res 154: 93‐120, 2006.
 12.Archambault PS, Ferrari‐Toniolo S, Caminiti R, Battaglia‐Mayer A. Visually‐guided correction of hand reaching movements: The neurophysiological bases in the cerebral cortex. Vision Res 110: 244‐256, 2015.
 13.Ashby FG. Statistical Analysis of fMRI Data. Cambridge, MA: MIT Press, 2011.
 14.Atkinson AP, Adolphs R. The neuropsychology of face perception: Beyond simple dissociations and functional selectivity. Philos Trans R Soc Lond B Biol Sci 366: 1726‐1738, 2011.
 15.Attneave F. Some informational aspects of visual perception. Psychol Rev 61: 183‐193, 1954.
 16.Bagloee SA, Tavana M, Asadi M, Oliver T. Autonomous vehicles: Challenges, opportunities, and future implications for transportation policies. Journal of Modern Transportation 24: 284‐303, 2016.
 17.Banerjee A, Dean HL, Pesaran B. A likelihood method for computing selection times in spiking and local field potential activity. J Neurophysiol 104: 3705‐3720, 2010.
 18.Banks MS, Gepshtein S, Landy MS. Why is spatial stereoresolution so low? J Neurosci 24: 2077‐2089, 2004.
 19.Bar M. Visual objects in context. Nat Rev Neurosci 5: 617‐629, 2004.
 20.Bar M, Aminoff E. Cortical analysis of visual context. Neuron 38: 347‐358, 2003.
 21.Bar M, Kassam KS, Ghuman AS, Boshyan J, Schmid AM, Dale AM, Hamalainen MS, Marinkovic K, Schacter DL, Rosen BR, Halgren E. Top‐down facilitation of visual recognition. Proc Natl Acad Sci U S A 103: 449‐454, 2006.
 22.Barlow H. The mechanical mind. Annu Rev Neurosci 13: 15‐24, 1990.
 23.Barlow HB. Pattern recognition and the responses of sensory neurons. Ann N Y Acad Sci 156: 872‐881, 1969.
 24.Barnett L, Seth AK. The MVGC multivariate Granger causality toolbox: A new approach to Granger‐causal inference. J Neurosci Methods 223: 50‐68, 2014.
 25.Baron‐Cohen S, Harrison JE. Synaesthesia: Classic and Contemporary Readings. Cambridge, MA: Blackwell, 1997.
 26.Bart E, Hegdé J. Invariant Recognition of Visual Objects. Lausanne, Switzerland: Frontiers Media SA, 2013.
 27.Barton JJ. Higher cortical visual function. Curr Opin Ophthalmol 9: 40‐45, 1998.
 28.Bassett DS, Sporns O. Network neuroscience. Nat Neurosci 20: 353‐364, 2017.
 29.Berzuini C, Dawid P, Bernardinelli L. Causality: Statistical Perspectives and Applications. Hoboken, NJ: Wiley, 2012.
 30.Biederman I. Perceiving real‐world scenes. Science 177: 77‐80, 1972.
 31.Biswal BB. Resting state fMRI: A personal history. Neuroimage 62: 938‐944, 2012.
 32.Blake DT. Network supervision of adult experience and learning dependent sensory cortical plasticity. Compr Physiol 7: 977‐1008, 2017.
 33.Blake R, Logothetis N. Visual competition. Nat Rev Neurosci 3: 13‐21, 2002.
 34.Bloom FE, Morrison JH, Young WG. Neuroinformatics: A new tool for studying the brain. J Affect Disord 92: 133‐138, 2006.
 35.Braddick O. Encyclopedia of Perception. Thousand Oaks, CA: SAGE Publications, Inc., 2010.
 36.Bremmer F, Krekelberg B. Seeing and acting at the same time: Challenges for brain (and) research. Neuron 38: 367‐370, 2003.
 37.Britten KH, Shadlen MN, Newsome WT, Movshon JA. The analysis of visual motion: A comparison of neuronal and psychophysical performance. J Neurosci 12: 4745‐4765, 1992.
 38.Brown LL, Schneider JS, Lidsky TI. Sensory and cognitive functions of the basal ganglia. Curr Opin Neurobiol 7: 157‐163, 1997.
 39.Brown RJ, Norcia AM. A method for investigating binocular rivalry in real‐time with the steady‐state VEP. Vision Res 37: 2401‐2408, 1997.
 40.Bruce C, Desimone R, Gross CG. Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J Neurophysiol 46: 369‐384, 1981.
 41.Bruno RM. Synchrony in sensation. Curr Opin Neurobiol 21: 701‐708, 2011.
 42.Buckner RL. The cerebellum and cognitive function: 25 years of insight from anatomy and neuroimaging. Neuron 80: 807‐815, 2013.
 43.Buckner RL, Krienen FM, Yeo BT. Opportunities and limitations of intrinsic functional connectivity MRI. Nat Neurosci 16: 832‐837, 2013.
 44.Bullier J. Integrated model of visual processing. Brain Res Brain Res Rev 36: 96‐107, 2001.
 45.Bullier J, Hupe JM, James AC, Girard P. The role of feedback connections in shaping the responses of visual cortical neurons. Prog Brain Res 134: 193‐204, 2001.
 46.Burgess N, O'Keefe J. Models of place and grid cell firing and theta rhythmicity. Curr Opin Neurobiol 21: 734‐744, 2011.
 47.Cabeza R, Ciaramelli E, Moscovitch M. Cognitive contributions of the ventral parietal cortex: An integrative theoretical account. Trends Cogn Sci 16: 338‐352, 2012.
 48.Callaway EM. Neural substrates within primary visual cortex for interactions between parallel visual pathways. Prog Brain Res 149: 59‐64, 2005.
 49.Campos JJ, Langer A, Krowitz A. Cardiac responses on the visual cliff in prelocomotor human infants. Science 170: 196‐197, 1970.
 50.Cao JW, Lin ZP. Extreme learning machines on high dimensional and large data applications: A survey. Math Probl Eng 2015: 1‐13, 2015.
 51.Cavanagh P. What's up in top‐down processing? In: Gorea A, editor. Representations of Vision: Trends and Tacit Assumptions in Vision Research. New York: Cambridge University Press, 1991, pp. 295‐304.
 52.Chacron MJ, Longtin A, Maler L. Efficient computation via sparse coding in electrosensory neural networks. Curr Opin Neurobiol 21: 752‐760, 2011.
 53.Chen Y, Elenee Argentinis JD, Weber G. IBM Watson: How cognitive computing can be applied to big data challenges in life sciences research. Clin Ther 38: 688‐701, 2016.
 54.Chentanez T, Keatisuwan W, Akaraphan A, Chaunchaiyakul R, Lechanavanich C, Hiranrat S, Chaiwatcharaporn C, Glinsukon T. Reaction time, impulse speed, overall synaptic delay and number of synapses in tactile reaction neuronal circuits of normal subjects and thinner sniffers. Physiol Behav 42: 423‐431, 1988.
 55.Christensen R. Bayesian Ideas and Data Analysis: An Introduction for Scientists and Statisticians. Boca Raton, FL: CRC Press, 2011.
 56.Churchland PS. A neurophilosophical slant on consciousness research. Progress in Brain Research 149: 285‐293, 2005.
 57.Churchland PS, Ramachandran VS, Sejnowski TJ. A critique of pure vision. In: Koch C, Davis J, editors. Large‐Scale Neuronal Theories of the Brain. Cambridge, MA: MIT Press, 1994, pp. 23‐60.
 58.Coates P, Coleman S. Phenomenal Qualities: Sense, Perception, and Consciousness. New York: Oxford University Press, 2015.
 59.Cohen MR, Maunsell JH. Attention improves performance primarily by reducing interneuronal correlations. Nat Neurosci 12: 1594‐1600, 2009.
 60.Cohen MR, Newsome WT. Estimates of the contribution of single neurons to perception depend on timescale and noise correlation. J Neurosci 29: 6635‐6648, 2009.
 61.Cowey A, Stoerig P. Blindsight in monkeys. Nature 373: 247‐249, 1995.
 62.Crapse TB, Basso MA. Insights into decision making using choice probability. J Neurophysiol 114: 3039‐3049, 2015.
 63.Crasto CJ. Neuroinformatics. Totowa, NJ: Humana, 2007.
 64.Crick F. Visual perception: Rivalry and consciousness. Nature 379: 485‐486, 1996.
 65.Crick F, Koch C. Are we aware of neural activity in primary visual cortex? Nature 375: 121‐123, 1995.
 66.Crick F, Koch C. Consciousness and neuroscience. Cereb Cortex 8: 97‐107, 1998.
 67.Crick F, Koch C. A framework for consciousness. Nat Neurosci 6: 119‐126, 2003.
 68.Cumming BG, Nienborg H. Feedforward and feedback sources of choice probability in neural population responses. Curr Opin Neurobiol 37: 126‐132, 2016.
 69.Cytowic RE. Synesthesia: A Union of the Senses. Cambridge, MA: MIT Press, 2002.
 70.Cytowic RE, Eagleman D. Wednesday is Indigo Blue: Discovering the Brain of Synesthesia. Cambridge, MA: MIT Press, 2009.
 71.Dan Y, Atick JJ, Reid RC. Efficient coding of natural scenes in the lateral geniculate nucleus: Experimental test of a computational theory. J Neurosci 16: 3351‐3362, 1996.
 72.Davidoff JB. Differences in Visual Perception: The Individual Eye. London: Crosby Lockwood Staples, 1975.
 73.Davies ER. Computer and Machine Vision: Theory, Algorithms, Practicalities. Amsterdam; Boston: Elsevier, 2012.
 74.DeAngelis GC, Cumming BG, Newsome WT. Cortical area MT and the perception of stereoscopic depth. Nature 394: 677‐680, 1998.
 75.DeAngelis GC, Newsome WT. Organization of disparity‐selective neurons in macaque area MT. J Neurosci 19: 1398‐1415, 1999.
 76.DeAngelis GC, Newsome WT. Perceptual “read‐out” of conjoined direction and disparity maps in extrastriate area MT. PLoS Biol 2: E77, 2004.
 77.Dennett DC. Consciousness Explained. Boston: Little, Brown and Co., 1991.
 78.Desimone R, Ungerleider LG. Neural mechanisms of visual processing in monkeys. In: Boller F, Grafman J, editor. Handbook of Neuropsvchology. Amsterdam: Elsevier, 1989.
 79.DeYoe EA, Van Essen DC. Concurrent processing streams in monkey visual cortex. Trends Neurosci 11: 219‐226, 1988.
 80.Dienes Z. Understanding Psychology As A Science: An Introduction to Scientific and Statistical Inference. New York: Palgrave Macmillan, 2008.
 81.Dobzhansky T. Nothing in biology makes sense except in the light of evolution. Am Biol Teach 35: 125‐129, 1973.
 82.Dolan RJ, Fink GR, Rolls E, Booth M, Holmes A, Frackowiak RS, Friston KJ. How the brain learns to see objects and faces in an impoverished context. Nature 389: 596‐599, 1997.
 83.Doya K. Bayesian Brain: Probabilistic Approaches to Neural Coding. Cambridge, MA: MIT Press, 2007.
 84.Driver J, Spence C. Cross‐modal links in spatial attention. Philos Trans R Soc Lond B Biol Sci 353: 1319‐1331, 1998.
 85.Einhauser W, Kruse W, Hoffmann KP, Konig P. Differences of monkey and human overt attention under natural conditions. Vision Res 46: 1194‐1209, 2006.
 86.Einhauser W, Mundhenk TN, Baldi P, Koch C, Itti L. A bottom‐up model of spatial attention predicts human error patterns in rapid scene recognition. J Vis 7: 6.1‐6.13, 2007.
 87.Eliasmith C, Mandik P. Qualia. In: Hochstein E, editors. Dictionary of Philosophy of Mind, Waterloo, Canada: Sites.google.com, 2004.
 88.Elston GN. Cortical heterogeneity: Implications for visual processing and polysensory integration. J Neurocytol 31: 317‐335, 2002.
 89.Emmans D, Laihinen A, Halsband U. Comparative Neuropsychology and Brain Imaging. Zürich: LIT Verlag, 2015.
 90.Engel AK, Moll CK, Fried I, Ojemann GA. Invasive recordings from the human brain: Clinical insights and beyond. Nat Rev Neurosci 6: 35‐47, 2005.
 91.Erwin J, Hof PR. Aging in Nonhuman Primates. New York: Karger, 2002.
 92.Fabre‐Thorpe M, Richard G, Thorpe SJ. Rapid categorization of natural images by rhesus monkeys. Neuroreport 9: 303‐308, 1998.
 93.Fahle M, Poggio T. Perceptual Learning. Cambridge, MA: MIT Press, 2002.
 94.Fantz RL. The origin of form perception. Sci Am 204: 66‐72, 1961.
 95.Farah MJ, Ratcliff G. The Neuropsychology of High‐Level Vision: Collected Tutorial Essays. Hillsdale, NJ: Lawrence Erlbaum Associates, 1994.
 96.Fawcett JM, Risko EF, Kingstone A. The Handbook of Attention. Cambridge, MA: MIT Press, 2015.
 97.Fei‐Fei L, Iyer A, Koch C, Perona P. What do we perceive in a glance of a real‐world scene? J Vis 7: 10, 2007.
 98.Felleman DJ, Van Essen DC. Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1: 1‐47, 1991.
 99.Frassle S, Lomakina EI, Razi A, Friston KJ, Buhmann JM, Stephan KE. Regression DCM for fMRI. Neuroimage 155: 406‐421, 2017.
 100.Frassle S, Sommer J, Jansen A, Naber M, Einhauser W. Binocular rivalry: Frontal activity relates to introspection and action but not to perception. J Neurosci 34: 1738‐1747, 2014.
 101.Freedman DJ. Familiarity breeds plasticity: Distinct effects of experience on putative excitatory and inhibitory neurons in inferior temporal cortex. Neuron 74: 8‐11, 2012.
 102.Freedman DJ, Assad JA. Experience‐dependent representation of visual categories in parietal cortex. Nature 443: 85‐88, 2006.
 103.Friston K, Moran R, Seth AK. Analysing connectivity with Granger causality and dynamic causal modelling. Curr Opin Neurobiol 23: 172‐178, 2013.
 104.Friston KJ. Functional and effective connectivity: A review. Brain Connect 1: 13‐36, 2011.
 105.Futuyma DJ. Evolution. Sunderland, MA: Sinauer Associates, 2009.
 106.Geisler WS. Contributions of ideal observer theory to vision research. Vision Res 51: 771‐781, 2011.
 107.Geisler WS, Diehl RL. Bayesian natural selection and the evolution of perceptual systems. Philos Trans R Soc Lond B Biol Sci 357: 419‐448, 2002.
 108.Geisler WS, Kersten D. Illusions, perception and Bayes. Nat Neurosci 5: 508‐510, 2002.
 109.Ghazanfar AA, Schroeder CE. Is neocortex essentially multisensory? Trends Cogn Sci 10: 278‐285, 2006.
 110.Ghose GM. Learning in mammalian sensory cortex. Curr Opin Neurobiol 14: 513‐518, 2004.
 111.Ghose GM, Yang T, Maunsell JH. Physiological correlates of perceptual learning in monkey V1 and V2. J Neurophysiol 87: 1867‐1888, 2002.
 112.Gibson EJ, Walk RD. The “visual cliff.” Sci Am 202: 64‐71, 1960.
 113.Gilbert CD, Sigman M, Crist RE. The neural basis of perceptual learning. Neuron 31: 681‐697, 2001.
 114.Gilmore GC, Spinks RA, Thomas CW. Age effects in coding tasks: Componential analysis and test of the sensory deficit hypothesis. Psychol Aging 21: 7‐18, 2006.
 115.Girard P, Hupe JM, Bullier J. Feedforward and feedback connections between areas V1 and V2 of the monkey have similar rapid conduction velocities. J Neurophysiol 85: 1328‐1331, 2001.
 116.Glasser MF, Smith SM, Marcus DS, Andersson JL, Auerbach EJ, Behrens TE, Coalson TS, Harms MP, Jenkinson M, Moeller S, Robinson EC, Sotiropoulos SN, Xu J, Yacoub E, Ugurbil K, Van Essen DC. The Human Connectome Project's neuroimaging approach. Nat Neurosci 19: 1175‐1187, 2016.
 117.Gliga T. Handbook of developmental social neuroscience. Neuropsychol Rehabil 20: 637‐638, 2010.
 118.Glimcher PW. Indeterminacy in brain and behavior. Annu Rev Psychol 56: 25‐56, 2005.
 119.Goldstein LH, McNeil JE. Clinical Neuropsychology: A Practical Guide to Assessment and Management for Clinicians. Chichester, West Sussex; Malden, MA: Wiley‐Blackwell, 2013.
 120.Goodale MA, Milner AD. Separate visual pathways for perception and action. Trends Neurosci 15: 20‐25, 1992.
 121.Goswami G, Powell BM, Vatsa M, Singh R, Noore A. FR‐CAPTCHA: CAPTCHA based on recognizing human faces. PLoS One 9: e91708, 2014.
 122.Grady CL. Age‐related changes in cortical blood flow activation during perception and memory. Ann N Y Acad Sci 777: 14‐21, 1996.
 123.Grady CL, Maisog JM, Horwitz B, Ungerleider LG, Mentis MJ, Salerno JA, Pietrini P, Wagner E, Haxby JV. Age‐related changes in cortical blood flow activation during visual processing of faces and location. J Neurosci 14: 1450‐1462, 1994.
 124.Graham AM, Pfeifer JH, Fisher PA, Lin W, Gao W, Fair DA. The potential of infant fMRI research and the study of early life stress as a promising exemplar. Dev Cogn Neurosci 12: 12‐39, 2015.
 125.Gregory MD, Agam Y, Selvadurai C, Nagy A, Vangel M, Tucker M, Robertson EM, Stickgold R, Manoach DS. Resting state connectivity immediately following learning correlates with subsequent sleep‐dependent enhancement of motor task performance. Neuroimage 102 (Pt 2): 666‐673, 2014.
 126.Grill‐Spector K, Kanwisher N. Visual recognition: As soon as you know it is there, you know what it is. Psychol Sci 16: 152‐160, 2005.
 127.Grill‐Spector K, Malach R. The human visual cortex. Annu Rev Neurosci 27: 649‐677, 2004.
 128.Gross CG. Processing the facial image: A brief history. Am Psychol 60: 755‐763, 2005.
 129.Gross CG, Bruce CJ, Desimone R, Fleming R, Gattass R. Cortical visual areas of the temporal lobe: Three areas in the macaque. In: Woolsey CN, editor. Cortical Sensory Organization. New York: Humana Press, 1981, pp. 187‐216.
 130.Gross CG, Rodman HR, Gochin PM, Colombo MW. Inferior temporal cortex as a pattern recognition device. In: Baum E., editor. Computational Learning and Cognition. Philadelphia: Society for Industrial and Applied Mathematics, 1993, pp. 44‐73.
 131.Grossberg S. Linking visual development and learning to information processing: Preattentive and attentive brain dynamics. In: DeWeerd P, Pinaud R, Tremere L, editors. Plasticity in the Visual System: From Genes to Circuits. New York: Springer/Kluwer Academic Press, 2005, pp. 323‐346.
 132.Guidotti R, Del Gratta C, Baldassarre A, Romani GL, Corbetta M. Visual learning induces changes in resting‐state fMRI multivariate pattern of information. J Neurosci 35: 9786‐9798, 2015.
 133.Guyonneau R, Vanrullen R, Thorpe SJ. Temporal codes and sparse representations: A key to understanding rapid processing in the visual system. J Physiol Paris 98: 487‐497, 2004.
 134.Hafting T, Fyhn M, Molden S, Moser MB, Moser EI. Microstructure of a spatial map in the entorhinal cortex. Nature 436: 801‐806, 2005.
 135.Halligan PW, Kischka U, Marshall JC. Handbook of Clinical Neuropsychology. Oxford; New York: Oxford University Press, 2010.
 136.Hansen PC, Kringelbach ML, Salmelin R. MEG: An Introduction to Methods. New York: Oxford University Press, 2010.
 137.Hardcastle VG. Consciousness and the neurobiology of perceptual binding. Semin Neurol 17: 163‐170, 1997.
 138.Harnad S. To cognize is to categorize: Cognition is categorization. In: Cohen H, Lefebvre C, editors. Handbook of Categorization in Cognitive Science. San Diego, CA: Elsevier, 2005, pp. 19‐43.
 139.Hasson U, Hendler T, Ben Bashat D, Malach R. Vase or face? A neural correlate of shape‐selective grouping processes in the human brain. J Cogn Neurosci 13: 744‐753, 2001.
 140.Hegdé J. Time course of visual perception: Coarse‐to‐fine processing and beyond. Prog Neurobiol 84: 405‐439, 2008.
 141.Hegdé J, Felleman DJ. Reappraising the functional implications of the primate visual anatomical hierarchy. Neuroscientist 13: 416‐421, 2007.
 142.Hegde J, Kersten D. A link between visual disambiguation and visual memory. J Neurosci 30: 15124‐15133, 2010.
 143.Hegdé J, Van Essen DC. A comparative study of shape representation in macaque visual areas V2 and V4. Cereb Cortex 17: 1100‐1116, 2007.
 144.Hegdé J, Van Essen DC. Temporal dynamics of shape analysis in macaque visual area V2. J Neurophysiol 92: 3030‐3042, 2004.
 145.Henderson JM, Hollingworth A. Eye movements and visual memory: Detecting changes to saccade targets in scenes. Percept Psychophys 65: 58‐71, 2003.
 146.Henderson JM, Hollingworth A. High‐level scene perception. Annu Rev Psychol 50: 243‐271, 1999.
 147.Henson RN. Repetition suppression to faces in the fusiform face area: A personal and dynamic journey. Cortex 80: 174‐184, 2016.
 148.Hochstein S, Ahissar M. View from the top: Hierarchies and reverse hierarchies in the visual system. Neuron 36: 791‐804, 2002.
 149.Hodge MJS, Radick G. The Cambridge Companion to Darwin. Cambridge; New York: Cambridge University Press, 2009.
 150.Hof PR, Mobbs CV. Functional Neurobiology of Aging. San Diego, CA: Academic Press, 2001.
 151.Hoffman KL, Logothetis NK. Cortical mechanisms of sensory learning and object recognition. Philos Trans R Soc Lond B Biol Sci 364: 321‐329, 2009.
 152.Hoshi E. Cortico‐basal ganglia networks subserving goal‐directed behavior mediated by conditional visuo‐goal association. Front Neural Circuits 7: 158, 2013.
 153.Howard IP, Rogers BJ. Perceiving in Depth. New York: Oxford University Press, 2012.
 154.Hubbard EM, Arman AC, Ramachandran VS, Boynton GM. Individual differences among grapheme‐color synesthetes: Brain‐behavior correlations. Neuron 45: 975‐985, 2005.
 155.Hubbard EM, Piazza M, Pinel P, Dehaene S. Interactions between number and space in parietal cortex. Nat Rev Neurosci 6: 435‐448, 2005.
 156.Hubbard EM, Ramachandran VS. Neurocognitive mechanisms of synesthesia. Neuron 48: 509‐520, 2005.
 157.Huurneman B, Boonstra FN, Cox RF, Cillessen AH, van Rens G. A systematic review on ‘Foveal Crowding’ in visually impaired children and perceptual learning as a method to reduce crowding. BMC Ophthalmol 12: 27, 2012.
 158.Hyder F. Dynamic Brain Imaging: Multi‐Modal Methods and In Vivo Applications. New York: Humana, 2009.
 159.Isik L, Meyers EM, Leibo JZ, Poggio T. The dynamics of invariant object recognition in the human visual system. J Neurophysiol 111: 91‐102, 2014.
 160.James W, Rouben Mamoulian Collection (Library of Congress). The Principles of Psychology. New York: H. Holt and Company, 1890.
 161.Jeannerod M. Neurophysiological and Neuropsychological Aspects of Spatial Neglect. Amsterdam; New York; North‐Holland: Elsevier Science Pub. Co., 1987.
 162.Jefferis GS, Livet J. Sparse and combinatorial neuron labelling. Curr Opin Neurobiol 22: 101‐110, 2012.
 163.Johnson NA. Darwinian Detectives: Revealing the Natural History of Genes and Genomes. Oxford; New York: Oxford University Press, 2007.
 164.Kaas JH. The evolution of brains from early mammals to humans. Wiley Interdiscip Rev Cogn Sci 4: 33‐45, 2013.
 165.Kaas JH. Evolution of columns, modules, and domains in the neocortex of primates. Proc Natl Acad Sci U S A 109 (Suppl 1): 10655‐10660, 2012.
 166.Kaas JH. The evolution of neocortex in primates. Prog Brain Res 195: 91‐102, 2012.
 167.Kaas JH. Why does the brain have so many visual areas? Journal of Cognitive Neuroscience 1: 121‐135, 1989.
 168.Kaas JH, Stepniewska I. Evolution of posterior parietal cortex and parietal‐frontal networks for specific actions in primates. J Comp Neurol 524: 595‐608, 2016.
 169.Kaldy Z, Sigala N. The neural mechanisms of object working memory: What is where in the infant brain? Neurosci Biobehav Rev 28: 113‐121, 2004.
 170.Karnath HO. Spatial attention systems in spatial neglect. Neuropsychologia 75: 61‐73, 2015.
 171.Karnath HO, Fruhmann Berger M, Kuker W, Rorden C. The anatomy of spatial neglect based on voxelwise statistical analysis: A study of 140 patients. Cereb Cortex 14: 1164‐1172, 2004.
 172.Karnath HO, Milner AD, Vallar G. The Cognitive and Neural Bases of Spatial Neglect. Oxford; New York: Oxford University Press, 2002.
 173.Karthik S, Paul A, Karthikeyan N. Deep Learning Innovations and their Convergence with Big Data. Hershey, PA: Information Science Reference, 2018.
 174.Kay KN, Naselaris T, Prenger RJ, Gallant JL. Identifying natural images from human brain activity. Nature 452: 352‐355, 2008.
 175.Kelly C, Biswal BB, Craddock RC, Castellanos FX, Milham MP. Characterizing variation in the functional connectome: Promise and pitfalls. Trends Cogn Sci 16: 181‐188, 2012.
 176.Kendall AL, Hantraye P, Palfi S. Striatal tissue transplantation in non‐human primates. Prog Brain Res 127: 381‐404, 2000.
 177.Kensinger EA. Cognition in ageing and age‐related disease. In: Handbook of the Neuroscience of Aging. New York: Academic Press, 2009.
 178.Kersten D, Mamassian P, Yuille A. Object perception as Bayesian inference. Annu Rev Psychol 55: 271‐304, 2004.
 179.Kettlewell HB. Insect survival and selection for pattern: Most camouflage and survival mechanisms, though highly perfected, can be adapted to changing environments. Science 148: 1290‐1296, 1965.
 180.Killian NJ, Jutras MJ, Buffalo EA. A map of visual space in the primate entorhinal cortex. Nature 491: 761‐764, 2012.
 181.Kim HF, Hikosaka O. Parallel basal ganglia circuits for voluntary and automatic behaviour to reach rewards. Brain 138: 1776‐1800, 2015.
 182.Kiorpes L. Visual development in primates: Neural mechanisms and critical periods. Dev Neurobiol 75: 1080‐1090, 2015.
 183.Kiorpes L, Price T, Hall‐Haro C, Movshon JA. Development of sensitivity to global form and motion in macaque monkeys (Macaca nemestrina). Vision Res 63: 34‐42, 2012.
 184.Knierim JJ, Van Essen DC. Visual cortex: Cartography, connectivity, and concurrent processing. Curr Opin Neurobiol 2: 150‐155, 1992.
 185.Knill DC, Pouget A. The Bayesian brain: The role of uncertainty in neural coding and computation. Trends Neurosci 27: 712‐719, 2004.
 186.Kording KP, Wolpert DM. Bayesian integration in sensorimotor learning. Nature 427: 244‐247, 2004.
 187.Kourtzi Z, Connor CE. Neural representations for object perception: Structure, category, and adaptive coding. Annu Rev Neurosci 34: 45‐67, 2011.
 188.Kravitz DJ, Saleem KS, Baker CI, Ungerleider LG, Mishkin M. The ventral visual pathway: An expanded neural framework for the processing of object quality. Trends Cogn Sci 17: 26‐49, 2013.
 189.Krekelberg B, Lappe M. Neuronal latencies and the position of moving objects. Trends Neurosci 24: 335‐339, 2001.
 190.Kriegel U. Current Controversies in Philosophy of Mind. New York: Routledge, 2013.
 191.Krizhevsky A, Sutskever I, Hinton G. ImageNet classification with deep convolutional neural networks. In: NIPS (Neural Information Processing Systems). San Francisco, CA: Morgan Kaufmann Publishers, 2012.
 192.Kruschke JK. Doing Bayesian Data Analysis: A Tutorial with R and BUGS. Burlington, MA: Academic Press, 2011.
 193.Kumar P, Tiwari A. Ubiquitous Machine Learning and Its Applications. Hershey, PA: IGI Global, 2017.
 194.Lacey S, Sathian K. Multisensory object representation: Insights from studies of vision and touch. Prog Brain Res 191: 165‐176, 2011.
 195.Lamme VA, Roelfsema PR. The distinct modes of vision offered by feedforward and recurrent processing. Trends Neurosci 23: 571‐579, 2000.
 196.Lamme VA, Super H, Spekreijse H. Feedforward, horizontal, and feedback processing in the visual cortex. Curr Opin Neurobiol 8: 529‐535, 1998.
 197.Laubach M. Who's on first? What's on second? The time course of learning in corticostriatal systems. Trends Neurosci 28: 509‐511, 2005.
 198.Laurent G. Olfactory network dynamics and the coding of multidimensional signals. Nat Rev Neurosci 3: 884‐895, 2002.
 199.LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 521: 436‐444, 2015.
 200.Legatt AD, Arezzo J, Vaughan HG, Jr. Averaged multiple unit activity as an estimate of phasic changes in local neuronal activity: Effects of volume‐conducted potentials. J Neurosci Methods 2: 203‐217, 1980.
 201.Leopold DA, Logothetis NK. Activity changes in early visual cortex reflect monkeys’ percepts during binocular rivalry. Nature 379: 549‐553, 1996.
 202.Levi DM. Crowding–‐An essential bottleneck for object recognition: A mini‐review. Vision Res 48: 635‐654, 2008.
 203.Lewis CM, Baldassarre A, Committeri G, Romani GL, Corbetta M. Learning sculpts the spontaneous activity of the resting human brain. Proc Natl Acad Sci U S A 106: 17558‐17563, 2009.
 204.Li K, Malhotra PA. Spatial neglect. Pract Neurol 15: 333‐339, 2015.
 205.Lindenfors P. Neocortex evolution in primates: The “social brain” is for females. Biol Lett 1: 407‐410, 2005.
 206.Logothetis NK, Leopold DA, Sheinberg DL. What is rivalling during binocular rivalry? Nature 380: 621‐624, 1996.
 207.Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A. Neurophysiological investigation of the basis of the fMRI signal. Nature 412: 150‐157, 2001.
 208.Luck SJ, Kappenman ES. Oxford Handbook of Event‐Related Potential Components. Oxford: Oxford University Press, 2012.
 209.Ma L, Narayana S, Robin DA, Fox PT, Xiong J. Changes occur in resting state network of motor system during 4 weeks of motor skill learning. Neuroimage 58: 226‐233, 2011.
 210.Malashichev YB, Deckel AW. Behavioral and Morphological Asymmetries in Vertebrates. Georgetown, TX: Landes Bioscience: Eureka.com, 2006.
 211.Marcus GF, Freeman JA. The Future of the Brain: Essays by the World's Leading Neuroscientists. Princeton: Princeton University Press, 2015.
 212.Mariën P, Abutalebi J. Neuropsychological Research: A Review. Hove England; New York: Psychology Press, 2008.
 213.Marmarelis PZ, Marmarelis VZ. The White‐Noise Method in System identification. In: Analysis of Physiological Systems: The White‐Noise Approach. Boston, MA: Springer US, 1978, pp. 131‐180.
 214.Marr D. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. San Francisco: W.H. Freeman, 1982.
 215.Martin E. Imaging of brain function during early human development. In: Rutherford MA, editor. MRI of the Neonatal Brain. London: Mary A. Rutherford, 2016.
 216.Maunsell JH, van Essen DC. The connections of the middle temporal visual area (MT) and their relationship to a cortical hierarchy in the macaque monkey. J Neurosci 3: 2563‐2586, 1983.
 217.McAdams CJ, Maunsell JH. Effects of attention on the reliability of individual neurons in monkey visual cortex. Neuron 23: 765‐773, 1999.
 218.McAdams CJ, Maunsell JH. Attention to both space and feature modulates neuronal responses in macaque area V4. J Neurophysiol 83: 1751‐1755, 2000.
 219.McIntosh RD, Schenk T. Two visual streams for perception and action: Current trends. Neuropsychologia 47: 1391‐1396, 2009.
 220.Merigan WH, Maunsell JH. Macaque vision after magnocellular lateral geniculate lesions. Vis Neurosci 5: 347‐352, 1990.
 221.Merigan WH, Maunsell JH. How parallel are the primate visual pathways? Annu Rev Neurosci 16: 369‐402, 1993.
 222.Middleton FA, Strick PL. Basal‐ganglia ‘projections’ to the prefrontal cortex of the primate. Cereb Cortex 12: 926‐935, 2002.
 223.Mill J. Rodent models: Utility for candidate gene studies in human attention‐deficit hyperactivity disorder (ADHD). J Neurosci Methods 166: 294‐305, 2007.
 224.Mills TC. Time Series Econometrics: A Concise Introduction. Houndmills, Basingstoke, Hampshire; New York: Palgrave Macmillan, 2015.
 225.Mishkin M, Ungerleider LG, Macko KA. Object vision and spatial vision: Two cortical pathways. Trends in Neurosciences 6: 414‐417, 1983.
 226.Moran JM, Jolly E, Mitchell JP. Social‐cognitive deficits in normal aging. J Neurosci 32: 5553‐5561, 2012.
 227.Mort DJ, Malhotra P, Mannan SK, Rorden C, Pambakian A, Kennard C, Husain M. The anatomy of visual neglect. Brain 126: 1986‐1997, 2003.
 228.Moser EI, Moser MB, Roudi Y. Network mechanisms of grid cells. Philos Trans R Soc Lond B Biol Sci 369: 20120511, 2014.
 229.Moser EI, Roudi Y, Witter MP, Kentros C, Bonhoeffer T, Moser MB. Grid cells and cortical representation. Nat Rev Neurosci 15: 466‐481, 2014.
 230.Moser MB, Rowland DC, Moser EI. Place cells, grid cells, and memory. Cold Spring Harb Perspect Biol 7: a021808, 2015.
 231.Mukamel R, Fried I. Human intracranial recordings and cognitive neuroscience. Annu Rev Psychol 63: 511‐537, 2012.
 232.Murphey DK, Yoshor D, Beauchamp MS. Perception matches selectivity in the human anterior color center. Curr Biol 18: 216‐220, 2008.
 233.Mutz F, Veronese LP, Oliveira‐Santos T, de Aguiar E, Cheein FAA, de Souza AF. Large‐scale mapping in complex field scenarios using an autonomous car. Expert Syst Appl 46: 439‐462, 2016.
 234.Naselaris T, Kay KN, Nishimoto S, Gallant JL. Encoding and decoding in fMRI. Neuroimage 56: 400‐410, 2011.
 235.Nelson CA, Collins ML. Handbook of Developmental Cognitive Neuroscience. Cambridge, MA: MIT Press, 2008.
 236.Newsome WT, Britten KH, Salzman CD, Movshon JA. Neural mechanisms of motion perception. In: Cold Spring Harbor Symposia in Quantitative Biology. Cold Spring Harbor, NY: Cold Spring Harbor Lab Press, 1990, p. 697‐705.
 237.Nielsen FA, Christensen MS, Madsen KH, Lund TE, Hansen LK. fMRI neuroinformatics. IEEE Eng Med Biol Mag 25: 112‐119, 2006.
 238.Nobre K, Kastner S. The Oxford Handbook of Attention. Oxford; New York: Oxford University Press, 2014.
 239.Nunez PL, Srinivasan R. Electric Fields of the Brain: The Neurophysics of EEG. Oxford; New York: Oxford University Press, 2006.
 240.Okuyama‐Uchimura F, Komai S. Mouse ability to perceive subjective contours. Perception 45: 315‐327, 2016.
 241.Olshausen BA, Field DJ. Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research 37: 3311‐3325, 1997.
 242.Op de Beeck HP, Dicarlo JJ, Goense JB, Grill‐Spector K, Papanastassiou A, Tanifuji M, Tsao DY. Fine‐scale spatial organization of face and object selectivity in the temporal lobe: Do functional magnetic resonance imaging, optical imaging, and electrophysiology agree? J Neurosci 28: 11796‐11801, 2008.
 243.Orger MB, Smear MC, Anstis SM, Baier H. Perception of Fourier and non‐Fourier motion by larval zebrafish. Nat Neurosci 3: 1128‐1133, 2000.
 244.Osborne LC, Palmer SE, Lisberger SG, Bialek W. The neural basis for combinatorial coding in a cortical population response. J Neurosci 28: 13522‐13531, 2008.
 245.Panzeri S, Brunel N, Logothetis NK, Kayser C. Sensory neural codes using multiplexed temporal scales. Trends Neurosci 33: 111‐120, 2010.
 246.Papanicolaou AC. The Oxford Handbook of Functional Brain Imaging in Neuropsychology and Cognitive Neurosciences. New York: Oxford University Press, 2017.
 247.Parodi S, Riccardi G, Castagnino N, Tortolina L, Maffei M, Zoppoli G, Nencioni A, Ballestrero A, Patrone F. Systems medicine in oncology: Signaling network modeling and new‐generation decision‐support systems. Methods Mol Biol 1386: 181‐219, 2016.
 248.Pascalis O, de Haan M, Nelson CA. Is face processing species‐specific during the first year of life? Science 296: 1321‐1323, 2002.
 249.Passingham R. How good is the macaque monkey model of the human brain? Curr Opin Neurobiol 19: 6‐11, 2009.
 250.Pasupathy A, Miller EK. Different time courses of learning‐related activity in the prefrontal cortex and striatum. Nature 433: 873‐876, 2005.
 251.Pelli DG. Crowding: A cortical constraint on object recognition. Curr Opin Neurobiol 18: 445‐451, 2008.
 252.Pelli DG, Tillman KA, Freeman J, Su M, Berger TD, Majaj NJ. Crowding and eccentricity determine reading rate. J Vis 7: 20 21‐36, 2007.
 253.Perry CJ, Baciadonna L, Chittka L. Unexpected rewards induce dopamine‐dependent positive emotion‐like state changes in bumblebees. Science 353: 1529‐1531, 2016.
 254.Pessoa L, McMenamin B. Dynamic networks in the emotional brain. Neuroscientist 23: 383‐396, 2016.
 255.Petersen SE, Fox PT, Posner MI, Mintun M, Raichle ME. Positron emission tomographic studies of the cortical anatomy of single‐word processing. Nature 331: 585‐589, 1988.
 256.Petersen SE, Fox PT, Posner MI, Mintun M, Raichle ME. Positron emission tomographic studies of the processing of single words. J Cogn Neurosci 1: 153‐170, 1989.
 257.Platek SM, Shackelford TK. Foundations in Evolutionary Cognitive Neuroscience. Cambridge; New York: Cambridge University Press, 2009.
 258.Poggio T, Torre V, Koch C. Computational vision and regularization theory. Nature 317: 314‐319, 1985.
 259.Polonsky A, Blake R, Braun J, Heeger DJ. Neuronal activity in human primary visual cortex correlates with perception during binocular rivalry. Nat Neurosci 3: 1153‐1159, 2000.
 260.Poort J, Raudies F, Wannig A, Lamme VA, Neumann H, Roelfsema PR. The role of attention in figure‐ground segregation in areas V1 and V4 of the visual cortex. Neuron 75: 143‐156, 2012.
 261.Pouget A, Snyder LH. Computational approaches to sensorimotor transformations. Nat Neurosci 3 (Suppl): 1192‐1198, 2000.
 262.Pourtois G, Rauss KS, Vuilleumier P, Schwartz S. Effects of perceptual learning on primary visual cortex activity in humans. Vision Res 48: 55‐62, 2008.
 263.Prior J, Van Herwegen J. Practical Research with Children. New York: Routledge, 2016.
 264.Prochazka A, Ellaway P. Sensory systems in the control of movement. Compr Physiol 2: 2615‐2627, 2012.
 265.Pylyshyn ZW. Visual indexes, preconceptual objects, and situated vision. Cognition 80: 127‐158, 2001.
 266.Raiguel S, Vogels R, Mysore SG, Orban GA. Learning to see the difference specifically alters the most informative V4 neurons. J Neurosci 26: 6589‐6602, 2006.
 267.Ribeiro AS, Lacerda LM, Ferreira HA. Multimodal Imaging Brain Connectivity Analysis (MIBCA) toolbox. PeerJ 3: e1078, 2015.
 268.Richards JE, Rader N. Affective, behavioral, and avoidance responses on the visual cliff: Effects of crawling onset age, crawling experience, and testing age. Psychophysiology 20: 633‐641, 1983.
 269.Riera JJ, Ogawa T, Goto T, Sumiyoshi A, Nonaka H, Evans A, Miyakawa H, Kawashima R. Pitfalls in the dipolar model for the neocortical EEG sources. J Neurophysiol 108: 956‐975, 2012.
 270.Rilling JK. Comparative primate neuroimaging: Insights into human brain evolution. Trends Cogn Sci 18: 46‐55, 2014.
 271.Robertson LC. Binding, spatial attention and perceptual awareness. Nat Rev Neurosci 4: 93‐102, 2003.
 272.Roe AW, Chelazzi L, Connor CE, Conway BR, Fujita I, Gallant JL, Lu H, Vanduffel W. Toward a unified theory of visual area V4. Neuron 74: 12‐29, 2012.
 273.Roelfsema PR. Cortical algorithms for perceptual grouping. Annu Rev Neurosci 29: 203‐227, 2006.
 274.Rogers J, Gibbs RA. Comparative primate genomics: Emerging patterns of genome content and dynamics. Nat Rev Genet 15: 347‐359, 2014.
 275.Rolls ET. Invariant visual object and face recognition: Neural and computational bases, and a model, VisNet. Front Comput Neurosci 6: 35, 2012.
 276.Rosch E. Natural categories. Cognitive Psychology 4: 328‐350, 1973.
 277.Rubin N. Figure and ground in the brain. Nat Neurosci 4: 857‐858, 2001.
 278.Salzman CD, Britten KH, Newsome WT. Cortical microstimulation influences perceptual judgements of motion direction. Nature 346: 174‐177, 1990.
 279.Sathian K, Buxbaum LJ, Cohen LG, Krakauer JW, Lang CE, Corbetta M, Fitzpatrick SM. Neurological principles and rehabilitation of action disorders: Common clinical deficits. Neurorehabil Neural Repair 25: 21S‐32S, 2011.
 280.Schmid MC, Maier A. To see or not to see‐‐‐Thalamo‐cortical networks during blindsight and perceptual suppression. Prog Neurobiol 126: 36‐48, 2015.
 281.Schmidt JT. Activity‐driven sharpening of the retinotectal projection: The search for retrograde synaptic signaling pathways. J Neurobiol 59: 114‐133, 2004.
 282.Schmolesky MT, Wang Y, Hanes DP, Thompson KG, Leutgeb S, Schall JD, Leventhal AG. Signal timing across the macaque visual system. J Neurophysiol 79: 3272‐3278, 1998.
 283.Schooler LJ, Shiffrin RM, Raaijmakers JG. A Bayesian model for implicit effects in perceptual identification. Psychol Rev 108: 257‐272, 2001.
 284.Schoups A, Vogels R, Qian N, Orban G. Practising orientation identification improves orientation coding in V1 neurons. Nature 412: 549‐553, 2001.
 285.Schwartz S, Maquet P, Frith C. Neural correlates of perceptual learning: A functional MRI study of visual texture discrimination. Proc Natl Acad Sci U S A 99: 17137‐17142, 2002.
 286.Seger CA. How do the basal ganglia contribute to categorization? Their roles in generalization, response selection, and learning via feedback. Neuroscience and Biobehavioral Reviews 32: 265‐278, 2008.
 287.Seger CA, Miller EK. Category learning in the brain. Annu Rev Neurosci 33: 203‐219, 2010.
 288.Seger CA, Peterson EJ. Categorization = decision making + generalization. Neurosci Biobehav Rev 37: 1187‐1200, 2013.
 289.Seitz AR, Dinse HR. A common framework for perceptual learning. Curr Opin Neurobiol 17: 148‐153, 2007.
 290.Sejnowski TJ, Churchland PS, Movshon JA. Putting big data to good use in neuroscience. Nature Neuroscience 17: 1440‐1441, 2014.
 291.Sekihara K, Nagarajan SS. Electromagnetic Brain Imaging. New York: Springer, Berlin, Heidelberg, 2015.
 292.Sereno AB, Maunsell JH. Shape selectivity in primate lateral intraparietal cortex. Nature 395: 500‐503, 1998.
 293.Sereno MI, Allman JM. Cortical visual areas in mammals. In: Leventhal AG, editor. The Neural Basis of Visual Function. London: Macmillan, 1991, pp. 160‐172.
 294.Sereno MI, Huang RS. Multisensory maps in parietal cortex. Curr Opin Neurobiol 24: 39‐46, 2014.
 295.Seung S. Connectome: How the Brain's Wiring Makes us Who We Are. Boston: Houghton Mifflin Harcourt, 2012.
 296.Seung S. Connectome: How the Brain's Wiring Makes us Who We Are. Boston: Mariner Books, Houghton Mifflin Harcourt, 2013.
 297.Shadlen MN, Movshon JA. Synchrony unbound: A critical evaluation of the temporal binding hypothesis. Neuron 24: 67‐77, 111‐125, 1999.
 298.Sharpee TO, Sugihara H, Kurgansky AV, Rebrik SP, Stryker MP, Miller KD. Adaptive filtering enhances information transmission in visual cortex. Nature 439: 936‐942, 2006.
 299.Shepherd GM. Corticostriatal connectivity and its role in disease. Nat Rev Neurosci 14: 278‐291, 2013.
 300.Sherman SM. Functioning of circuits connecting thalamus and cortex. Compr Physiol 7: 713‐739, 2017.
 301.Sherman SM, Guillery RW. Exploring the Thalamus and Its Role in Cortical Function. Cambridge, MA: MIT Press, 2006.
 302.Shettleworth SJ. Cognition, Evolution, and Behavior. Oxford; New York: Oxford University Press, 2010.
 303.Shipp S. The functional logic of corticostriatal connections. Brain Struct Funct 222: 669‐706, 2017.
 304.Shmuel A, Augath M, Oeltermann A, Logothetis NK. Negative functional MRI response correlates with decreases in neuronal activity in monkey visual area V1. Nat Neurosci 9: 569‐577, 2006.
 305.Shooner C, Hallum LE, Kumbhani RD, Ziemba CM, Garcia‐Marin V, Kelly JG, Majaj NJ, Movshon JA, Kiorpes L. Population representation of visual information in areas V1 and V2 of amblyopic macaques. Vision Res 114: 56‐67, 2015.
 306.Sigala N, Logothetis NK. Visual categorization shapes feature selectivity in the primate temporal cortex. Nature 415: 318‐320, 2002.
 307.Simoncelli EP, Olshausen BA. Natural image statistics and neural representation. Annu Rev Neurosci 24: 1193‐1216, 2001.
 308.Sincich LC, Horton JC. The circuitry of V1 and V2: Integration of color, form, and motion. Annu Rev Neurosci 28: 303‐326, 2005.
 309.Smith SM, Fox PT, Miller KL, Glahn DC, Fox PM, Mackay CE, Filippini N, Watkins KE, Toro R, Laird AR, Beckmann CF. Correspondence of the brain's functional architecture during activation and rest. Proc Natl Acad Sci U S A 106: 13040‐13045, 2009.
 310.Snyder LH. Coordinate transformations for eye and arm movements in the brain. Curr Opin Neurobiol 10: 747‐754, 2000.
 311.Spanne A, Jorntell H. Questioning the role of sparse coding in the brain. Trends Neurosci 38: 417‐427, 2015.
 312.Spiteri Y, Galea EM. Psychology of Neglect. Hauppauge, NY: Nova Science Publishers, 2012.
 313.Sporns O. The human connectome: A complex network. Ann N Y Acad Sci 1224: 109‐125, 2011.
 314.Sporns O. Discovering the Human Connectome. Cambridge, MA: MIT Press, 2012.
 315.Sporns O. Cerebral cartography and connectomics. Philos Trans R Soc Lond B Biol Sci 370: 1‐12, 2015.
 316.Sporns O, Honey CJ, Kotter R. Identification and classification of hubs in brain networks. PLoS One 2: e1049, 2007.
 317.Stein BE. The New Handbook of Multisensory Processing. Cambridge, MA: MIT Press, 2012.
 318.Stein BE, Jiang W, Wallace MT, Stanford TR. Nonvisual influences on visual‐information processing in the superior colliculus. Prog Brain Res 134: 143‐156, 2001.
 319.Stone JV. Independent Component Analysis: A Tutorial Introduction. Cambridge, MA: MIT Press, 2004.
 320.Stoodley CJ. The cerebellum and cognition: Evidence from functional imaging studies. Cerebellum 11: 352‐365, 2012.
 321.Stoodley CJ, Valera EM, Schmahmann JD. Functional topography of the cerebellum for motor and cognitive tasks: An fMRI study. Neuroimage 59: 1560‐1570, 2012.
 322.Striedter GF, Avise JC, Ayala FJ. In the Light of Evolution: Volume VI: Brain and Behavior. Washington (DC): National Academies Press, 2013.
 323.Sugase Y, Yamane S, Ueno S, Kawano K. Global and fine information coded by single neurons in the temporal visual cortex. Nature 400: 869‐873, 1999.
 324.Summerfield C, Egner T, Greene M, Koechlin E, Mangels J, Hirsch J. Predictive codes for forthcoming perception in the frontal cortex. Science 314: 1311‐1314, 2006.
 325.Teixeira S, Machado S, Velasques B, Sanfim A, Minc D, Peressutti C, Bittencourt J, Budde H, Cagy M, Anghinah R, Basile LF, Piedade R, Ribeiro P, Diniz C, Cartier C, Gongora M, Silva F, Manaia F, Silva JG. Integrative parietal cortex processes: Neurological and psychiatric aspects. J Neurol Sci 338: 12‐22, 2014.
 326.Thilagam PS. Advanced Computing, Networking and Security: International Conference, ADCONS 2011, Surathkal, India, December 16‐18, 2011, Revised selected papers. Berlin; New York: Springer, 2012.
 327.Thorpe S, Fize D, Marlot C. Speed of processing in the human visual system. Nature 381: 520‐522, 1996.
 328.Thorpe SJ, Fabre‐Thorpe M. Neuroscience. Seeking categories in the brain. Science 291: 260‐263, 2001.
 329.Tolias AS, Keliris GA, Smirnakis SM, Logothetis NK. Neurons in macaque area V4 acquire directional tuning after adaptation to motion stimuli. Nat Neurosci 8: 591‐593, 2005.
 330.Tommasi L, Peterson MA, Nadel L. Cognitive Biology: Evolutionary and Developmental Perspectives on Mind, Brain, and Behavior. Cambridge, MA: MIT Press, 2009.
 331.Tong F, Pratte MS. Decoding patterns of human brain activity. Annu Rev Psychol 63: 483‐509, 2012.
 332.Treue S. Visual attention: The where, what, how and why of saliency. Curr Opin Neurobiol 13: 428‐432, 2003.
 333.Trommershauser J, Kording K, Landy MS. Sensory Cue Integration. Oxford; New York: Oxford University Press, 2011.
 334.Troscianko T, Benton CP, Lovell PG, Tolhurst DJ, Pizlo Z. Camouflage and visual perception. Philos Trans R Soc Lond B Biol Sci 364: 449‐461, 2009.
 335.Tsodyks M, Gilbert C. Neural networks and perceptual learning. Nature 431: 775‐781, 2004.
 336.Tyler CW, Likova LT. Crowding: A neuroanalytic approach. J Vis 7: 16.11‐16.19, 2007.
 337.Ullman S. High‐level Vision: Object Recognition and Visual Cognition. Cambridge, MA: MIT Press, 2000.
 338.Vallar G, Perani D. The anatomy of unilateral neglect after right‐hemisphere stroke lesions. A clinical/CT‐scan correlation study in man. Neuropsychologia 24: 609‐622, 1986.
 339.van den Heuvel MP, Sporns O. An anatomical substrate for integration among functional networks in human cortex. J Neurosci 33: 14489‐14500, 2013.
 340.van der Helm PA. Cognitive architecture of perceptual organization: From neurons to gnosons. Cogn Process 13: 13‐40, 2012.
 341.Van Essen DC. Functional organization of primate visual cortex. In: Jones EG, Peters A, editors. Cerebral Cortex. New York: Plenum Press, 1975, pp. 259‐329.
 342.Van Essen DC. Visual areas of the mammalian cerebral cortex. Annu Rev Neurosci 2: 227‐263, 1979.
 343.Van Essen DC. Organization of visual areas in macaque and human cerebral cortex. In: Chalupa L, Werner JS, editors. The Visual Neurosciences. Cambridge, MA: MIT Press, 2004, pp. 507‐521.
 344.Van Essen DC, Anderson CH, Felleman DJ. Information processing in the primate visual system: An integrated systems perspective. Science 255: 419‐423, 1992.
 345.Van Essen DC, Felleman DJ. On hierarchies: Response to Hilgetag et al. Science 271: 777, 1996.
 346.Van Essen DC, Ugurbil K, Auerbach E, Barch D, Behrens TE, Bucholz R, Chang A, Chen L, Corbetta M, Curtiss SW, Della Penna S, Feinberg D, Glasser MF, Harel N, Heath AC, Larson‐Prior L, Marcus D, Michalareas G, Moeller S, Oostenveld R, Petersen SE, Prior F, Schlaggar BL, Smith SM, Snyder AZ, Xu J, Yacoub E, Consortium WU‐MH. The Human Connectome Project: A data acquisition perspective. Neuroimage 62: 2222‐2231, 2012.
 347.Van Vleet TM, DeGutis JM. The nonspatial side of spatial neglect and related approaches to treatment. Prog Brain Res 207: 327‐349, 2013.
 348.VanRullen R, Thorpe SJ. Surfing a spike wave down the ventral stream. Vision Res 42: 2593‐2615, 2002.
 349.Verrelli BC, Tishkoff SA. Signatures of selection and gene conversion associated with human color vision variation. Am J Hum Genet 75: 363‐375, 2004.
 350.Vesia M, Crawford JD. Specialization of reach function in human posterior parietal cortex. Exp Brain Res 221: 1‐18, 2012.
 351.Voogd J, Schraa‐Tam CK, van der Geest JN, De Zeeuw CI. Visuomotor cerebellum in human and nonhuman primates. Cerebellum 11: 392‐410, 2012.
 352.Wandell BA, Smirnakis SM. Plasticity and stability of visual field maps in adult primary visual cortex. Nat Rev Neurosci 10: 873‐884, 2009.
 353.Wang C, Cleland BG, Burke W. Synaptic delay in the lateral geniculate nucleus of the cat. Brain Res 343: 236‐245, 1985.
 354.Wang D, Buckner RL, Liu H. Cerebellar asymmetry and its relation to cerebral asymmetry estimated by intrinsic functional connectivity. J Neurophysiol 109: 46‐57, 2013.
 355.Watanabe T, Sasaki Y. Perceptual learning: Toward a comprehensive theory. Annu Rev Psychol 66: 197‐221, 2015.
 356.Webster MJ, Ungerleider LG, Bachevalier J. Development and plasticity of the neural circuitry underlying visual recognition memory. Can J Physiol Pharmacol 73: 1364‐1371, 1995.
 357.Weiss Y, Simoncelli EP, Edelson EH. Motion illusions as optimal percepts. Nat Neurosci 5: 598‐604, 2002.
 358.Werner JS, Chalupa LM. The New Visual Neurosciences. Cambridge, MA: The MIT Press, 2014.
 359.Westwood DA, Goodale MA. Converging evidence for diverging pathways: Neuropsychology and psychophysics tell the same story. Vision Res 51: 804‐811, 2011.
 360.Whitney D, Levi DM. Visual crowding: A fundamental limit on conscious perception and object recognition. Trends Cogn Sci 15: 160‐168, 2011.
 361.Wu MC, David SV, Gallant JL. Complete functional characterization of sensory neurons by system identification. Annu Rev Neurosci 29: 477‐505, 2006.
 362.Xiao Y, Rao R, Cecchi G, Kaplan E. Improved mapping of information distribution across the cortical surface with the support vector machine. Neural Netw 21: 341‐348, 2008.
 363.Yang T, Maunsell JH. The effect of perceptual learning on neuronal responses in monkey visual area V4. J Neurosci 24: 1617‐1626, 2004.
 364.Yoshida M, Itti L, Berg DJ, Ikeda T, Kato R, Takaura K, White BJ, Munoz DP, Isa T. Residual attention guidance in blindsight monkeys watching complex natural scenes. Curr Biol 22: 1429‐1434, 2012.
 365.Yuille A, Kersten D. Vision as Bayesian inference: Analysis by synthesis? Trends Cogn Sci 10: 301‐308, 2006.
 366.Zanzotto FM. Brain Informatics: International Conference, BI 2012, Macau, China, December 4‐7, 2012: Proceedings. Berlin; New York: Springer, 2012.

 

Further Reading

Li Z. Understanding vision: Theory, models, and data. New York, NY: Oxford University Press, 2014.

Mitra P and Bokil H. Observed brain dynamics. Oxford; New York: Oxford University Press, 2008.

Nelson CA and Collins ML. Handbook of developmental cognitive neuroscience. Cambridge, Mass.: MIT Press, 2008.

Sporns O. Discovering the human connectome. Cambridge, Mass.: MIT Press, 2012.

Palmer SE. Vision science: photons to phenomenology. Cambridge, Mass.: MIT Press, 1999.

Rolls ET and Deco G. The noisy brain: stochastic dynamics as a principle of brain function. Oxford; New York: Oxford University Press, 2010.

Seung S. Connectome: how the brain's wiring makes us who we are. Boston: Houghton Mifflin

Harcourt, 2012.

Sporns O. Discovering the human connectome. Cambridge, Mass.: MIT Press, 2012.

Werner JS and Chalupa LM. The new visual neurosciences. Cambridge, MA: MIT Press, 2013.

Wolfe JM, Kluender KR, Levi DM, Bartoshuk LM, Herz RS, Klatzky RL, Lederman SJ, Merfeld DM. Sensation & Perception. Sunderland, MA: Sinauer Associates, 2014.

 

 

Teaching Material

J. Hegdé. Neural Mechanisms of High-Level Vision. Compr Physiol 8: 2018, 903-953.

Didactic Synopsis

Major Teaching Points:

  • Visual perception is essentially an inferential process, in that the visual system infers the likely interpretation of a given image by evaluating the various underlying statistical (i.e., probabilistic) factors. This is not to say that the inferences are necessarily optimal or that they are made consciously.
  • For last several decades, neurophysiological research has been guided by the implicit assumption that the goal of visual processing is to help construct a veridical internal representation of the external visual world. This, along with earlier methodological difficulties in monitoring the activity of large number of neurons in multiple areas, led to a couple of decades of neurophysiological research focused on delineating functional specialization of individual visual areas, which has to do with figuring out “which visual area does what” in representing the external visual world.
  • More recent research, however, suggests that a “goal-oriented connectomic” view is a better framework for understanding visual processing as a sensory process wherein large number of different brain areas act, not as individual areas but as parts of as a larger network, to implement the animal's behavioral goal at hand.
  • The networks that implement a given behavior vary dynamically and probabilistically depending on various factors, including the sensory information, behavioral goal, possible rewards or punishments, etc.
  • Vision is not a single process, but a collection of processes.
  • Even though vision is the dominant sensory modality for humans and other primates, it does not function in isolation, but interacts actively with other senses and the brain systems that implement behavior.

Didactic Legends

The figures—in a freely downloadable PowerPoint format—can be found on the Images tab along with the formal legends published in the article. The following legends to the same figures are written to be useful for teaching.

Figure 1 The abstract painting Interchanged (1955) by Willem de Kooning. At a reported purchase price of US $300 million, it is one of the most expensive paintings in the world. See text for additional details.

Teaching points: Low-level features of an image refer to basic properties of image elements, such as the color or brightness of the various image regions, the orientation of the various lines and surfaces, etc. High-level properties of the image refer to the larger, more holistic import of the image. High-level understanding of an image is more than the sum of understanding all of its low-level features. Abstract paintings such as this help illustrate the distinction between low- and high-level visions especially well, because the painting is deliberately constructed so as to maximally dissociate the two sets of visual processes. However, in most real-world images, the processing of low-level features leads to a high-level level understanding so readily that it can be hard to grasp the distinction between the two.

Figure 2 The importance of recurrent processing in visual perception. When viewed for the first time, this two-tone “Mooney” image appears to be an unrecognizable pattern of black and white blobs. That is, it is hard to interpret this image based on sensory information (i.e., bottom-up or feed-forward processing) alone. However, after viewing a full grayscale or color counterpart (for which see Figure 3A—which, being smaller in size, has no pixel-to-pixel correspondence with the above image), the Mooney image becomes easy to interpret. Note that viewing the disambiguating image rapidly, drastically, and enduringly alters our perception of the Mooney image, although the Mooney image itself remains physically unchanged. Feed-forward theories of vision cannot help explain such phenomena. Recurrent (or “reentrant”) neural signals bring to bear such top-down influences as prior knowledge and the behavioral context to help constrain the interpretation of the visual image. Learning to interpret Mooney images can be understood as an extreme case of knowledge-mediated disambiguation that is part and parcel of normal visual perception (51, 82, 125, 142).

Teaching points: Vision is not a deterministic process; the same visual image can produce dramatically different perceptual outcomes depending on the perceptual state of the viewer. Two-tone, “Mooney” images such as this help demonstrate this by showing that perception can change dramatically in the total absence of any changes in the sensory information. Such perceptual changes would not occur if the sensory input solely determined the perceptual outcome.

Figure 3 Some complexities of natural images and the information processing required to making sense of them. Panels A through D show real-world scenes that help illustrate some of the complexities of such scenes. For instance, there are multiple objects of the same type in each picture. They all vary greatly in image size, illumination, shadows, occlusion, viewpoint, etc. But the visual system recognizes them as objects of the same kind. In other words, the brain must be able to discard a variety of image features as irrelevant to recognizing the images. Note that, in case of each scene in this figure, our understanding of the scene evolves over time. Note also that the objects, their spatial relationships, and even the semantic similarities and differences among them are such that the percept that each scene elicits is more than just the sum of the parts. In addition, each picture contains implicit cues to motion and/or depth that static, 2D pictures such as these do not do justice to. In fact, natural scenes differ from each other and from the relatively simple stimuli used in many a vision study, such as a sinusoidal grating on a neutral gray background, in myriad ways. Studies have shown that the visual system is adapted to contend with the statistics of natural scenes (307), so that neural responses to “artificial” stimuli may provide a substantially different and potentially misleading picture of how the brain works. On the other hand, note that our visual system performs very well in recognizing scenes with “unnatural” statistics, such as the street scene in panel C and the indoor scene in panel D, even though it evidently did not encounter the “unnatural” statistics of human-made objects until quite recently on the evolutionary time scale. This is because the brain is quite good at adapting to a variety of nonoptimal inputs.

Teaching points: Given the complexity of the visual world, visual perception cannot be brought about by feedforward processing alone. Feedback and lateral processing bring to bear information about the sensory and behavioral context, prior knowledge, etc. on visual processing. These images also help demonstrate the fact that vision is not a unitary process but a multifaceted one, because the precise nature of the visual processing depends critically, among other things, on the nature of the scene and of the task at hand.

Figure 4 Visual system can often recognize objects with great precision with very little information. A veridical description of the object, even if it were possible, is not necessary. (A) In the picture on the left, very little of the dog is visible. But we have little trouble recognizing the dog. Many probably can even readily name the breed of the dog. (B) Few would have trouble recognizing the person from the picture. (C) The tell-tale hat. For most readers, this picture simply shows a hat hung on some plumbing. But this famous picture, on the cover of the very first issue of Physics Today in May 1948, showed the famous “pork pie” hat of J. Robert Oppenheimer, the father of the atomic bomb. Therefore, the picture was highly topical and was readily understood at the time. But for most of us living today, this picture requires either some historical knowledge or explanation. Thus, our perception of a visual image is influenced by a great many nonvisual factors. Neurophysiological understanding of such influences on cognition represents a major challenge.

Teaching points: Visual system can often recognize objects with great precision with very little information. This is because, as noted earlier, the visual system is adept at coming up with the best possible inference based on all available information. Thus, when there is very little sensory information available in a given visual scene, the visual system “fills in the gaps” using its prior knowledge of the visual world.

Figure 5 There is more to vision than feed-forward processing. This picture of an unknown little girl was taken by South African photojournalist Kevin Carter in the famine-stricken South Sudan in March 1993. The girl had reportedly collapsed from weakness on her way to a United Nations food center. (The Vulture and the Little Girl by Kevin Carter. Pulitzer Prize for Feature Photography, 1994. Reproduced with permission.) Note that feed-forward processing by itself would completely miss the import of the picture.

Teaching points: Feed-forward processing does not fully account for how vision works. For instance, feedforward processing by itself would completely miss the point of this picture.

Figure 6 (A) A self-driving car. (B) Two instances of the CAPTCHA (or Completely Automated Public Turing test to tell Computers and Humans Apart) internet device designed to prevent automated logins (121, 326). In the example on the left, the website asks the user to perform a straightforward object categorization task, namely distinguish pictures of people wearing glasses from pictures of people without glasses. This tends to be quite successful in preventing machines from logging on to the website. However, this success is not so much because it would be all that difficult nowadays to train a suitably designed computer program to distinguish the two categories of human faces (see, e.g., (2, 191)), but essentially because, at present, such programs tend to be hyperspecialized, and tend not to generalize beyond their training sets to other visual objects or tasks (121, 199). That is, such an “intelligent” program, once trained to tell aforementioned types of faces apart, can be readily stumped at present by rather slight changes in the underlying categories (panel B, right), or the task (not shown) (2, 191). However, recent research suggests that the problem of overspecialization of intelligent machines is likely to straightforwardly surmountable, so that CAPTCHA strategies such as the one illustrated in this panel are unlikely to be effective for long. That is, machine vision is getting ever better at mimicking human high-level vision (199).

Teaching points: Visual abilities and limitations of modern computers help illustrate some of the high-level visual faculties needed to operate in the real world, and the underlying computational issues. Most of the current limitations of machine vision systems lie in the realm of high-level vision and semantic understanding of images, that is, understanding the larger meaning of images, such as that in Figure 5, beyond merely recognizing the various objects in the image. But machine vision is rapidly catching up with high-level capabilities of humans.

Figure 7 An outline of Marr's theory of visual processing. See text for details.

Teaching points: Marr's influential model of visual perception, now largely disproven, posits that visual processing proceeds in a feed-forward, modular, and hierarchical manner and results in a veridical internal representation of the visual world.

Figure 8 Visual anatomical hierarchy in the macaque monkey described by Felleman and Van Essen, 1991 (98). Each colored rectangle represents a distinct cortical visual area. The open rectangles at top denote higher cortical areas that are not considered primarily visual area. The two gray rectangles at bottom denote the retinal ganglion cells (RGC) and the lateral geniculate nucleus (LGN). Lines connecting the cortical areas denote interconnections, usually reciprocal, between a given pairs of areas. The various brain regions are represented in a tiered, or hierarchical, fashion based on objective anatomical criteria, most important of which are the laminar patterns of feed-forward and feedback connections (also see (98, 216, 342)). For additional details and abbreviations, see Felleman and Van Essen, 1991 (98). An alternative formulation of the hierarchy, originally formulated by Mishkin, Gross, and their colleagues (130) (also see (78, 129)) is largely similar, but it parcels the cortex into many fewer areas and recognizes fewer interconnections. Also, some the visual area names are different in this scheme. For instance, areas AIT and CIT (anterior and central inferotemporal areas, respectively) in the Felleman and Van Essen scheme are equivalent to area TE (temporal area) in the Gross et al. scheme, and area PIT (posterior inferotemporal area) in the Van Essen scheme is equivalent to area TEO (temporooccipital area) in the Gross et al. scheme. Both schemes are used in this review, based on the scheme used by the study in question. Reproduced, with permission, from (98).

Teaching points: From an anatomical viewpoint, visual areas of the brain can be arranged as a richly interconnected hierarchy. This does not necessarily mean that the visual information processing for which it provides the neural substrate is also hierarchical. This is ultimately because information does not flow in a single, feed-forward direction within the anatomical hierarchy. That is, the information does not always travel “up” the anatomical hierarchy, but instead flows in multiple different directions.

Figure 9 Our evolving understanding of the functional organization of the primate visual system. Panels A and B depict our view of the two main visual processing streams in the macaque brain in the early 1980s. (A) The dorsal and ventral visual pathways as originally formulated by Mishkin and colleagues in 1993 is denoted by solid arrows (OB → OA → PG pathway being the anatomically dorsal pathway, and OB → OA → TEO → TE being the ventral pathway) (225). The continuation of these pathways to FDΔ and FDv, respectively (dashed arrows) represented the collective outcome of many additional studies. (B) Anatomical locations of the various regions, some renamed according to a more modern naming convention (188). (C) Key changes in the receptive field properties of neurons along the ventral pathway. Areas are color-coded as in panel B. Panel D summarizes our understanding of the same pathways some 30 years later, as summarized by Kravitz and colleagues in 2013 (188). Note that it is clear that the two pathways have turned out to be far more interconnected than previously envisioned. What is lost in terms of pedagogical simplicity is more than made up for by the nuance and granularity of this network picture, presaged decades ago by Kerrigan and Maunsell (220). Self-evidently, far more remains to be learned, including how these networks interact with other networks in the brain and subserve behavior. Adapted, with permission, from (188). Human brain (not shown) is also purported to have two evolutionarily homologous processing pathways, although there is even less empirical information to support this notion.

Teaching points: Our understanding of the functional organization of the visual system has changed drastically in the last few decades. The evolution of our understanding of the dorsal versus ventral visual pathways is a case in point. Initial studies seemed to support the conventional hierarchical, modular view of visual information processing. But subsequent research has revealed extensive anatomical interconnections and functional interactions among the various visual areas within and across the two pathways, and the computational consequences of these interactions have become clearer. The inescapable conclusion is that the visual system works as an intricate and dynamic network, and not as a bottom-to-top conveyer belt.

Figure 10 What's to infer? Isn't it all there in the image? The answer is no. Retinal image is simply a 2D pattern of image intensities. Image intensities of a particular image are represented in panel A as a color-coded surface plot, where the height and color of a point denotes the image intensity at that point. Note that when the image is represented in this fashion, it makes no sense to us. But when the same image is represented as corresponding variations in image intensities (panel B), we readily recognize it as an image of a brook in the woods. However, the information in the two representations is exactly the same. The difference is that, in panel B, the image is in an input “format” that our eyes can process. Beyond that, the inferential processes to “make sense” of the pattern of intensities are exactly the same.

Teaching points: Visual processing is a fundamentally inferential process. This is because the retinal image is inherently ambiguous. The retinal image by itself “makes no sense” unless it is appropriately interpreted by subsequent visual processing.

Figure 11 Visual illusions help illustrate the inferential nature of visual perception. They also demonstrate that the inference need not be a deliberate, volitional, or conscious process. (A) Ames room. Two people stand in opposite corners of the room. One appears to be much taller than the other, even though they are roughly the same height. Instead, it is the room that is distorted to produce this illusion. Picture courtesy of Ian Stannard, Flickr. Reproduced with permission. (B) Hollow-mask illusion. This picture shows the front of the mask (right) and the hollow back of the mask (left). Nonetheless, both look like normal, convex faces. For a video demonstration of the hollow mask illusion, see https://www.youtube.com/watch?v=sKa0eaKsdA0. Note that, in this video, the mask appears to flip its direction of rotation at the same time the other side of the mask begins appearing. Theoretical studies show that visual illusions are often perfectly rational inferences given the evidence (see, e.g., (108, 357).

Teaching points: Visual illusions provide some of the most self-evident demonstrations that visual perception is fundamentally inferential in nature wherein the brain takes in all the available, pertinent information and comes up with an interpretation that fits the information at hand. Note that knowing that the percept is illusory does not make it go away, nor can we easily make the illusory percept disappear by trying to achieve the nonillusory percept, that is, that the persons in panel A are of roughly the same height, or that the face on the right on panel B is really concave. This is because of the brain's proclivity to go with what it deems to be a rational inference given the totality of the evidence. It is often impossible for the mind to veto this inference.

Figure 12 The spatiotemporal domain of the methods available for the study of the functional organization of nervous system in 2014, compared to the methods available in 1988 (inset). Each colored region represents a range of spatial and temporal resolutions for a given method. Open regions represent measurement techniques; filled regions, perturbation techniques; EEG, electroencephalography; MEG, magnetoencephalography; PET, positron emission tomography; VSD, voltage-sensitive dye; TMS, transcranial magnetic stimulation; 2-DG, 2-deoxyglucose. Redrawn, with permission, from (290).

Teaching points: The repertoire of available neurophysiological techniques has improved greatly within the last few decades. The repertoire of techniques for analyzing neurophysiological data also has undergone phenomenal improvements, as has our ability to cross-compare data obtained from different techniques in humans as well as animals.

Figure 13 Neuronal responses in monkey visual area V1 during binocular rivalry. See text for details. Adapted, with permission, from (33, 201, 206).

Teaching points: Visual perception can change even when the visual stimulus itself remains unchanged. Binocular rivalry is one such case. Such perceptual changes that in the total absence of any change in the visual stimulus have often been used as experimental tools for dissociating the confounding effects of sensory stimulation on the visual percept.

Figure 14 Figure-ground segregation and perceptual organization. (A) How many circles can you see in this image? In this image, referred to as the Coffer Illusion, you should be able to see 16 circles. Figure courtesy of Dr. Anthony Norcia, Stanford University. Reproduced with permission. (B) Camouflage is an extreme case of figure-ground segregation, where the object of interest is hard to recognize even when “in plain view.” This image shows two variants of the pepper moth Biston betularia, one black and the other with light peppered coloring. The black variant is effectively camouflaged against colored tree bark whereas the light variant is easy to recognize (i.e., it “pops out”). The opposite is true when the same two variants are seen against a background of light bark and lichens. The black variant emerged for the first time in the industrial midlands of Britain in the 19th century, where tree barks were turning black with industrial soot. Soon the black variants became the more common variant, because the predators of the moths had much greater success breaking the camouflage of the lighter variants because it was harder for the lighter variants to find light backgrounds to camouflage themselves against. This was an instance of the prey “gaming” the predators’ high-level visual faculties to enhance its own survival (179, 334).

Teaching points: Both figure-ground segregation (i.e., the perceptual process whereby an object of behavioral interest is perceptually distinguished from the background) and perceptual grouping (i.e., the complementary of process of mentally grouping related parts of the scene), are important aspects of high-level understanding of visual scenes.

Figure 15 Selectivity for faces in the macaque superior temporal polysensory area (STP). (A) Anatomical location of area STP (yellow highlight). (B) Responses of a single STP neuron to various visual objects, including variations in face stimuli. Note that, among the stimuli tested, the neuron responds best to a face with all the key facial features (left column, second stimulus from top). Cutting this stimulus into 16 pieces and showing the pieces in a shuffled order essentially eliminated the response (right column, third stimulus form top). The icon at bottom right denotes the size and visual field location of the receptive field. C, contralateral visual field (i.e., contralateral to the recording location). I, ipsilateral. Such neurons with large receptive fields that span both the visual hemifields are common in the visually responsive areas of the central and anterior temporal cortex. Adapted, with permission, from (40).

Teaching points: Neurons in the temporal area show remarkable selectivity for faces. This figure shows one of the most striking and widely known demonstrations of such narrow selectivity.

Figure 16 Coarse-to-fine tuning of shape categories in the macaque IT. The stimulus set consisted of 38 stimuli, a subset of which is shown in the inset. The global shape categories (inset, vertical axis) consisted of human faces, monkey faces, and geometric shapes. The fine categories (inset, horizontal axis) consisted of the various facial identities and expressions. The plots show the cumulative information transmission rate of a sample of IT neurons about both the global and fine categories (red and blue lines, respectively). The thick horizontal line along the x axis denotes the stimulus duration. Adapted, with permission, from (140, 323).

Teaching points: Responses of many face-selective neurons in the monkey temporal cortex change in a coarse-to-fine fashion in time. This figure shows one of the most clear-cut demonstrations of this effect. However, it is important to remember that responses of most neurons in most areas tend to change over time in a much more complex fashion that rarely lends itself to pithy slogans.

Figure 17 Grid cells help represent the external visual space. Grid cells were originally reported in rats, but have since been found in many species, including monkeys. (A) Responses of a single grid cell in the entorhinal cortex of the rat. Left, black lines denote the trajectory of a rat freely moving in a box. The cell fired spikes (red dots) when the rat was at specific locations within the box. These locations were organized in a grid-like fashion that spanned the box. Right, the firing rates of the cell represented as a heatmap, where “warmer” colors denote higher firing rates. Adapted, with permission, from (134). (B) Responses of a single grid cell in the entorhinal cortex of the macaque monkey. Unlike the rat referred to in panel A, the monkey was stationary. It sat in a primate chair with its head held steady, but freely moved its eyes as it looked at real-world pictures (not shown). Left, red dots denote the locations in a picture (not shown) that the monkey fixated, or gazed steadily at for a brief period. Center, firing rate of the grid cell shown in heatmap format. The scale bars at bottom each denote 6° of visual angle. Adapted, with permission, from (180).

Teaching points: Grid cells in the entorhinal cortex help represent the external visual space. Such neuronal representations of the external space appear to be quite common across various mammalian species.

Figure 18 Coding of visual context. The classical receptive field (CRF) of a given neuron is the portion of the visual field in which the neuron is most responsive to visual stimuli. The surrounding region in which visual stimulation modulates the responses to CRF stimulation is referred to as the nonclassical receptive field (nCRF) or nonclassical surround. This figure shows one of the earliest demonstrations of the modulatory effect of nCRF on CRF, in this case the response of a single neuron in macaque MT. Individual neurons in MT often response best when the stimulus moves in a particular direction, often referred to as its preferred or optimal direction. The neuron shown responded best when the dots in the CRF moved horizontally from left to right. CRF is denoted by the dashed rectangle in the icons at top. (Left panel) Modulatory effect of stationary surround on motion stimuli in the CRF. The direction of the movement of dots in the CRF was systematically varied, while the dots in the nCRF were held stationary. (Right panel) Modulatory effect of surround motion on motion stimuli in the CRF. The responses of the same neuron when the dots moved in its optimal direction, while the direction of the dots in the nCRF was systematically varied. Note that the neuron's responses vary systematically (i.e., the responses are “tuned”) with respect to both CRF motion, and motion in both the center and surround. Thus, neuron can convey information of the motion “context,” or motion of a given moving object relative to nearby stationary or moving objects. Adapted, with permission, from (5).

Teaching points: The responses of visual neurons are often modulated by the visual context, that is, by visual stimuli outside the neuron's classical receptive field. This figure shows one of the earliest demonstrations of the effects of stimulating the nonclassical surround on the stimuli within the classical receptive field.

Figure 19 Corticostriatal loops in human brain. (A) Four distinguishable but mutually overlapping loops are usually recognized (colored labeled arrows), based primarily on the types of tasks in which they play prominent roles. Cortical inputs arrive largely via the striatum and ultimately are directed back into the cortex via the thalamus. The (ultimate) cortical output of the basal ganglia reaches largely to the same cortical areas that give rise to the initial inputs to the basal ganglia. The visual loop is known to play a prominent role in the learning of visual object categories, but during object categorization tasks using learned categories, the executive loop also plays a prominent role. Corticostriatal loops in the nonhuman primate brain are largely similar (not shown). Adapted, with permission, from (287); also see (222, 250). (B) A more detailed circuit map of the visual loop shows the flow of information within the loop. GPe: Globus pallidus, external portion. GPi: Globus pallidus, internal portion. SNr: Substantia nigra pars reticulata. SNc: Substantia nigra pars compacta. STN: Subthalamic nucleus. VTA: Ventral tegmental area. Adapted, with permission, from (286).

Teaching points: Corticostrial loops refer to a circular pathway in which information flows from the cortex to the various parts of the striatum and back to the cortex. Corticostriatal loops play an important role in perception and perceptual learning. This figure illustrates the four main corticostriatal loops in the human brain and the general connectivity pattern of each loop.

Figure 20 An illustration of Granger causality. This figure shows cause and effect relationship between two hypothetical neural responses (top and bottom rows, respectively). The cause-and-effect relationship exists throughout the responses, but is most readily apparent by visual inspection of those portions of the responses where the response is prominently modulated (arrows). Granger causality uses the entire length of both responses to quantitatively measure the cause-and-effect relationship even when the relationship may be too subtle or complex to be visually evident. In the present case, the response shown in the top row is said to “Granger-cause” the response in the bottom row. Note that the term “Granger causation” denotes an inferred cause-and-effect relationship, which may or not include direct causation. Thus, the concept of Granger causality is in some respects narrower, and in some other respects broader, than the concept of direct causation (13, 29, 224). But in either case, the cause must necessarily precede the effect.

Teaching points: Granger causality is a powerful analytical method which can quantify causal relationship between two or more time-domain data (technically known as time series data). It was originally developed by Clive Granger (1934-2009; Novel Prize in Economics, 2003) to mine for, and measure, cause-and-effect relationships in time series data in economics, such as two or more stock prices. Granger causality is one of the many instances in which neurophysiology has hugely benefited by eagerly embracing experimental and analytical approaches from other fields of research.

Figure 21 Resting state brain networks are very similar to brain networks active during tasks. (A) Smith and colleagues (309) extracted 20 mutually independent patterns of activation in the resting state networks (RSNs) from a database of 36 adult human subjects using the independent components analysis (ICA) (319). Brain regions identified by each of these ICAs can be thought of as an independent network. Smith and colleagues then compared RSN ICAs with ICAs of task-activated networks from nearly 30,000 subjects from the BrainMap (BM) database (brainmap.org). Comparisons for ten most informative ICAs are shown side by side in this panel. For each ICA, activation is shown in a color-coded format for a coronal, sagittal, and horizontal section (top, middle, bottom row, respectively). (B) Smith and colleagues (309) then analyzed the extent to which the top ten of the RSN ICAs play a role in various types of behavioral tasks (or “behavioral task domains” defined by the BrainMap database). Higher color values denote a correspondingly larger role by the given network in a given task. Note that each given type of task recruits different RSNs to different extents. Conversely, each RSN is active during multiple, different task paradigms, with the degree of participation varying according to the task. Thus, the RSNs represent a repertoire that the brain recruits and employs to various degrees depending on the task at hand. For details and some important caveats, see (309). Adapted, with permission, from (309).

Teaching points: The brain networks active during tasks partially overlap resting state networks in nuanced, meaningful ways. This figure illustrates the brain responses during a given active task characterized by the given weighted combination of the resting state networks that are recruited in a task-dependent fashion.

Figure 22 Networks in human brain at rest. Nodes denote the center of mass of the corresponding brain regions, and the edges (i.e., colored lines) represents intrinsic connections between a pair of regions identifiable in the resting brain. Blue nodes represent the “rich club” brain regions, which are well-connected brain regions that are well-connected with other well-connected brain regions. Gray dots denote nonrich club regions. Red lines denote connections between rich club regions. Orange lines denote “feeder” connections that connect a rich club region with a nonrich club region. Yellow lines denote “feeder” connections that connect a nonrich club region with another nonrich club region. Adapted, with permission, from (339). Note that this figure does not show subcortical or cerebellar networks, which decidedly play crucial roles in brain function.

Teaching points: Brain areas are organized as densely interconnected networks. The pattern of interconnections among brain regions tend to follow a “rich club” pattern, where well-connected regions tend to be interconnected among each other, rather than with poorly interconnected regions.

Figure 23 Neuronal responses in monkey middle temporal area (MT) reflect perceptual decisions in awake, behaving monkeys. (A) Stimuli and task paradigm. The studies by Newsome and colleagues (37, 236, 278) used dots moving either in the preferred direction of the cell under study, or in the opposite (“null”) direction, depending on the trial. The investigators “tuned” the strength of the motion information by changing the proportion of dots that moved in the same direction (i.e., percentage correlation of motion). (B) The “neurometric function” (solid dots and solid fitted curve) an individual neuron that closely paralleled the “psychometric function” of the animal's behavioral responses (open dots and dashed fitted curve). (C) The close parallels between the neurometric versus psychometric functions held across different animal subjects. The dashed line notwithstanding indicates the best fitting linear trend. (D) Demonstration of a causal relationship between the responses of individual neurons and the animal's percepts using microstimulation. The psychometric function of an animal with or without microstimulation (solid dots and fitted line and open and dashed fitted line, respectively). See text and (37, 236) for details. Adapted, with permission, from (37, 236) and (37, 236).

Teaching points: Individual neurons in many visual areas reliably represent the perceptual outcome. This figure shows one of the earliest and most influential demonstrations of this principle. Perhaps even more importantly, the various techniques showcased in this study also paved for the study of less striking (and far more common) scenarios where individual neurons only carry limited information about the perceptual outcome.

Figure 24 A schematic illustration of correlation, decorrelation, and sparsening of the responses at the population level during the initial rapid transient responses (panel A), or at later stages (panels B-E). Each panel shows a highly idealized “population” consisting of four neurons (circles). Each quadrant of a given circle denotes the response of the neuron to a given stimulus, color-coded according to the color scale at bottom left. See text for details. Adapted, with permission, from (140).

Teaching points: Sparsening refers to a widespread neurophysiological phenomenon where a given visual stimulus elicits an initial transient response from many neurons in a given population, but most of the responses return to background levels shortly thereafter. Decorrelation is a related temporal dynamic phenomenon where different neurons in a population respond differently to a given stimulus. Sparsening and decorrelation are somewhat complementary phenomena, because increasing sparseness actually increases correlation (or reduces decorrelation). This figure schematically illustrates some of the many different ways in which response sparsening can occur within a neuronal population.

Figure 25 Visual cliff demonstrates development of depth perception. (A) The visual cliff is a laboratory apparatus that helps test depth perception in human infants and animals. It consists of an actual cliff covered with a sturdy but transparent plexiglass (112). The cliff is textured with a high-contrast checkerboard pattern, so that the cliff is clearly visible through the plexiglass. An infant called by his mother from the opaque side of the apparatus readily crawls to her (112). On the other hand, he is reluctant to venture over the perceived cliff (panel B). Even when the infants know by patting the glass that it is solid, they still tend to be reluctant to cross. Infants’ decision as to whether or not to cross the visual cliff are also influenced by whether the gestures of the parent are encouraging, neutral, or discouraging (268). Such behaviors show sophisticated inferences based on a joint evaluation of various depth cues, risks, and rewards. Studies show that healthy human infants have such depth perception even before they are able to crawl (49). The visual cliff effect has been reported in many mammalian species (94). For a video of visual cliff effect, see https://www.youtube.com/watch?v=p6cqNhHrMJA.

Teaching points: Visual cliff is a classic demonstration that depth perception is a mental faculty that develops quite early during postnatal development in mammalian species. Visual cliff studies helped establish the field of systematic characterization of brain function during early development.

Figure 26 fMRI in young human infants. (A) Renderings of what infants at various ages are likely to see when they view a teddy bear. (B) Visual responses in the neonate. Visually responsive regions are located in the anterior aspect of the calcarine sulcus in either hemisphere. Moreover, the visually evoked responses are lower compared to the periods of rest. (C) Visual responses in a different 5-month old infant. Visual stimulation activates a much posterior aspect of the calcarine sulcus. Also, the visually evoked responses are higher than the responses during rest. Panels B and C are courtesy of Dr. Ernst Martin (215) and reproduced with permission.

Teaching points: Visual responses in very young human infants are drastically different from those in human adults. Thus, brain development is not just an elaboration of a functional organizational blueprint that exists at birth.

Figure 27 Preferential responses in the human cerebellum during high-level cognitive task. Panel A shows the differential PET responses in a heatmap format, where brighter colors represent greater response. Panel B schematically summarizes the regions (red squares) that showed the task-dependent preferential response. Note that the responses are highly lateralized. Adapted, with permission, from (256).

Teaching points: It was long thought that the cerebellum is primarily involved in fine-tuning of movements. Recent anatomical and neuroimaging studies have revealed that the cerebellum is involved in computations that involve high-level cognitive tasks, including high-level vision. Of particular import to the students of neurophysiology is that what is currently known about this aspect of cerebellar function is likely to be just the tip of the iceberg.

Figure 28 Hemineglect. This figure shows the results from a drawing test (119, 135) from a single patient with left hemineglect, resulting from a localized lesion in the right temporal lobe. The patient was asked the draw the dial of a clock. In most clinical cases, lesions tend to be less circumscribed and more widespread than in the patient whose drawing is shown here. Hence, drawings by most hemineglect patients tend to be much more complex, and less clear-cut, than the “text book” case shown in this figure (see (119, 135) for reviews).

Teaching points: Lesions in various regions of the brain can produce specific deficits in high-level vision. A standard method of clinically characterizing the deficit in a given patient is to show the patient a (often standardized) set of drawings and ask him or her to reproduce the drawings. Close examination of the patient's drawings can reveal much about the specific visual deficits in the given patient. Such drawing tests are typically only a part of a battery of tests that are administered, since the actual cases tend to be highly complex, and often multimodal, so as to warrant multiple different types of tests.

Figure 29 Our evolving understanding of multimodal anatomical connections with the visual system. (A) Traditional view of the cortical anatomy of the primate brain recognized very few areas with multimodal anatomical connections (colored areas). (B) A more modern scheme of the cortical anatomy of multisensory areas. Colored areas represent regions where anatomical and/or electrophysiological studies have demonstrated multisensory interactions. Dashed gray outlines represent opened sulci. See (109) for details, including the criteria used for determining multimodal connectivity at the anatomical level. Adapted, with permission, from (109).

Teaching points: Brain is multimodal; vision does not act alone. This figure helps illustrate our rapidly developing appreciation of just how multimodal the brain is. Even regions that we once thought were exclusively visual areas are now known to have strong interactions with other sensory modalities.

Figure 30 Visual-haptic object processing activates lateral occipital complex (LOC) in the occipitotemporal pathway. Ahmedi and colleagues (8) compared BOLD responses to four conditions: visual objects, somatosensory (or haptic) objects, visual textures, and haptic textures. Statistical map of the contralateral hemisphere from a single subject are shown in panel A (3D folded view), panel B (inflated view of the same hemisphere), and panel C (flattened view of the same hemisphere). Bottom, BOLD responses to the four conditions are shown in the somatosensory cortex (bottom left) and LOC in the occipitotemporal junction (bottom right). Col S, collateral sulcus; Cal S, calcarine sulcus; CS, central sulcus; IPS, intraparietal sulcus; lateral S, lateral sulcus; STS, superior temporal sulcus. Adapted, with permission, from (8).

Teaching points: Visual-haptic object processing activates lateral occipital complex, a brain region that plays a prominent role in visual processing. This result helps highlight the fact that cross-modal processing is a general principle of brain function, and not a special case scenario.

Figure 31 Color-graphemic synesthesia. (A) (left) A stimulus that can elicit synesthesia in color-graphemic synesthetes. (A) (right) A rendition of what the synesthete likely to have perceived. Note that, since color-graphemic synesthetes tend to perceive different numbers as different colors, the triangle made up of 2’s stands out, or “pops out,” perceptually for them. By contrast, nonsynesthetes perceive all the numbers to be of the same color, so that for them, the triangle is not readily distinguishable from the background. (B) Neural responses during color-graphemic synesthesia as measured by fMRI. BOLD responses to graphemic stimuli were contrasted against the responses to nongraphemic stimuli in synesthetes (left) and control subjects and results are rendered on inflated, bottom-up views of brains of representative subjects. Both control subjects and synesthetes showed common activation of the “grapheme region” (Gr). In addition to this common activation, graphemes activated the color selective areas of the retinotopic region V4 (hV4) in synesthetes but not in nonsynesthetes. See (156) for details. Adapted, with permission, from (156).

Teaching points: Color-graphemic synesthetes perceive numbers as colors. Visual stimuli such as those shown in panel A not only help identify such synesthetes but also help study the brain mechanisms of this type of synesthesia. fMRI results show that color graphemic stimuli activate some brain regions in both synesthetes and controls subjects. But synesthetes show additional activation of the color-selective portion of the retinotopic region V4 (hV4), whereas nonsynesthetes do not show hV4 activation.

 


Related Articles:

Network Supervision of Adult Experience and Learning Dependent Sensory Cortical Plasticity
Attention: Behavior and Neural Mechanisms
The Beginnings of Visual Perception: The Retinal Image and its Initial Encoding. Appendix: Fourier Transforms and Shift‐Invariant Linear Operators
Electrophysiology of Cognition
Control of Eye Movements
Hemispheric Specialization
Neural Mechanisms of Color Vision
Perception
Preattentive Human Vision: Link Between Neurophysiology and Psychophysics
Sensory Function in Animals
Sensory Research in Historical Perspective: Some Philosophical Foundations of Perception
The Superior Colliculus and Visual Function
Teaching Material

Contact Editor

Submit a note to the editor about this article by filling in the form below.

* Required Field

How to Cite

Jay Hegdé. Neural Mechanisms of High‐Level Vision. Compr Physiol 2018, 8: 903-953. doi: 10.1002/cphy.c160035