What part of the brain controls face recognition

Face Individualization Through fMR Adaptation

Both the FFA and OFA are involved in such individualization of faces, as demonstrated by fMRI adaptation, or repetition suppression, in which the neural response to a given stimulus is reduced when that stimulus is repeated (Grill-Spector & Malach, 2001). Release from adaptation to face identity (i.e., face A preceded by face B vs. face A preceded by face A) has been found in both areas but not, or only weakly, in the pSTS (Davies-Thompson, Gouws, & Andrews, 2009; Figure 6). fMR-adaptation studies have shown that both the FFA and OFA are sensitive to changes of a subset of the facial parts when they are inserted in whole faces or to relative distances between these parts in whole faces (Rhodes, Michie, Hughes, & Byatt, 2009). There is also a higher response for different than identical faces for upright but not for inverted faces in both the FFA and OFA (Yovel & Kanwisher, 2005). Both areas also show increased activation to the illusion of change of identity on the top halves of faces when these top halves are aligned with different bottom halves (Schiltz & Rossion, 2006; Figure 6). These observations have been taken as evidence for holistic/configural representation of individual faces in the right FFA in particular. It has also been suggested that the FFA codes individual faces relative to a norm in a face space (Loffler, Yourganov, Wilkinson, & Wilson, 2005), but these observations are controversial (Davidenko, Remus, & Grill-Spector, 2012; Kahn & Aguirre, 2012).

The Lesions of Acquired Prosopagnosia

Functional imaging studies have revealed that face processing involves a core face network that consists of the fusiform face area, the occipital face area, and the posterior portion of the superior temporal sulcus (Fig. 10.3) (Kanwisher et al., 1997; Haxby et al., 2000; Gobbini and Haxby, 2007). Beyond this is an extended network that includes the anterior inferior temporal area (Kriegeskorte et al., 2007; Rajimehr et al., 2009) and other regions such as the inferior frontal gyrus and precuneus (Haxby et al., 2000). While faces activate these areas in both hemispheres (Haxby et al., 2000; Nestor et al., 2013), the effect is typically larger on the right (Kanwisher et al., 1997), which agrees with the observation that prosopagnosia is more common with right than with left unilateral hemispheric lesions.

Structural imaging studies show that a variety of lesions can cause acquired prosopagnosia (Fig. 10.4). The classic lesion is bilateral damage to the fusiform gyri in medial occipitotemporal cortex (Meadows, 1974; Damasio et al., 1982a). Later, it became apparent that prosopagnosia could also occur with a unilateral right occipitotemporal lesions (de Renzi, 1986; Landis et al., 1986; Michel et al., 1986; Sergent and Villemure, 1989; Schweinberger et al., 1995; Takahashi et al., 1995), likely involving the fusiform gyrus (Barton et al., 2002). Rare cases of prosopagnosia with a unilateral left occipitotemporal lesion have been reported. Some of these have been in left-handed individuals (Tzavares et al., 1973; Mattson et al., 2000; Barton, 2008b), suggesting that this anomaly may merely reflect a reverse hemispheric dominance. In others, there was evidence to suggest subtle damage of the right hemisphere as well (Eimer and McCarthy, 1999; Eimer, 2000; Wright et al., 2006).

The ability to link structural damage in individual prosopagnosic patients to effects on functional networks was advanced by the creation of protocols that reliably demonstrate the network of regions active during face perception in single subjects (Fox et al., 2008a). Use of these protocols have shown that prosopagnosia from occipitotemporal damage is associated with right or bilateral loss of the fusiform and/or occipital face areas (Rossion et al., 2003; Schiltz et al., 2006b; Davies-Thompson et al., 2014). Conversely, the core face network can be preserved in those with anterior temporal lesions (Davies-Thompson et al., 2014), which are associated with the amnestic variant (Evans et al., 1995a; Barton et al., 2003; Barton and Cherkasova, 2003). The inferior frontal gyrus is also damaged in some patients (Barton, 2008c), but the contribution of frontal lesions to prosopagnosia is not known. Last, there are patients with extensive damage that involves both anterior temporal and medial occipitotemporal damage on one or both sides. These findings suggest that acquired prosopagnosia is not a single entity, but a family of disorders that have different functional mechanisms deriving from lesions to different structural components of the network, but which converge on the same end-result, namely, impaired face recognition (Davies-Thompson et al., 2014).

Common causes of acquired prosopagnosia are posterior cerebral artery infarctions, head trauma, and viral encephalitis (Damasio et al., 1982a; Takahashi et al., 1995; Barton et al., 2002), all of which can cause unilateral or bilateral damage. Patients with unilateral lesions can also have tumors, abscesses, hematomas, or surgical resections (Malone et al., 1982; Landis et al., 1986; de Renzi et al., 1991), particularly right anterior temporal lobectomy. A progressive course suggests a focal neurodegenerative process, often beginning in the right anterior temporal region (Tyrell et al., 1990; Evans et al., 1995a; Joubert et al., 2003), which in some cases is also associated with degeneration of the inferior longitudinal fasciculus (Grossi et al., 2014). Rarely prosopagnosia can occur transiently with migraine (Martins and Cunha e Sa, 1999; Sándor et al., 2006). There is one intriguing study that reported that individuals with migraines perform worse on tests of face recognition, though not quite at a prosopagnosic level of severity (Yetkin-Ozden et al., 2015).

Specialized Face Recognition

Face recognition is one of the most important social perception skills. Brain regions dedicated to human face processing include the amygdala, fusiform face area, the occipital face area, a region of the ventromedial temporal cortex, and the superior temporal sulcus. These brain regions allow us to identify and store patterns for thousands of individual faces. Human face perception depends on identifying specific features, such as the eyes, nose, and mouth, and on perceiving the specific spatial arrangement of those features.

Kanwisher (2010) reported that the face-processing area was one of five areas of the human cortex that appear to be solely dedicated to specific forms of recognition. The fusiform face area on the bottom surface of the cerebral cortex just above the cerebellum responds selectively to specific faces. The extrastriate body area on the lateral surface of the brain next to the visual motion area responds selectively to bodies and body parts of specific individuals. An area of the temporoparietal junction responds selectively to information describing the mental states of other people. A tiny area of the ventral occipitotemporal cortex responds to visually presented words. The parahippocampal place area, a part of the parahippocampal complex, responds selectively to familiar and unfamiliar places and mainly to the spatial layout of a place.

Researchers Zhu, Zhang, Luo, Dilks, and Liu (2011) reported that activity of the occipital face area and fusiform face area determined the overall holistic processing of faces. Activity in these two brain regions was also correlated with skill in recognizing familiar faces and in identifying unfamiliar faces as unfamiliar. Activity in the occipital face area and fusiform face area, however, was not correlated with skill in identifying objects. The researchers argued that their findings indicated that the face recognition circuit was separate from the circuits for recognizing objects, for recognizing global forms and global motions, and separate from global scene processing. Zhu and colleagues (2011) also argued that synchronized spontaneous neural activity between the occipital and fusiform face areas indicated that the entire circuit was crucial for face recognition.

Rutishauser et al. (2011) recorded activity from more than 200 single amygdala neurons in seven neurosurgical patients with implanted depth electrodes. They reported half of these amygdala neurons responded to faces or parts of faces, and 20% of the 200 neurons responded only to the image of a whole face. They noted that these neurons showed sensitivity to the deletion of even small components of the face. Rutishauser et al. (2011) concluded that the face neurons in the amygdala code the identity of a person based on the entire face.

Prosopagnosia

Prosopagnosia is an inability to recognize familiar or famous faces despite otherwise intact visual perceptual functions. Face recognition involves a circuit from the visual cortex to the occipital face area to the fusiform face area and ultimately to the anterior temporal semantic convergence region, primarily in the right hemisphere. Faces are unique because they occur in only the visual modality and because people can distinguish thousands of faces that differ in only minor configuration (relationship of the facial features). Patients with prosopagnosia can see faces, recognize that they are faces, and appreciate facial emotions; however, they remain unable to recognize specific faces as corresponding to known individuals. Patients with prosopagnosia often compensate with salient cues or non-face cues, such as hairstyle, glasses, facial hair, or clothes. Moreover, because prosopagnosia is specific to the visual modality, hearing someone’s voice often results in immediate recognition.

Prosopagnosia can be primary due to a loss of facial feature recognition or configural patterns, or semantic due to problems accessing identifying information. In the primary form, the patients may or may not be able to form a normal percept, depending on where the lesion is. A basic apperceptive form of primary prosopagnosia may involve the occipital face area on the right and manifest with difficulties recognizing individual facial features. Those patients who have disease in the right-hemisphere fusiform face area may have damage to deposited configural or holistic patterns for known faces, exemplified by the normally greater difficulty in detecting local feature changes in a face when upside down (the “Thatcher Effect”). These patients may also have impaired ability to recognize other within-category items, such as cars, dogs, cats, and others. In contrast to primary prosopagnosias, the semantic type of prosopagnosia has deficits in semantic information about the person, usually from right anterior temporal involvement in disorders such as semantic dementia. They have problems accessing person identification “nodes,” despite retaining a sense of familiarity for the faces.

2.21.3.5.3 Two other face-selective regions: occipital face area and superior temporal sulcus

In addition to the FFA, two other face-selective regions are commonly found in most fMRI studies: a region in the posterior STS and a lateral occipital region known as the occipital face area (OFA; see Figure 5). Several studies have shown clear dissociations between the role of the STS and FFA in face processing. The first study reporting such a dissociation revealed greater FFA activation when subjects attend to identity information than to gaze information and the opposite effect in the STS (Hoffman, E. A. and Haxby, J. V., 2000). These and other studies have led to the suggestion that the STS extracts dynamic aspects of face information such as emotion or gaze direction whereas the FFA represents nondynamic face information such as identity and sex (Haxby, J. V. et al., 2000). Consistent with this finding, Grill-Spector K. et al. (2004) did not find modulation by successful face recognition in the STS, and Yovel G. and Kanwisher N. (2005) revealed similar response to same and different pairs of faces in the STS (no fMR adaptation). Furthermore, although both the FFA and the STS show lower response to inverted than upright faces, only the FFA inversion effect is correlated across subjects with the behavioral inversion effect as measured in a face identity discrimination task (Yovel, G. and Kanwisher, N., 2005).

The role of the OFA in face processing is less clear. For example, Rotshtein P., et al. (2005) presented subjects with pairs of different morphed famous faces that were either perceived as the same identity or were perceived as different identities. The FFA showed fMR adaptation for different morphs that were perceived as the same identity but not to different morphed faces that were perceived as different identities, which is consistent with its sensitivity to identity information. In contrast, the OFA appeared to be sensitive to physical differences, because it showed no adaptation to different morphed stimuli regardless of whether they were perceived as the same identity or different identities. Yovel G. and Kanwisher N. (2005) also found dissociation between the OFA and FFA in a study that examined the role of face-selective regions in the behavioral face inversion effect. Whereas the FFA showed a higher response to upright than inverted faces and a correlation with the behavioral face inversion effect, the OFA showed a similar response to upright and inverted faces and no correlation with the behavioral inversion effect. These findings suggest that the OFA is primarily sensitive to physical information in the face image rather than its specific identity. In summary, the three face-selective regions seem to play different roles in the processing of the complex, rich information in the facial image.

Functional Neuroanatomy of Face Processing: Two Alternative Models

Faces elicit highly selective and reliable neural activations in the occipital–temporal cortex (Figure 1). These face-selective activations are typically found in the inferior occipital cortex (OFA (occipital face area)), the fusiform gyrus (FFA (fusiform face area)), and the posterior part of the superior temporal sulcus (STS) (pSTS-FA (posterior superior temporal sulcus face area)) (Kanwisher & Yovel, 2006). According to the most dominant neural model of face processing, suggested by Haxby and colleagues, these face-selective areas are part of the core system of face processing as they are engaged in the visual appearance of the face (Gobbini & Haxby, 2007; Haxby, Hoffman, & Gobbini, 2000). Within this core system, it has been suggested that the FFA is engaged in the processing of invariant aspects of faces, which convey the identity of the face, whereas the pSTS-FA is engaged in the processing of the changeable aspects of faces including expression, eye gaze, head rotation, and lip movement, which convey social information about the person we observe (Figure 2(a)). This type of division of labor between the processing of identity in the FFA and social/changeable aspects of face processing in the pSTS-FA has been extensively cited and discussed in the neuroimaging of face processing literature and currently dominates the field. It is noteworthy that this model has been developed primarily based on studies that presented static face images. Interestingly, recent studies that have compared the response of these face areas to static and dynamic stimuli provide convincing evidence that the pSTS-FA shows much stronger response to dynamic versus static faces (Fox, Iaria, & Barton, 2009; Pitcher, Dilks, Saxe, Triantafyllou, & Kanwisher, 2011). This difference is found in the posterior STS-FA but is even larger in the anterior STS-FA, which shows very weak response to static faces. In contrast, the OFA and FFA show similar response to dynamic and static faces, even though dynamic faces are typically more engaging than static face images. This dissociation suggests that the STS and the ventral face areas (OFA and FFA) extract different types of information from faces. The STS that is known to be involved in the processing biological motion (Puce & Perrett, 2003) is specialized in the processing of facial information conveyed by motion, whereas the OFA and FFA are specialized in extracting information about the form of the face (O'Toole, Roark, & Abdi, 2002) (Figure 2(b)). Whereas the division to motion and form has been originally suggested as an extension to the Haxby model (O'Toole et al., 2002), the two models may actually generate opposing predictions. In particular, the pSTS-FA may be sensitive to any type of processing that involves dynamic faces, including both real and implied motion, regardless of whether it requires the extraction of face identity or facial expression. Similarly, the FFA, in addition to the processing of face identity from static faces, may also represent form variations across different facial expressions that can be extracted from still images of faces. Thus, unlike the Haxby model, which predicts no involvement of the FFA in extraction of facial expression, according to the form–motion division proposed here, both the FFA and pSTS-FA may be sensitive to facial expression but extract different types of information from it.

In the next section, we will review neuroimaging studies that have examined the neural response of these three face-selective areas to expression, trait attributes, and eye gaze. Although the majority of studies so far have focussed on the processing of static stimuli, we can still examine whether current data fit the idea that the pSTS-FA and FFA are both sensitive to facial expression but extract different types of information from it and propose an alternative model that better fits these and other findings discussed later.

3.2.2 The Occipital Face Area

While the FFA has received considerable attention for being face-sensitive, there are many other areas that are also involved in the visual recognition of faces. Moreover, increasing evidence suggests that the FFA is not sufficient in and of itself for the recognition of faces.148 One example comes from a patient with neurological damage. Researchers148 compared patient D.F., who had prosopagnosia (inability to recognize faces) and damage to the lateral occipital cortex and the medial occipito-parietal region, to age-matched controls. Behaviorally, patient D.F. was able to perform some lower-level face tasks, such as discriminating faces from other non-face objects, but could not perform higher-level face tasks, including expression and identity recognition, or face gender classification. Surprisingly, D.F. showed near-normal activation in the FFA compared to controls. However, D.F. had no activation in the area of the inferior occipital gyrus, termed the occipital face area (OFA), due to her damage in that area. All controls showed activation in the OFA, presumably allowing them to complete higher-level face tasks that D.F. could not complete. These results suggest that normal FFA activity does not equate to normal face identification performance, and that other areas must be involved to function normally.

The OFA has not received the extensive analysis that the FFA has been treated to over the last couple of decades, and as such, not as much is known about the OFA’s specific function in the face processing network. The OFA is more often than not located in the right hemisphere, consistent with other face-sensitive areas.149 While the OFA region has been observed since the advent of the fMRI, it was Gauthier et al.143 who coined the term “occipital face area” for the region in the occipital gyrus. Later, the OFA was studied more extensively, and some posit that it is an early visual detection agent in the neural network of face processing.150,151 Indeed, the OFA appears to receive information early from the visual cortex before being analyzed by higher cortical areas.150,151 Along these same lines, the OFA has been implicated to be involved in processing fluid physical face changes, rather than more static identity cues.150 More recently, the OFA has been shown to be selectively responsive to face parts, such as the eyes and mouth.149,152–154 In one study using transcranial magnetic stimulation (TMS), a device that delivers strong magnetic pulses to a selective areas of the brain, researchers artificially and temporarily produced the equivalent of lesions in the OFA.154 As suspected, temporarily disabling the OFA resulted in an inability to discriminate face parts, but an intact ability to discriminate houses. Interestingly, the researchers conducted another study where they delivered TMS pulses at varying times after stimulus onset. They showed that pulses delivered at 60 ms and 100 ms disrupted face part processing, but delivering pulses at other times did not, suggesting that the OFA is associated with early face detection processes.

TMS studies on object and face processing

The processing of visual shapes, including objects and faces, is carried out predominantly in the ventral visual cortex (for a review see Taylor and Downing, 2011). This region contains a large number of visual areas, the position of which varies between individuals and which cannot be localized accurately without fMRI data for each participant. It is perhaps for this reason that TMS research on the ventral stream has started to gather pace only recently.

The lateral occipital complex (LOC) (Malach et al., 1995) is defined as a region that responds preferentially to images of objects or large segments of objects compared with “scrambled” images of objects. TMS studies have demonstrated the necessity of this region in object processing. For example, the application of TMS over LOC significantly reduces reaction times in a simple shape-discrimination task (Ellison and Cowey, 2006). The disruptive effect of LOC TMS appears to be restricted to the contralateral visual hemifield, regardless of which hemisphere is stimulated (Chouinard et al., 2009). TMS has also revealed the causal role of this region in the maintenance of shape-related information (Schwarzkopf et al., 2010).

The LOC has also been shown to be causally involved in face processing in specific circumstances. It has been proposed that upright faces are represented by mechanisms specialized for upright faces, whereas inverted face representation depends on more general object recognition mechanisms. This was tested in a TMS study by Pitcher et al. (2011), in which participants matched upright and inverted faces while TMS was delivered over each participant's functionally localized right occipital face area (OFA) or right LOC. TMS delivered over the right LOC impaired inverted only face discrimination, supporting the view that upright faces are represented by face-specific mechanisms, whereas inverted faces are represented by both face-specific and object-specific mechanisms involving the LOC.

Finally, TMS has revealed inhibitory connections within the ventral processing stream involving the LOC. In a study by Mullin and Steeves (2011), TMS applied to the left LOC impaired object categorization but surprisingly facilitated scene categorization. This finding demonstrates that the visual system retains the ability to process scenes during disruption to object processing in the LOC. Moreover, the facilitation of scene processing suggests that the LOC inhibits areas involved in global scene processing, thus revealing a network of inhibitory connections between ventral stream regions.

Faces, relative to objects, elicit selective fMRI activity in various regions of human occipitotemporal cortex (Puce et al., 1995). Two well known face-selective areas are located in the inferior occipital gyrus (OFA) and the lateral fusiform gyrus (fusiform face area, FFA). The former is accessible to TMS and has been investigated by numerous TMS studies. Stimulation of the right OFA has been found selectively to impair a face discrimination task while having no effect on object and body discrimination tasks (Pitcher et al., 2009). In contrast, TMS applied over neighboring areas (LOC and the extrastriate body area, EBA) had no impact on face discrimination. This result demonstrates that TMS possesses the necessary spatial specificity selectively to impair the functions of specific extrastriate regions independently of the surrounding regions.

The role of the OFA appears to be restricted to specific aspects of face perception. For example, TMS applied over the right OFA impaired the discrimination of face parts but not the spacing between these parts (Pitcher et al., 2007). Gilaie-Dotan et al. (2010) used state-dependent TMS to investigate whether OFA is involved in generic face- and shape-related processing or high-level conceptual processing of identity. Critically, no TMS effects on identity were found, indicating that OFA is likely to be involved in the processing of more generic features of their preferred stimulus categories. More recently, Silvanto et al. (2010) used TMS to show that interference with activity in OFA modulated priming effects on a task requiring perception of simple, symmetrical, two-dimensional nonface shapes, implying a role for this region that is not limited strictly to faces.

Neutral Face Tasks

Social cognition was investigated in ASD via the use of emotionally neutral face tasks during fMRI, finding either no activation or lower activation in the FFA (Hubl et al., 2003; Pierce et al., 2001; Schultz et al., 2000). In addition, increased activation was found in object-related brain regions in the ASD group. Another study demonstrated decreased activation in the FFA, occipital face area, and STG in adults with autism vs. healthy controls (Humphreys et al., 2008). However, researchers found increased FFA activation in response to a neutral face task when subjects were cued to the eye region by a central fixation cross, suggesting an association between FFA activation and degree of gaze fixation in ASD (Hadjikhani et al., 2004).

Investigators studied abnormal functional connectivity in ASD during a neutral face processing task, finding that greater social impairment was associated with decreased FFA–amygdala connectivity and increased amygdala–right IFG connectivity (Figure 3.1.3) (Kleinhans et al., 2008). These findings suggest that abnormalities within the limbic system may contribute to social deficits in ASD. A subsequent fMRI study investigated whether abnormal habituation to neutral face stimuli characterized amygdala dysfunction in ASD (Kleinhans et al., 2009). Decreased amygdala habituation was associated with increased social impairment in the ASD group.

Why Faces?

As social beings, humans rely heavily on information from faces. Faces contain a wealth of information about age, gender, emotion, and identity, which are all extremely important factors in determining how we interact with each other. This abundance of information is contained in a relatively restricted stimulus set: Faces are made up of a set of two eyes over a nose and a mouth. The spatial configuration of these elements is homogeneous across individuals. As a result, a dedicated system has developed in the brain that is responsible for processing this extremely special and meaningful stimulus. Selective damage to this system can result in prosopagnosia.