New Research In
Physical Sciences
Social Sciences
Featured Portals
Articles by Topic
Biological Sciences
Featured Portals
Articles by Topic
- Agricultural Sciences
- Anthropology
- Applied Biological Sciences
- Biochemistry
- Biophysics and Computational Biology
- Cell Biology
- Developmental Biology
- Ecology
- Environmental Sciences
- Evolution
- Genetics
- Immunology and Inflammation
- Medical Sciences
- Microbiology
- Neuroscience
- Pharmacology
- Physiology
- Plant Biology
- Population Biology
- Psychological and Cognitive Sciences
- Sustainability Science
- Systems Biology
Spatiotemporal dynamics of word retrieval in speech production revealed by cortical high-frequency band activity
Edited by Gary S. Dell, University of Illinois at Urbana–Champaign, Champaign, IL, and approved May 2, 2017 (received for review December 16, 2016)
Significance
Word retrieval is essential to language production, relying on the activation of conceptual and word representations in memory followed by the selection of the correct word. The detailed spatiotemporal cortical dynamics of this core language process are not well-known. By using direct cortical recordings, we show that the activation of concepts or word representations and their selection co-occur in time and engage widespread brain networks and overlapping brain regions. In contrast with modular brain models of language production, our data do not support a clear division of labor between brain regions during these early stages of language production. Rather, we suggest that overlapping brain mechanisms optimize word retrieval.
Abstract
Word retrieval is core to language production and relies on complementary processes: the rapid activation of lexical and conceptual representations and word selection, which chooses the correct word among semantically related competitors. Lexical and conceptual activation is measured by semantic priming. In contrast, word selection is indexed by semantic interference and is hampered in semantically homogeneous (HOM) contexts. We examined the spatiotemporal dynamics of these complementary processes in a picture naming task with blocks of semantically heterogeneous (HET) or HOM stimuli. We used electrocorticography data obtained from frontal and temporal cortices, permitting detailed spatiotemporal analysis of word retrieval processes. A semantic interference effect was observed with naming latencies longer in HOM versus HET blocks. Cortical response strength as indexed by high-frequency band (HFB) activity (70–150 Hz) amplitude revealed effects linked to lexical-semantic activation and word selection observed in widespread regions of the cortical mantle. Depending on the subsecond timing and cortical region, HFB indexed semantic interference (i.e., more activity in HOM than HET blocks) or semantic priming effects (i.e., more activity in HET than HOM blocks). These effects overlapped in time and space in the left posterior inferior temporal gyrus and the left prefrontal cortex. The data do not support a modular view of word retrieval in speech production but rather support substantial overlap of lexical-semantic activation and word selection mechanisms in the brain.
- word retrieval
- language production
- electrocorticography
- cortical high-frequency band activity
- semantic interference
Adults fluidly utter two to three words per second selected from as many as 100,000 regularly used words in the mental lexicon (1). Word retrieval accesses and fits an appropriate word to ongoing speech and is core to language production, as evidenced by the severe impact of word-retrieval deficits such as anomia.* Despite the importance of word retrieval in language and the immense personal and societal cost caused by its disruption in neurological disorders, its neural basis is poorly understood. The present study sheds light on the spatiotemporal dynamics of word activation and selection at the subsecond scale by using direct cortical recordings obtained in neurosurgical patients. In particular, we investigate whether these processes are supported by overlapping versus distinct brain regions.
Conceptually driven word retrieval is enabled through the activation of a set of semantic features or concepts, the activation of the corresponding words or lexical representations, and finally the selection of the target word. Word retrieval is often investigated by using semantic context manipulations. Word retrieval is facilitated in semantically related contexts, an effect referred to as semantic priming [observed in language comprehension (3) and language production (4)], or hampered, leading to the observation of semantic interference effects in behavioral measures (5, 6). The specific stage of word retrieval reflected by the semantic priming and interference effects has been subject to debate. Specifically, semantic priming has been attributed to the conceptual (e.g., ref. 7) or to the lexical activation levels (e.g., ref. 8). The semantic interference effect has been traditionally interpreted as reflecting competition at the level of lexical selection (e.g., ref. 1), although some studies suggest that the effect is explained by changes in semantic-to-lexical connection weights (4, 9) (for a more detailed discussion of the competitive vs. noncompetitive nature of lexical selection, see refs. 8,9,11). Our study was not designed to distinguish the theoretical loci of these effects. Thus, we refer to lexical-semantic activation as being indexed by semantic priming but return to the distinction between semantic and lexical activation in the Discussion. We also discuss word selection in the broader sense, assuming this process to be sensitive to semantic interference. Importantly for the present purpose, some models assume word selection occurs through internal dynamics of lexical-semantic representations (12), whereas others suggest that an external mechanism acts upon the activation of these representations to help select the correct candidate word (e.g., refs. 10, 13). In this study, we use the semantic priming and interference effects to test whether the external selection module suggested by the second class of models is hosted by brain regions different from those engaged in initial lexical-semantic activation.
The blocked-cyclic picture-naming paradigm (5) is widely used to study the cognitive and neurological correlates of word retrieval (e.g., refs. 6, 14⇓⇓⇓–18). In this paradigm, pictures are presented one by one in semantically homogeneous (HOM; i.e., all pictures are from the same semantic category) or heterogeneous (HET) blocks (i.e., all pictures are from different semantic categories). The pictures are repeated several times per block (typically between four and six times), leading to a main repetition priming effect (4, 9). Performance as assessed with naming latencies and error rates is typically worse in HOM than in HET blocks from the second cycle onward. Thus, in HOM blocks, repetition priming is countered by a semantic interference effect indexing word selection difficulty, which is increased when semantically related competitors receive additional activation.
A frontotemporal network of brain regions has been associated with word retrieval. In particular, the left inferior frontal gyrus (IFG) has been associated with word selection. This region has been proposed to provide top-down control to help overcome interference caused by semantically related alternatives (15, 17), thus hosting the aforementioned external selection “module” (10). Medial frontal regions such as the presupplementary motor area (pre-SMA) and the anterior cingulate cortex have also been associated with response selection in and outside the field of language production (19⇓–21). The left posterior temporal regions, including the middle temporal gyrus (MTG) (22) and the inferior temporal gyrus (ITG) (23), have been proposed to play a central role in word retrieval. Most reports focus on word selection, as indexed by semantic interference (i.e., more activity in HOM vs. HET contexts). Interestingly, however, some fMRI (21, 24), but also magnetoencephalography (MEG) (14, 25) and EEG, picture-naming studies of word retrieval (16, 18) have shown that the reverse effect, semantic facilitation or priming, is also observed by using paradigms eliciting semantic interference effects on reaction times (RTs). This effect is manifested by early increased activation in HET blocks versus HOM blocks, reflecting more effortful lexical-semantic activation in HET than HOM blocks as a result of reduced semantic priming in HET versus HOM blocks. This suggests that signatures of lexical-semantic activation in speech production can be observed even when the main behavioral effect is in the opposite direction. The cortical spatiotemporal interplay of lexical-semantic activation and word selection is unclear, but recent studies have suggested a frontotemporal division of labor whereby the left temporal lobe would be predominantly involved in supporting lexical-semantic activation and the frontal lobe would support top-down control processes narrowing the search for the target word (21, 25). In the present study, we address the precise spatiotemporal network underlying word retrieval in speech production in the human brain by using direct cortical recordings in neurosurgical patients, offering millisecond- and centimeter-scale resolution. Recent intracranial EEG studies have provided rare insight into the spatiotemporal dynamics of speech production (26, 27) and the speech output stages in the motor and sensory cortices (28), but none have focused on the cortical spatiotemporal dynamics of word retrieval (however, see refs. 29⇓–31 for hippocampal word retrieval-related activity). In the present study, we used the blocked-cyclic picture-naming paradigm, a psycholinguistic task specifically tailored to focus on word retrieval processes in language production. We provide insights into the spatiotemporal dynamics of lexical-semantic activation and word selection in word retrieval during speech production.
Results
Patients and Behavior.
Nine patients participated in the study, including seven with left hemisphere coverage (SI Appendix, Fig. S1). Here we report the effects of semantic context and its interaction with other factors under analysis. Other effects not involving semantic context are reported in the SI Appendix. The electrophysiological data analysis was focused on left hemisphere regions previously associated with word retrieval. The two patients with right hemisphere coverage had minimal coverage over the lateral frontal, medial frontal, and posterior temporal cortices (SI Appendix, Fig. S5, provides an overview of the semantic context effects per electrode in the right hemisphere time-locked to the stimulus and to the response).
Of the seven patients with left hemisphere coverage, one (patient IR02; SI Appendix, Fig. S1, orange), whose seizure focus was in the posterior medial prefrontal cortex (PFC; in the pre-SMA area; SI Appendix, Fig. S2, shows the resected area), had poor performance (error rate > 40%) in this task, and his behavioral and electrocorticography (ECoG) data were analyzed separately. His semantic interference effect on naming latencies (321 ms) was more than 3 SDs larger than that of the other patients (mean, 43 ms; SD, 82 ms). This case study indicates that, when brain tissue in the posterior medial PFC is abnormal, interference caused by semantically related alternatives is more difficult to overcome. This suggests a causal role of the pre-SMA in word selection, as suggested by fMRI studies (19, 20).
In the remaining eight patients, we found the expected pattern of results in the behavioral data (mean naming latencies and SDs per semantic context and per presentation number are presented in Fig. 1A and SI Appendix, Table S1). Because the semantic interference effect can be absent or even reversed in the first presentation and because performance is more variable in this first cycle (32, 33), we performed the analysis without the first presentation of the stimuli (as in refs. 17, 34; but see SI Appendix for an analysis including presentation 1). There was a main effect of semantic context on log-transformed† naming latencies, Wald χ2(1) = 4.82, P = 0.028: Participants were slower in HOM versus HET blocks, revealing a semantic interference effect (SI Appendix, Table S2A, provides βraw, CI, SE, and t values). Finally, there was an interaction between semantic context and presentation number, Wald χ2(1) = 6.38, P = 0.012: with increasing repetitions, naming latencies increased in HOM versus HET blocks. The error rate was low overall [median, 3.64%; interquartile range (IQR), 1.82–8.85], and there was no significant effect of any of the experimental parameters we controlled for on accuracy rates when the first presentation of the stimuli was removed (as detailed in SI Appendix and Fig. 1B).
Electrocorticography.
We focused our electrophysiological analysis on high-frequency band (HFB) activity (70–150 Hz) because HFB power has been found to be the most reliable spectral measure of cortical activation in language production tasks (26, 27) and is the most commonly used spectral profile in intracranial language research (35). In addition, HFB is ubiquitous in the human cortex, is known to be a robust correlate of local neuronal activation, and is reliable on a single-trial basis (36, 37).‡ We first examined the presence of HFB in each electrode in 1,000-ms stimulus and response-locked time windows (Methods and SI Appendix, Fig. S3). Of the 617 artifact- and seizure-free electrodes across patients, 304 had significant HFB time-locked to the stimulus (median, 37 electrodes per patient; IQR, 31–46), and 307 had significant HFB time-locked to vocal onset (median, 37 electrodes per patient; IQR, 34–39; SI Appendix, Fig. S4). Thus, a median of 49% of included electrodes were task-active electrodes that were stimulus-locked (IQR, 48–53), and a median of 51% of included electrodes were task-active electrodes that were response-locked (IQR, 46–58). Active electrodes were observed in all cortical lobes. The analyses of experimental effects were carried out on these active electrodes in the frontal and temporal lobes. We used linear mixed-effects models to analyze how HFB amplitude was modulated by semantic context and its interaction with the other factors. These included presentation number, stimulus position, cortical structure,§ and time window (i.e., divided in five 200-ms chunks, stimulus- and response-locked) in the left frontal and temporal cortices in the six patients with normal language production (Methods).
Stimulus-Locked Semantic Context Effects.
In the stimulus-locked analyses, semantic context effects were found in the temporal- and frontal-lobe models. The distribution of raw β-weights per window on the left lateral surface for the semantic context effects that were stimulus-locked are presented in Fig. 2A.
In the temporal lobe, semantic context interacted with window, Wald χ2(1) = 7.75, P = 0.005: Semantic interference increased the further away from stimulus onset (Fig. 3A and SI Appendix, Table S4). There was also a three-way interaction among semantic context, window, and structure, Wald χ2(3) = 11.23, P = 0.011, indicating that the semantic interference effect increased only in the ITG, whereas the semantic priming effect increased in the other structures: MTG vs. ITG, βraw = −1.57; CI = −2.592 to 5.59 × 10−1; SE = 5.18 × 10−1; t = −3.04; superior temporal gyrus (STG) vs. ITG, βraw = −1.60; CI = −2.63 to 5.78 × 10−1; SE = 5.23 × 10−1; t = −3.03; ventral vs. ITG, βraw = −8.84 × 10−1; CI = −2.23 to 4.63 × 10−1; SE = 6.72 × 10−1; t = −1.29. This explains the absence of an overall main effect of semantic context, Wald χ2(1) = 2.19, P = 0.139, in the temporal lobe. The semantic interference effect in the ITG emerged in the 400–600-ms time window after stimulus onset, similar to that observed in the frontal lobe (Fig. 4A). Before that time window, the dominant effect in this brain region was semantic priming (this was observed for three of four patients with electrode coverage in the ITG; SI Appendix, Fig. S6A). This suggests that this region is initially involved in lexical-semantic activation followed by word selection, indicating that the same brain region may be involved in these two complementary processes supporting word retrieval at different time points.
In the frontal lobe, there was a marginal semantic context effect, Wald χ2(1) = 3.21, P = 0.073, and an interaction between semantic context and window, Wald χ2(1) = 4.54, P = 0.033 (Fig. 3A and SI Appendix, Table S3, provide statistical details). Importantly, the direction of the evolution of the semantic context effect depended on the region of the frontal cortex involved. There was a three-way interaction among semantic context, window, and structure, Wald χ2(3) = 8.96, P = 0.030. In the lateral PFC and medial primary motor cortex and premotor cortex (M1/PMC), in comparison with the lateral M1/PMC, semantic interference tended to increase with time: lateral PFC vs. lateral M1/PMC, βraw = 6.09 × 10−1; CI = 0.05–1.16; SE = 2.83 × 10−1; t = 2.15; medial M1/PMC vs. lateral M1/PMC, βraw = 8.31 × 10−1; CI = −0.13 to 1.80; SE = 4.92 × 10−1; t = 1.69. There was no significant difference in the direction of the interaction between the lateral M1/PMC and the medial PFC (SI Appendix, Table S3). Semantic context did not interact with any of the other factors analyzed. These results underlie the role of the lateral PFC and medial M1/PMC in semantic interference resolution for word selection starting approximately 400 ms after stimulus onset.
We also found substantial temporal overlap between the semantic interference and priming effects in the temporal and frontal lobes. Indeed, whereas semantic interference increased in the ITG, lateral PFC, and medial M1/PMC, semantic priming increased in the other structures (as reported earlier; SI Appendix, Tables S3 and S4). There was no significant difference between the time windows in which the maximal semantic interference effect was reached in the ITG, lateral PFC, and medial M1/PMC compared with when the maximal priming effect was reached in the other structures, t(20.78) = 0.85, P = 0.405 (average of 600–800 ms after stimulus onset). This observation is in agreement with substantial temporal overlap between the two processes, in agreement with models allowing some degree of interaction between lexical-semantic activation and word-selection brain regions (e.g., refs. 13, 43).
Response-Locked Semantic Context Effects.
Response-locked effects of semantic context were clearer in the frontal- than in the temporal-lobe models (Fig. 2B). In the temporal lobe, there was no main effect of semantic context response-locked, Wald χ2(1) = 2.11, P = 0.146, nor any two- or three-way interaction of semantic context with any of the other factors under analysis (SI Appendix, Table S6, provides statistical details). The observation that semantic context effects were not as clear for response-locked compared with stimulus-locked conditions in the temporal lobe suggests that temporal-lobe regions, and especially the ITG, are engaged in word retrieval in a stimulus-bound manner.
In the frontal lobe, there was a main effect of semantic context, Wald χ2(1) = 6.45, P = 0.011: There was more HFB activity in HOM than HET blocks in all frontal structures under analysis (Fig. 3B and SI Appendix, Table S5). Thus, the response-locked effects of semantic context were clearer than the stimulus-locked ones in the frontal lobe. This suggests a sustained involvement of the PFC in semantic interference resolution. In addition, semantic interference decreased the closer to the vocal onset, as indicated by an interaction between semantic context and window, Wald χ2(1) = 4.47, P = 0.03. As can be seen in the averages, semantic interference was present until approximately 350 ms before vocal onset.
The stimulus-bound engagement of the temporal cortex therefore contrasts with the more sustained involvement of the PFC and underlies the different roles of these brain regions in word retrieval.
HFB–RT Correlations.
These results do not take into account how cortical response strength relates to trial-by-trial performance in these regions during word retrieval. To address this, we examined how within-trial mean HFB for stimulus- and response-locked time windows correlated with RTs as measured with naming latencies. We calculated Spearman rank correlation tests at each electrode site (SI Appendix, Fig. S7, details methods and ρ correlation coefficient per time window and per electrode time-locked to the stimulus and to the response).
As was clearly visible in the response-locked analysis of the frontal-lobe data, structures showing semantic interference in given time-windows showed predominantly positive HFB–RT correlations, whereby higher within-trial mean HFB values were associated with longer RTs in the same time windows (Fig. 4B). HFB–RT correlations overall became less positive the closer to vocal onset, Wald χ2(1) = 13.79, P < 0.001, and were maximal before 350 ms before vocal onset (Fig. 4B). This was true for all or most patients, depending on the brain structure (all patients in lateral PFC, four of five in the lateral M1/PMC, and one of one in the medial M1/PMC, but only one of two in the medial PFC; SI Appendix, Fig. S6B). There was also a main effect of structure, Wald χ2(3) = 10.44, P = 0.015, as HFB–RT correlations were overall more positive in the lateral M1/PMC than in the other frontal-lobe structures.
In the stimulus-locked analysis of the frontal lobe, there were no significant effects of window, structure, or their interaction on HFB–RT correlations (Fig. 4A and SI Appendix, Fig. S7A and Table S7A). In the temporal lobe, the HFB–RT correlation patterns were not as comparable to that of the semantic context effect time-locked to the stimulus or to the response (SI Appendix).
Overall, where semantic interference was observed, stronger cortical response strength as indexed by HFB amplitude was associated with longer naming latencies. When word retrieval is more difficult, increased response-locked activity as a function of increasing RTs is predominant in the frontal lobe (Fig. 4B and SI Appendix, Fig. S7B).
Frontal Lobe Versus ITG Interactions.
We also investigated if cortical response strength covaried between the main regions involved in word selection as indexed by the semantic interference effect (SI Appendix describes methods). Significant semantic interference effects were found in the frontal lobe and in the ITG. Among the six patients we tested with left hemisphere coverage, one had electrodes over the frontal lobe (lateral and medial) and the ITG (i.e., patient ST32; SI Appendix, Fig. S1, dark blue). In this patient, we tested whether mean HFB correlated on a trial-by-trial basis between these sites.
In the stimulus-locked analysis, we found significant correlations between the lateral PFC and ITG between 400 and 1,000 ms after stimulus onset (ρ = 0.437; corrected P < 0.001), corresponding to the interval when semantic interference was observed in these regions, but also between stimulus onset and 400 ms after stimulus onset (ρ = 0.313; corrected P < 0.001; SI Appendix, Fig. S9A). This was not the case between the other frontal structures showing semantic interference effects and the ITG between 400 and 1,000 ms after stimulus onset, nor between stimulus onset and 400 ms after stimulus onset (SI Appendix).
Response-locked, we found significant correlations between the lateral PFC and ITG between −750 and −350 ms before vocal onset (ρ = 0.518; corrected P < 0.001), corresponding to the interval when semantic interference was observed in these regions, but also between −350 ms and 250 ms around vocal onset (ρ = 0.505; corrected P < 0.001; SI Appendix, Fig. S9B). This was also true between the medial PFC and the ITG between −750 and −350 ms after stimulus onset (ρ = 0.177; corrected P = 0.027) and between −350 ms and 250 ms around vocal onset (ρ = 0.204; corrected P < 0.001). Between the medial M1/PMC and the ITG, the correlation was significant only between −350 ms and 250 ms around vocal onset (ρ = 0.273; corrected P < 0.001), and not between −750 and −350 ms before vocal onset (SI Appendix). This suggests that the lateral PFC and the medial PFC interact with the ITG on a trial-by-trial basis to support word retrieval. The later involvement of the medial M1/PMC suggests a possible role in verbal response monitoring, as suggested in previous publications (44, 45).
Discussion
We provide a detailed picture of the spatiotemporal cortical dynamics of lexical-semantic activation and word selection during overt speech production. Several conclusions can be drawn from our observations. First, semantic priming and interference effects were widespread across the cortical mantle, suggesting that distributed brain regions are involved in lexical-semantic activation and selection. Second, these effects coexisted in time and in space, providing evidence that distinct brain regions can be involved in more than one word retrieval-related process. Third, despite considerable overlap, semantic priming was predominant in some structures whereas semantic interference was predominant in others.
Widespread Semantic Priming and Interference Effects.
Several studies that used fMRI (e.g., refs. 46, 47) have argued for a widespread distribution of the semantic system. The linguistic system has usually been associated with a more restricted brain network (48, 49), sometimes highlighting only one core brain region per stage (48). Word retrieval or selection in particular has been associated with the midsection of the left MTG (48) and with parts of the left MFG (49). Our results indicate that word selection, as indexed by semantic interference, is in fact supported by a wide network of left frontal and temporal brain regions. We note that the question of whether the brain regions showing the observed effects are critical to the processes of interest cannot be addressed based on the present findings. The existing literature (22, 23) suggests that brain regions critical for word selection are more spatially constrained than the network revealed in our study. However, our results reveal that word selection is not supported solely by classic language-specific regions. This interpretation is in agreement with models suggesting that a selection mechanism external to the lexical access system is needed for word retrieval (e.g., ref. 10).
The locus of semantic priming has been subject to debate, with some attributing it to the conceptual level (e.g., ref. 7) and others attributing it to the lexical activation level (e.g., ref. 8). Although our study was not designed to address these propositions, the widespread distribution of the semantic priming effect observed with HFB activity suggests a close link with the broadly represented semantic system (46, 47). We note that it is also possible that the semantic priming effects we observed could reflect conceptual and lexical activation, or that the two stages largely overlap. Recent models have suggested that lexical representations include item-specific semantic knowledge (e.g., ref. 50). It seems less likely that a much larger brain network than anticipated would be involved in lexical activation alone based on the available neuropsychological literature (22, 23, 51). Indeed, lexical activation deficits without semantic impairments appear to result from lesions confined to the posterior temporal cortices (51). In addition, lexical activation is generally considered to be more automatic than word selection (1), and is hence less in need of the involvement of nonlinguistic brain regions associated with control processes.
Semantic Priming and Interference Effects Overlap in Time and Space.
In most time windows, we observed semantic priming and interference co-occurring in different brain structures, and, in some structures, we observed both effects occurring sequentially. In particular, in the left ITG, semantic priming was observed until 400 ms and was then replaced by semantic interference. This, along with the absence of interaction between brain structure and semantic context, indicates that the division of labor between the activation and selection processes is not absolute. This is in disagreement with a simplified picture proposed in meta-analyses and reviews of language production (48, 49), in which brain regions are generally assigned one particular cognitive function, supporting a modular view of processing. In this view, the posterior ITG is associated with semantic processing but not with word retrieval, which is supported by left PFC regions (49). Our results do not support this one-to-one mapping but instead suggest that a given brain region may be involved in the spread of lexical-semantic activation as well as in subsequent word selection. In this sense, our results support models describing a selection process internal to the lexical-semantic system (e.g., ref. 12) but suggest that an external selection mechanism is also engaged in this task. Our results therefore reconcile computational models suggesting that the selection mechanism is external to the lexical-semantic activation system (e.g., ref. 10) and models supporting a selection process internal to the lexical-semantic system (e.g., ref. 12).
Alternatively, if semantic priming indexes the activation of conceptual and not lexical representations, our results would indicate an overlap between conceptual activation and subsequent selection of the target lexical representation. In either case, and in alignment with recent proposals (e.g., ref. 50), our data support a widely distributed lexicosemantic system in which specific brain regions can be involved in more than one word retrieval-related process. We propose that such an organization is beneficial to optimal performance. Indeed, lexical-semantic activation and word selection are closely related and interdependent in speech production: one cannot, in theory, select a word without the prior activation of the corresponding concepts and lemma (1, 13). Thus, having the same cortical regions performing these processes could enhance word selection speed. An analogy with the motor and sensory cortices can be drawn with motor neurons found in the sensory cortex (52) and sensory neurons found in the motor cortex (53). Such an organization is believed to optimize sensory and motor adjustments, respectively. A similar perspective can be used to understand our results shedding light on our understanding of the neurobiological basis of language production.
Anatomical Division of Labor.
The temporal evolution of the semantic context effect depended on the brain structure engaged. In the left STG, MTG, and ventral temporal cortex, but also in the lateral M1/PMC and medial PFC, semantic priming increased the further away it was from stimulus onset.¶ In other structures, semantic interference increased the further away from stimulus onset and was maximal until 350 ms before vocal onset in all frontal structures. Therefore, during this subsecond time scale of observation, semantic priming was predominant in some structures whereas semantic interference was predominant in others.
The brain structures we found to predominantly reflect semantic priming have previously been associated with lexical-semantic activation in language production and comprehension, especially for the temporal-lobe structures (21, 22, 25, 51, 54). The semantic priming effects found in the lateral M1/PMC could be attributed to possible interactions between this area and semantic processing, even though the causal role of this region in the representation of semantic knowledge is unclear (55). Conversely, brain regions found to mostly reflect semantic interference have been previously associated with word selection, especially the lateral PFC and medial frontal cortex (15, 19, 25, 56). The posterior ITG has been associated with semantics (49), but also with word retrieval, as evidenced by negative correlations between anomic rate and resting-state brain metabolism in this area (23). Our results reconcile these interpretations and suggest that this brain region may be involved in both processes at different time points. The semantic priming and interference effects reached their maxima around the same time (on average between 600 and 800 ms after stimulus onset). Thus, our results support temporal overlap between lexical-semantic activation and word selection, suggesting that lexical-semantic activation does not end when word selection starts. This is in agreement with a recent EEG study (57) and most language-production models, in which some degree of cascaded processing between lexical-semantic activation and word selection is allowed (e.g., refs. 1, 13, 43). In addition, the fact that the semantic interference effect was mainly present before 350 ms before vocal onset is in agreement with published chronometric estimates (48). This suggests that the word-selection process is mostly over by this point in time, leaving time for the subsequent phonological encoding and articulatory processes to take place.
A similar division between temporal and frontal regions was observed in the HFB–RT correlation patterns. Frontal regions, which showed an overall larger semantic interference effect, showed stronger cortical response strength associated with longer RTs, especially when time-locked to vocal onset. This is similar to observations in other cognitive domains such as in working memory tasks, in which γ-band (30–60 Hz) amplitude in the frontal cortex increases with memory load (58). Mirroring the semantic interference effect, HFB–RT correlation coefficients were maximal up to 350 ms before vocal onset. These results are in agreement with the idea that the frontal cortex engages as a function of trial-by-trial difficulty in language production, as observed in other cognitive functions. In the context of this picture-naming task, the frontal cortex seems to play an adaptive cognitive control role in interference resolution for word selection.
Finally, HFB power was correlated trial by trial between the lateral PFC and medial PFC and the ITG in the time windows in which semantic-interference effects were observed, supporting the idea that the left PFC interacts with the left ITG in a trial-by-trial manner to support word selection. We note that the engagement of the left PFC in word retrieval has also been proposed to be task- or situation-specific. In particular, some have proposed that the left PFC acts as a top-down mechanism allowing bias of the level of activation of task-relevant alternatives (59), whereas others have proposed that the left PFC hosts a proactive control mechanism acting across cognitive domains (17, 60, 61). In support of this, two recent studies have failed to report a larger semantic-interference effect in patients with left PFC damage compared with controls (62, 63) using other paradigms eliciting semantic interference effects on behavioral measures (i.e., the picture-word interference and the cumulative interference paradigms; refs. 62, 63). Critically, these other paradigms do not involve the repetition of pictures within blocks, and hence do not allow top-down biasing of task-relevant items or proactive control to take place.
One caveat concerning the spatial and temporal precision of our claims is worth mentioning. ECoG recording restrictions resulted in sparse and spatially biased spatial sampling, and this constraint required collapsing across broad cortical structures for statistical analysis (as in ref. 64). Here, we also collapsed our analysis over 200-ms time windows to simultaneously test for spatial and temporal effects, thus limiting our temporal resolution to this scale.
To conclude, these results provide insights into the cortical dynamics of word retrieval in speech production. Our results show that a widespread network of brain regions supports different aspects of word retrieval. Medial and left PFC regions are involved in trial-by-trial interactions with the posterior ITG to help overcome interference caused by semantically related alternatives in word selection. Finally, unlike prior concepts of a strict modular organization of word retrieval, our ECoG results show that the same brain region may be involved in the activation of conceptual or lexical representations as well as word selection in different time windows.
Methods
Participants.
Nine patients (three women; median age at time of testing, 26 y; IQR, 23–42 y) undergoing neurological treatment for refractory epilepsy participated in the study. During clinical treatment, the patients were implanted with 74–157 electrodes (grids and strips; electrode spacing, 0.6–1 cm) covering extensive portions of the lateral cortices in both hemispheres (SI Appendix, Fig. S1). Seven patients had left and two patients had right hemisphere coverage. Electrode placement and medical treatment were dictated solely by the clinical needs of the patient. Electrophysiological signals were monitored by clinicians for approximately 1 wk. During lulls in clinical treatment, patients willing to participate in the study provided written and oral informed consent. Patients were tested at six different institutions: Stanford Hospital, Stanford, CA; California Pacific Medical Center, San Francisco, CA; University of California, San Francisco, Benioff Children’s Hospital and Research Center, Oakland, CA; University of California, Irvine, Health, Irvine, CA; Albany Medical College, Albany, NY; and The Johns Hopkins Hospital, Baltimore, MD. The institutional review board of each institution approved the research that was conducted at each respective location. Antiepileptic medications were discontinued 2–3 d beforehand, and patients were seizure-free for at least 5 h before testing. All individuals had normal language and normal or corrected-to-normal vision and were native speakers of English (n = 8) or Spanish (n = 1). They all performed the task in their native language. All but one patient were right-handed, and the one left-handed patient was left hemisphere-dominant for language.
Material and Design.
The stimuli were 550 × 240-pixel high-line drawings of common objects or animals selected from published collections (65, 66). Their name agreement was very high (median, 95%; IQR, 90–99%). They were presented in free viewing on a laptop computer screen 50–60 cm from the patient’s eyes. A total of 16 pictures were used in the experiment. They were issued from four different semantic categories (clothing items, animals, musical instruments, and human dwellings), and were presented four times within HOM versus HET blocks (11). Because participants also performed a Simon task (67) (not reported here), the pictures were colored in green or purple and were presented on the left or the right of the fixation point. Within each experimental run, the order in which the items were presented was mixed pseudorandomly by using the software MIX (68) such that consecutive items were phonologically unrelated, that is, two pictures in a row never had the same initial phoneme.
Procedure.
The experiment was controlled by Eprime 2.0 Professional (Psychology Software Tools) or BCI2000 (69) (n = 2 patients), allowing online recording of the participants’ verbal response. A trial consisted of the following events: (i) a fixation point (plus sign presented at the center of the screen) for 500 ms, (ii) a picture for 2,000 ms that participants had to name as fast and as accurately as possible, and (iii) a blank screen for 2,000 ms. Underneath a photodiode placed at the bottom left of the screen, a white rectangle appeared and disappeared along with the stimulus to mark the onset and offset of picture presentation. Vocal onsets were used as the response-onset measure. There were 4 blocks of 32 trials each. The participant could rest for as long as necessary between blocks. Before the task, participants were familiarized with the picture names, and the experimenter made verbal corrections when an incorrect response was produced. The experimental session lasted 10–15 min.
Data Acquisition.
Verbal responses were acquired at a sampling rate of 44 kHz. Electrophysiological and peripheral data (photodiode and microphone input) were acquired simultaneously by using a 128-channel recording system (Tucker Davis Technologies) at Stanford Hospital (3,052-Hz digitization); a different 128-channel recording system (Nihon Kohden) at California Pacific Medical Center, Children’s Hospital, and University of California, Irvine (1,000-Hz digitization); a 112-channel g.USBamp biosignal acquisition system (9,600-Hz digitization; g.tec) at Albany Medical College; and a 128-channel Stellate Harmonie recording system (1,000 Hz digitization; Natus Medical) at Johns Hopkins University. Data were recorded by using a subdural electrode reference and a scalp ground.
Electrode Localization.
Structural preoperative MRI and postimplantation CT scans were acquired for each patient. These scans were coregistered to the same space by using two nonlinear transformations based on normalized mutual information implemented in the Bioimage suite (70), as in a previous publication (27). The second transformation was used to correct for slight shifts in brain morphology caused by the electrodes. The results were then compared with an intraoperative photo image of the exposed grid after it was sutured to the dura. Brains and electrodes were transformed into Montreal Neurological Institute (MNI) space across subjects only for visual display. Electrodes were classified according to their anatomical location within each patient’s anatomical space. Electrode location was coded according to two levels: lobe (frontal and temporal) and structure (regrouping one or several gyri). The frontal lobe was divided into four structures: the lateral and medial M1/PMC, grouping frontal electrodes on or posterior to the precentral sulcus and anterior to the Rolandic sulcus; and the lateral and medial PFC, grouping the inferior, middle, and superior frontal gyrus (IFG, MFG, and SFG, respectively). The orbitofrontal and frontopolar cortices (grouping the ventral part of the frontal lobe and the most anterior part of the SFG and MFG, as defined by being anterior to the IFG’s anterior boundary but lying ventral to the anterior commissure axis) were not included in the analysis. The temporal lobe was divided into four structures: the STG, MTG, ITG, and ventral temporal lobe (not including the electrodes also visible on the lateral views). Each patient’s electrode location was defined by a neurologist.
Data Preprocessing and Analysis.
Behavioral data.
The accuracy of the responses and the verbal RTs were measured offline by using CheckVocal (71). Trials were excluded from the analysis of the correct responses if the participant did not respond or produced any kind of verbal error: partial or complete production of incorrect words or verbal dysfluencies (e.g., stuttering, utterance repairs).
Statistical analysis was performed within R version 3.1.1 (72) using the packages “lme4” to compute the mixed effect models (73) and “car” to compute analysis of deviance tables for the fixed effects of the mixed-effect models (74). We analyzed the data by using generalized linear (for RTs) and logistic (for accuracy rates) mixed-effects models (75, 76). The analyses were performed on log-transformed RTs and accuracy rates. We tested for fixed effects of semantic context (HOM vs. HET), presentation number (from two to four), and stimulus position (i.e., left or right of the fixation cross) as within-subject factors, and the interaction between semantic context and presentation number. As random effects, we had intercepts for participants and picture name, as well as by-subject random slopes for within-subject factors. P values were obtained by using type III analyses-of-deviance tables (because of the presence of an interaction), providing Wald χ2 tests for the fixed effects in the generalized linear mixed-effects models. For all models, we report Wald χ2 values and P values from the analysis of deviance tables (in the main text), as well as raw β estimates (βraw), 95% CIs around these β estimates, SEs, t values for RTs, and Wald Z and associated P values for significant effects on accuracy rates (SI Appendix).
ECoG data.
All ECoG channels were inspected by a neurologist to identify those with epileptiform activity and artifacts (e.g., as a result of poor contact or high-frequency noise). These channels and those that were located over tissue that was later resected were removed from the analysis. Epochs containing local artifacts on otherwise normal channels were removed from the analysis as well. Raw, continuous data were down-sampled to 1,000 Hz and filtered with a 60-Hz notch filter as described previously (77). The ECoG data were then rereferenced to a common average reference (defined as the mean of the remaining channels). Single channels of this ECoG data are referred to as “raw signal.”
The analytic amplitude (or power) of HFB was extracted from the raw signal by using a frequency-domain half-max, full-width Gaussian filter along with a Hilbert transform (as in ref. 27). The time course of the HFB power was then smoothed by using a Hanning window (50 samples), segmented time-locked to stimulus (between −1,000 and 2,000 ms around stimulus onset) and vocal onset (between −1,500 and 500 ms around vocal onset), and normalized to baseline power (stimulus-locked baseline, −1,000 to −500 ms before stimulus onset; response-locked baseline, −1,500 to −1,000 ms before vocal onset; resulting unit of HFB power in percent change from baseline) for all correct artifact-free trials. We tested whether an electrode had significant HFB by comparing the HFB power in each trial to zero by using one-sided Student t tests assuming unequal variance on consecutive 50-ms-long time windows between 0 and 1,000 ms time-locked to the stimulus and between −750 and 250 ms around vocal onset. The rate of type I errors in null-hypothesis testing was controlled for by calculating the false discovery rate (FDR) on the resulting P values. An electrode was considered “active” if it had at least one 50-ms-long segment that had significant HFB power after FDR correction (SI Appendix, Fig. S3).
To test for the time course of experimental effects, we averaged the HFB power in each trial over one to five 200-ms-long consecutive time windows for each active electrode time-locked to the stimulus and to the response (Figs. 2–4). The number of time windows included in the analysis for each electrode was determined by whether this electrode had significant HFB in the specific time window as determined by the prior HFB significance testing. We used the same time windows in each trial for a given electrode. We then ran mixed-effect models on within-trial mean HFB as the dependent variable controlling for the time window (windows 1–5) and structure, as well as the same parameters as for the behavioral data. We ran separate models for each cerebral lobe of interest (i.e., frontal and temporal) and tested for fixed effects of semantic context (HOM vs. HET), presentation number (two to four; the first presentation was removed from the analysis of the ECoG data similarly as for the behavioral data), window (windows 1–5), structure, and stimulus position (i.e., left or right of the fixation cross) as within-subject factors, and the interactions between semantic context and presentation number, as well as between semantic context, window, and structure. As random effects, we had intercepts for picture name and participant, as well as by-participant random slopes for the fixed effects of interest (i.e., semantic context, window, their interaction, and presentation number#). We could not control for structure in the random slopes because not every participant had electrodes in each structure. However, the fixed effects involving structure were present in a majority of patients (SI Appendix, Fig. S6). P values were obtained similarly as for the behavioral analyses. For illustrative purposes (Fig. 2), the same models were also run per electrode stimulus and response-locked.
Acknowledgments
We thank Matar Haller, Avgusta Shestyuk, and Adeen Flinker for technical input; James Lubell and Callum Dewar for creating subject reconstructions; all members of the R.T.K. laboratory for their help with data recording; and the research volunteers who took part in this study. This research was supported by a postdoctoral grant from the National Institute on Deafness and Other Communication Disorders of the National Institutes of Health (NIH) under Award F32DC013245 (to S.K.R.); NIH Grants EB00856, EB006356, and EB018783 (to G.S.); US Army Research Office Grants W911NF-08-1-0216, W911NF-12-1-0109, and W911NF-14-1-0440 (to G.S.); National Institute of Neurological Disorders and Stroke Grants R01NS078396 (to J.P.), 2R37NS21135 (to R.T.K.), and NS40596 (to N.E.C.); National Science Foundation Grant BCS1358907 (to J.P.); US Department of Veterans Affairs Clinical Sciences Research and Development Program Grants 10F-RCS-006 and CX000254 (to N.F.D.); the Nielsen Corporation (R.T.K.); and Fondazione Neurone (G.S.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, the Department of Veterans Affairs, or the United States government.
Footnotes
- ↵1To whom correspondence should be addressed. Email: sries{at}sdsu.edu.
Author contributions: S.K.R., N.F.D., and R.T.K. designed research; S.K.R., D.K.-S., K.D.L., P.B.W., R.A.K., K.I.A., P.B., G.S., J.J.L., J.P., and N.E.C. performed research; S.K.R., A.C., P.B., and R.T.K. contributed new reagents/analytic tools; S.K.R. and R.K.D. analyzed data; and S.K.R., A.C., P.B., G.S., N.F.D., and R.T.K. wrote the paper.
The authors declare no conflict of interest.
This article is a PNAS Direct Submission.
↵*Anomia is a severe word retrieval deficit observed in all aphasic patients as well as in neurodegenerative diseases and normal aging (2).
↵†The individual RTs were log-transformed to reduce skewness and approach a normal distribution.
↵‡Because most scalp EEG studies that use this paradigm have focused on event-related potentials, we also conducted an analysis of the intracranial ERPs recorded across ECoG recording sites. Several studies have shown that ERPs described at the scalp surface are often associated with more than one cortical generator (e.g., refs. 38⇓⇓–41). In addition, intracranial ERPs are found at recording sites that do not necessarily overlap with those at which HG is recorded (41, 42). In the present study, this was also the case: There was only approximately 40% overlap in the sites showing HG and those showing ERPs. In addition, almost no significant semantic context effects were found in the ERP analysis (SI Appendix provides more details).
↵§Four structures per lobe were defined: In the frontal cortex, lateral PFC, medial PFC, lateral M1/PMC, and medial M1/PMC; and in the temporal cortex, lateral STG, MTG, ITG, and ventral temporal cortex.
↵¶There was a reversal of polarity from semantic interference to semantic priming in four of five patients in the lateral M1/PMC (such consistency was not observed for the other structures showing an increase in semantic priming with time; SI Appendix, Fig. S6). The initial interference effect may reflect an early preparation signal from this structure to the other regions subsequently involved in word selection, although further research is needed to determine the functional significance of this effect.
↵#We could not include a random slope for the interaction between presentation number and semantic context as the models would not converge with this level of complexity. However, no interaction between semantic context and presentation number was found in the fixed effects for any of the models.
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1620669114/-/DCSupplemental.
References
- ↵
- ↵.
- Goodglass H
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵.
- Mahon BZ,
- Costa A,
- Peterson R,
- Vargas KA,
- Caramazza A
- ↵
- ↵
- ↵.
- Roelofs A,
- Piai V
- ↵
- ↵
- ↵
- ↵.
- Schnur TT, et al.
- ↵
- ↵.
- Ries SK,
- Greenhouse I,
- Dronkers NF,
- Haaland KY,
- Knight RT
- ↵.
- Janssen N,
- Hernández-Cabrera JA,
- van der Meij M,
- Barber HA
- ↵
- ↵
- ↵
- ↵
- ↵.
- Trebuchon-Da Fonseca A, et al.
- ↵
- ↵
- ↵
- ↵.
- Flinker A, et al.
- ↵
- ↵.
- Hamamé CM,
- Alario F-X,
- Llorens A,
- Liégeois-Chauvel C,
- Trébuchon-Da Fonseca A
- ↵.
- Llorens A, et al.
- ↵.
- Piai V, et al.
- ↵
- ↵
- ↵
- ↵
- ↵.
- Flinker A, et al.
- ↵
- ↵
- ↵
- ↵
- ↵.
- Kam JWY, et al.
- ↵
- ↵
- ↵
- ↵
- ↵.
- Binder JR,
- Desai RH,
- Graves WW,
- Conant LL
- ↵
- ↵
- ↵
- ↵.
- Strijkers K,
- Costa A
- ↵
- ↵.
- Matyas F, et al.
- ↵
- ↵.
- Clarke A,
- Taylor KI,
- Devereux B,
- Randall B,
- Tyler LK
- ↵
- ↵.
- Thompson-Schill SL,
- D’Esposito M,
- Aguirre GK,
- Farah MJ
- ↵.
- Fargier R,
- Laganaro M
- ↵.
- Howard MW, et al.
- ↵.
- Belke E,
- Stielow A
- ↵
- ↵
- ↵.
- Piai V,
- Riès SK,
- Swick D
- ↵.
- Riès SK,
- Karzmark CR,
- Navarrete E,
- Knight RT,
- Dronkers NF
- ↵
- ↵
- ↵.
- Bonin P,
- Peereman R,
- Malardier N,
- Méot A,
- Chalard M
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵.
- R Core Team
- ↵.
- Bates D,
- Mächler M,
- Bolker B,
- Walker S
- ↵.
- Fox J,
- Weisberg S
- ↵
- ↵
- ↵
Citation Manager Formats
More Articles of This Classification
Related Content
Cited by...
- No citing articles found.