Imaging bilinguals: When the neurosciences meet the language sciences

The Rodriguez-Fornells, Rotte, Heinze, Nösselt and Münte (2002) paper published in Nature, “Brain potential and functional MRI evidence for how to handle two languages with one brain”, is discussed by two of its authors, both neuroscientists, and by two language scientists. First, a short summary of the paper is given. This is followed by a critical commentary offered by the language scientists. The neuroscientists respond, and a final comment is offered by the language scientists. The four authors conclude that a two-way collaboration between neurosciences and language sciences should be encouraged in order to make headway in our understanding of language processing and representation in bilinguals.

Bilingual speakers of Spanish and Catalan, with high proficiency in both languages, and monolingual Spanish subjects served as volunteers. In the main ERP and fMRI experiments, subjects were shown a series of stimuli appearing one at a time in the middle of a video-screen. Stimulus lists comprised high and low frequency Spanish and Catalan words as well as pseudo-words, which were derived from either Spanish or Catalan words by changing one or several letters. Care was taken to exclude cognate words, which are very similar or identical in the two languages, from the stimulus material. Participants were instructed to press a button for Spanish words only and to withhold response for either Catalan or pseudo-words. Brain potentials were recorded from 32 scalp channels. The N400 component in the ERP was examined. Words from the target language (Spanish) showed a modulation of the N400 response (Kutas, Federmeier, Coulson, King and Münte, 2000) as a function of word frequency in both bilingual and monolingual subject groups, while the brain potentials to the Catalan words did not show a frequency dependent modulation of the N400. In a control experiment, performed on a smaller number of bilingual subjects, the task was changed such that now the Catalan words had to be responded to, while Spanish and pseudo-words had to be ignored. This control experiment indicated that a modulation of the N400 to the Catalan words was now present, while no such effect was seen for the Spanish words. In a further control experiment we showed that these effects were independent of the requirement to respond. The lack of an N400 modulation for words from the non-target language in the bilingual subjects was taken to indicate that the meaning of these words had not been accessed by the bilinguals.
Event-related fMRI was performed using the same task as in the main ERP experiment, i.e. with Spanish words serving as a target, but introducing consonant strings (for example, "dfmvr") as an additional stimulus category. A first important finding was that neither the monolingual nor the bilingual group showed reliable differences between the activation pattern of pseudowords and Catalan words. These stimuli were apparently treated very similarly by the two subject groups, thus corroborating the interpretation of ERP results, i.e. that Catalan words were in general not processed for meaning. Critically, only bilingual subjects showed activation of the left posterior inferior frontal area and the planum temporale, i.e. regions that have previously been found in experiments employing pseudo-word reading, phonological processing, and subvocal rehearsal (e.g. Petersen, Fox, Posner, Mintun and Raichle, 1989;Zatorre, Evans, Meyer and Gjedde, 1992;Paulesu, Frith and Frackowiak, 1993). This brain activation pattern, together with the N400 data, suggested to us that bilinguals prevent interference by using the brain and cognitive machinery normally reserved for the reading of unknown or pseudo-words, i.e. the sublexical pathway (Coltheart, Curtis, Atkins and Haller, 1993), while at the same time inhibiting the direct access route from orthography to the lexicon. This interpretation was also supported by greater activation of an anterior prefrontal region in bilinguals, which is generally viewed as supporting inhibition (e.g. Bunge, Ochsner, Desmond, Glover and Gabrieli, 2001).

Commentary (F. Grosjean and P. Li)
"Two languages with one brain" is a fascinating topic that has naturally attracted the attention of neuroscientists who have access to the latest neuroimaging technologies. More than three-dozen "imaging bilinguals" articles have been published, including the one by Rodriguez-Fornells et al. (2002;henceforth RF), which we discuss here. In what follows, we argue that the authors do not take into account crucial factors in bilingualism research and that they fail to interpret their data in terms of current theories of bilingual processing.
RF state that their monolingual speakers of Spanish and their Spanish/Catalan bilinguals were foreign students at two German universities. If this was the case, didn't both groups also know and use German and weren't they therefore bilingual and trilingual? What impact did this have on the results obtained? Such questions lead to the issue of what is meant by bilingual. In the language sciences, bilingualism is increasingly defined in terms of regular use of two or more languages (Grosjean, 1994) and it does not necessarily imply equal proficiency in the languages known (as RF's study seems to imply). In addition, it is well established that language history, language stability, the functions of each language, along with language mixing habits, all have an impact on processing results. The probable diversity of the subjects used by RF is further confounded by the unequal number of bilinguals in each experiment (15 in the main ERP study but only 4 in the first control!) and by the fact that some participated in several experiments and saw some of the same stimuli. In short, the RF results may be specific to the subjects used and may not be replicable with other bilinguals.
RF's starting point is that "bilingual individuals need effective mechanisms to prevent interference from one language while processing material in the other". This rather monolingual view of the bilingual fending off the other languages has been replaced by a much more dynamic view of bilingual language processing based on the language mode concept (Grosjean, 1998(Grosjean, , 2001Marian and Spivey, 2003). In some situations the bilingual must indeed only process one language (the mode is close to being monolingual) but in others, several languages are processed on-line with one taking the lead role (as in the case of mixed language where the base language is more active than the guest language; Li, 1996). The bilinguals in RF's main ERP experiment were not in a monolingual mode: they had activated their Spanish lexicon to a greater extent but they were still processing Catalan words despite being asked to respond to Spanish words only. Bilinguals made more errors (i.e. false-positive responses) to high frequency Catalan words and were generally delayed in preparing a motor response compared to monolinguals, as is evidenced by the lateralised readiness potentials (LRPs).
Why then were words from the non-target language "rejected" by the bilinguals? Probably not because they used a sublexical access route to the lexicon, as the authors speculate (there is no evidence in the literature that the lexical access mechanisms are any different in bilinguals and monolinguals), but for other reasons. First, in the main ERP and MRI experiments, since the task was to respond to Spanish words, it is possible that the Spanish lexicon was more active and the Catalan lexicon less so. This would help ensure response to Spanish words. Second, there were probably some graphemic cues specific to Catalan words that would exclude the latter from the process (e.g. the grave accent, letters such as "ç" and "x", sequences such as "l.l", "ny", "ix", "ss", "tx", "tge", "lts" etc.). Third, the high-frequency Spanish words may have got an extra boost by being more frequent than their counterparts in Catalan (95 vs. 68.4 occurrences per million). These, and other reasons (e.g. the varying proportion of words from the two languages in the experiments), would speak less in favor of "rejection" than of reduced activation of the words in the non-target language. These words did not reach the required activation threshold and hence were usually not responded to (Dijkstra and van Heuven, 1998).
RF end their paper with a statement that the generality of their findings should be tested with other experimental tasks. We can only concur with this, for the reasons given above, but also because the findings in bilingualism research and in brain imaging studies are often taskspecific (Joseph, 2001).
The gap between the neurosciences and the language sciences of bilingualism will be narrowed if both sides define and choose their bilinguals with care, use carefully selected stimuli, control for language mode, employ tasks that tap into normal language processing, and build together coherent theories of bilingual language representation and processing.

Response (T. F. Münte and A. Rodriguez-Fornells)
The comments by Grosjean and Li (henceforth GL) can be divided into those pertaining specifically to our experiment and those that have a more general character. In the following, we will briefly address the specific issues and then turn to the more important general issues.

Comments on specific issues
We concur with GL that our Spanish/Catalan subjects with high proficiency in two languages represent a rather extreme case of bilingualism. While we can see that other studies with different aims might call for different subject groups, we still view Spanish/Catalan bilinguals to be ideal for our purpose, as we were interested in the mechanisms allowing bilinguals to preferentially process one language while suppressing the other. A high level of proficiency in both languages is needed in such a study, and in Spanish/Catalan subjects this proficiency is guaranteed by the educational policies in Catalunya. This has led to the use of these subjects in a great number of studies on bilingualism (e.g. Pallier, Sebastian-Galles, Dupoux, Christophe and Mehler, 1998;Perani et al., 1998;Sebastian-Galles and Soto-Faraco, 1999;Costa, Caramazza and Sebastian-Galles, 2000;Pallier, Colome and Sebastian-Galles, 2001). Moreover, our subjects were assessed for current language habits by a questionnaire adapted from Weber-Fox and Neville (1997), which indicated regular use, as well as high proficiency, of both languages. We thus do not see, how a "probable diversity" of the subjects could be responsible for our results.
GL also point out that the different number of subjects in the main and in the control ERP experiments might be problematic. Statistical power is not an issue here, however, as the control experiment has demonstrated the ability of subjects to switch between languages, and the ERP pattern can be reversed as a function of the instructions.
Furthermore, GL -with regard to the lateralized readiness potential (LRP) -remark that our Spanish/Catalan subjects were not in a monolingual mode. In fact, however, the LRP results of our study show NO LRP ACTIVITY for the words from the non-target language. This suggests that these words were effectively rejected. This view is supported by the findings for the N400 component  not mentioned by GL in their comment.
In the bilingual as well as in the monolingual subjects there was no N400 modulation for Catalan words in the main experiment, which suggests that Catalan words were NOT processed for meaning by the bilinguals. In addition, the first control experiment showed that bilinguals can effectively switch their strategy according to instructions and that at that point Spanish words were not processed for meaning.
This selective processing of Spanish or Catalan words in the bilinguals was interpreted by us in light of the brain activation patterns in bilinguals, which, as pointed out in our summary, were reminiscent of activations seen in experiments using pseudo-word reading. As these, by necessity, engage the phonological route, they suggested to us that bilinguals might use this route in order to block out the information from the non-target language. In their comment, GL disregard these results, however. By contrast, we believe that brain activation patterns can be highly informative, because activations can be compared across multiple studies and tasks, as was done in our paper.
In any biological or psychological experiment, a particular limited phenomenon is studied under particular limited conditions. Our experiment suggests how certain bilingual subjects behave in a certain situation (reading of mixed word lists with one language relevant). Other mechanisms might help bilinguals to keep their languages separate in other situations. Thus, our experiment is limited like virtually every other brain imaging and psycholinguistic experiment. In several further studies, we have therefore extended our work to test the monolingual vs. bilingual mode during comprehension (Rodriguez-Fornells, Corral, Escera and Münte, in preparation) as well as bilingual, Spanish/German, production in a picturenaming task (Rodriguez-Fornells, Britti and Münte, in revision).

Comments on general issues
In their comment, GL endorse a collaboration between the language sciences and the neurosciences in the study of bilingualism. We could not agree more but we would like to point out that such a collaboration should not be a one-way street with neuroscientists proving theories devised by language scientists. That such an approach falls significantly short of the possibilities of such a collaboration can be illustrated by the following recent example. A heated debate in psycholinguistics concerns the representation and processing of regular and irregular verb forms (Marcus, Brinkmann, Clahsen, Wiese and Pinker, 1995;Marchman, Plunkett and Goodman, 1997;Pinker, 1997;Clahsen, 1999). Some theorists have advocated single mechanism models that represent and process both classes of verbs within a single system, while other researchers have proposed dual mechanism models with separate paths for regular and irregular verbs. Pinker (1997) has gone so far as to call the regular and irregular formation of verb forms the "fruit fly of linguistics". Several research laboratories including our own (e.g. Penke, Weyerts, Gross, Zander, Münte and Clahsen, 1997;Münte, Say, Clahsen, Schiltz and Kutas, 1999;Rodriguez-Fornells, Clahsen, Lleo, Zaake and Münte, 2001) have collected ERP and brain imaging data on regular and irregular word processing, which have been used by psycholinguists in support of single (Seidenberg and Hoeffner, 1997) and dual mechanism models (Clahsen, 1999) of morphological processing. However, as we have pointed out elsewhere (Münte, Rodriguez-Fornells and Kutas, 1999) the neuroscientific data on the matter suggest that NEITHER a single NOR a dual mechanism model appears to be entirely appropriate. For example, PET (positron emission tomographic) studies by Jaeger, Lockwood, Kemmerer, Van Valin, Murphy and Khalak (1996), and Indefrey, Brown, Hagoort, Herzog, Sach and Seitz (1997) have revealed that multiple (i.e. more than 10) brain areas distinguish between the processing of regular and irregular verbs. This, in turn, suggests that both classes of psycholinguistic models might give an incomplete picture about what computations are necessary to handle these different types of verbs. We have therefore proposed that the brain activation patterns seen in fMRI or PET as well as the modulations of the ERPs might be used to guide the development of more realistic psycholinguistic models. In the same way, of course, neuroscientific data, like the ones in our own study, might be used to stimulate and constrain psycholinguistic models addressing language processing in bilinguals, while these models in turn should be used to devise appropriate experiments. This will, we believe, eventually lead to a more fruitful collaboration between psycholinguists and neuroscientists.
On a more practical note, first-time (psycholinguistic) users and consumers of neuroimaging or electrophysiological techniques may find that their experimental possibilities are limited by methodological constraints, e.g. the necessity to have many trials per category or the problem of artifacts produced by vocalizations. These drawbacks are offset, in our opinion, by the fact that these techniques can deliver multidimensional spatiotemporal data on the timing, localization and parceling of cognitive processes underlying bilingual language processing. Moreover, they can even deliver data on stimuli that do not require overt responses. They can thus be viewed as a useful extension rather than a replacement of more traditional experimentation in psycholinguistics.
To conclude, while naturally we do not agree with most of the criticisms raised by GL, we welcome very warmly their proposal for a more fruitful collaboration between psycholinguists and neuroscientists.

Reply to response (P. Li and F. Grosjean)
Münte and Rodriguez-Fornells (henceforth MRF) provide us with a rather detailed response to our commentary. Although it contains many important points, we are not sure that MRF address the main issues we made in our commentary. Below, we first list the concerns we raised for which we do not see a response, and next we discuss MRF's comments on the other points we made.
There are a number of concerns for which we do not see a response. First, there is the fact that the monolingual speakers of Spanish were NOT in fact monolingual (they were probably bilingual) and that the Spanish/ Catalan bilinguals were probably trilingual. It should be remembered that they were all foreign students in Germany at the time of the study and hence Germanas a second language for the first group and as a third language for the second -could have played some role in the results obtained. Second, there is the fact that some of the subjects participated both in the main study and in the control study. Hence, we have no guarantee that the subjects' first experimental run did not influence the second (e.g. they may have remembered some items). Third, the bilinguals in the main ERP experiment were not in a monolingual mode and hence it is no surprise that the non-target-language was showing some activity. This is apparent in the LRP onset latencies and in the errors made. MRF do not respond specifically to the language mode issue or to the latency and error data comment; they do, however, address the LRP activity issue to which we will return below. Fourth, we proposed that there were at least three bases for "rejecting" non-target-language words: a more active target language lexicon, graphemic cues to the non-target language that helped to exclude it, and the higher frequency of some items in the target language. MRF do not address these factors. We believe that all of these concerns are important and that they might have influenced the results obtained. 1 As for MRF's comments on the other points we made, we should first state that we did not question that their 1 A further possibility is that rejection occurs at a rather late stage and reflects a decision process. Von Studnitz and Green (2002) showed that reduction in interference can arise without reducing the lexical activation of the non-target language. bilingual subjects were highly proficient in Spanish and Catalan or that these bilinguals were ideal for their study. Hence we will not discuss these two aspects but rather we will focus on MRF's other points (presented in italics below).
1) The high proficiency of the bilinguals does away with the diversity criticism. MRF appear to use the argument that their bilinguals were highly proficient in Spanish and Catalan to disagree with the fact that bilingual diversity could have had an impact on their results. However, language proficiency is just one factor in defining the diversity of bilinguals: others include language history, language stability, the functions of each language, and language mixing habits. These are well-accepted factors among researchers of bilingualism and have been shown to affect processing (see, for example, Grosjean, 1998); they may well have had an impact on the Rodriguez-Fornells et al. results.
2) The small number of subjects in the first control study is not a problem. MRF believe that four subjects are sufficient for such a study. We have doubts, as would most cognitive scientists working with subject populations and using inferential statistics.
3) The LRP results show no activity of the non-target language. In our commentary we argue that the non-target language (Catalan) was still active, though to a lesser extent. This was clear from the LRP onset latencies and the higher error rate to high frequency Catalan words. In fact, in their Nature paper, the authors acknowledge this when they write (p. 1027): "[the bilinguals] had some difficulty suppressing button presses to high-frequency irrelevant words". In their response, MRF do not address our concern but point out that (a) there was no LRP activity for Catalan words in the bilinguals, and (b) there was no ERP N400 modulation to Catalan words in both monolinguals and bilinguals. With regard to (a), we believe that there is a difference between no LRP activity and the inactivity of a language. LRP indicates the preparation of motor responses only, as is pointed out by Rodriguez-Fornells et al. In addition, there was a marked difference between monolinguals and bilinguals in terms of the amplitude and the speed to the target language. This clearly indicates that the bilinguals did not prepare their responses to the target language as effectively, due probably to the partial activation of the non-target language. With regard to (b), we note two important things. First, the authors use the difference between high and low frequency words in N400 as a measure of meaning access. (It should be recalled that according to Kutas and Hillyard (1980), N400 is an ERP component that detects semantic violations or incongruity in sentence processing.) It is a big step to go from the presence or absence of a frequency effect to the presence or absence of meaning access; the interplay of the two is not as direct as the authors seem to suggest. Second, there is a major difference between monolinguals and bilinguals. Monolinguals have no N400 to either type of word (high or low frequency), while bilinguals have N400 to both. Moreover, in the control experiment with four subjects, the ERP patterns were of two sorts: for Spanish words, they were similar to those of the bilinguals in the main experiment; for Catalan words, they were similar to those of the monolinguals in the main experiment. Thus, there were general differences in ERP and N400 patterns between the monolinguals and the bilinguals that the authors did not discuss, and these differences could undermine the authors' interpretation of the general difference between the two groups in terms of dual-route access.
4) The bilinguals might be using the phonological route to block out the information from the non-target language. MRF's assignment of a "lexical" route to monolinguals and a "sublexical" route to bilinguals seems to be at odds with most known theories and results in monolingual and bilingual language processing studies. For example, research by Perfetti and colleagues suggests that all monolinguals, even in phonologically non-transparent languages, use "sublexical" routes to access the mental lexicon (Perfetti, Bell and Delaney, 1988;Tan and Perfetti, 1998). We believe that a better explanation of Rodriguez-Fornells et al.'s results should be based on factors pointed out earlier, such as which lexicon was more active and the existence of graphemic cues specific to words of one language.
In conclusion, MRF point out that, "collaboration between the language sciences and the neurosciences . . . should not be a one-way street with neuroscientists proving theories devised by language scientists". Our intention in starting this dialogue with our colleagues is precisely to bridge the gap between the neurosciences and language sciences. Thus, we too are advocating a two-way collaboration (cf. the "if both sides define" paragraph in our commentary). We know that this is a view shared by an increasing number of neuroscientists and language scientists (see Vaid and Hull, 2002, for a review of the field as well as the special issue of Bilingualism: Language and Cognition, edited by David Green, 2001).