Neurophysiological Mechanisms of Word Meaning Acquisition

Размещено на http://www.allbest.ru/

The Government of the Russian Federation

Federal State Autonomous Educational Institution of Higher Professional Education

National Research University - Higher School of Economics

Institute of Cognitive Neuroscience

Master's program

“Cognitive sciences and technologies: from neuron to cognition”

Final qualifying work

Neurophysiological Mechanisms of Word Meaning Acquisition

Student group № 171

Razorenova, Alexandra Mikhailovna

Scientific adviser

associate professor, cand.sci.

Chernyshev, B.V.

Moscow, 2019



Introduction

word language speech

Speech is a unique phenomenon observed only in humans. Moreover, word-meaning acquisition is considered one of the crucial features of speech as a high-level cognitive function. The goal of the current study was to reveal speech perception and language acquisition mechanisms in the human brain, specifically at one-word level, using reach possibilities of modern technologies such as magnetoencephalography (MEG).

Vocabulary of any human language encloses enormous quantity of information, making it a non-trivial challenge for science to describe speech mechanisms established within the human brain - since the size of these databases exceeds any limits estimated in terms of modern theories of memory and learning. This challenge may be met in the frame of embodied cognition and associative learning. Associative learning paradigm implies that the word referential meaning is implemented in the brain via associative Hebbian-type learning. Supposedly, word-specific memory traces in the brain can be formed by way of mutual connection strengthening between different areas - as actions, objects or concepts are learnt when they are experienced in conjunction with the words used to describe them (Pulvermьller, 2005). A growing body of experimental research testifies to motor and sensory involvement in speech processing and generation (D'Ausilio et al., 2009; Hauk, Shtyrov, & Pulvermьller, 2008; Pulvermьller, 2005; Y. Shtyrov, Butorina, Nikolaeva, & Stroganova, 2014).

Neuroimaging experiments involving word learning may bring new insights into our understanding of word processing in the brain. The ability to quickly acquire word-picture associations was shown to depend on reorganization in neocortical networks including the left temporal area, especially the left temporal pole (Sharon, Moscovitch, & Gilboa, 2011), as well as temporoparietal, premotor and prefrontal regions (Steve Majerus et al., 2005; Mestres-Missй, Rodriguez-Fornells, & Mьnte, 2007; Paulesu et al., 2009; Sharon et al., 2011).

However, it should be noted that any word may be considered within two domains: phonological and semantic. Correspondingly, there are two groups of studies dedicated to novel word learning: some of them deal with phonological familiarization to pseudowords, while other studies investigate mechanisms of word meaning acquisition via passive associative learning that adds semantics to pseudowords. Still there is no consistent understanding how actually semantic association influence on pseudoword learning and their integration into lexical domain. The current study aimed to clarify the influence of semantics on pseudoword processing. We used a novel lexical trial-and-error learning paradigm to establish associations between pseudowords and actions. We controlled familiarization effect and inquired when and where processes associated with semantics take place in the human brain. MEG neuroimaging technique was used as it provides high spatial and temporal resolution.

Participants were presented with eight pseudowords; during learning blocks, four of them were assigned to specific body part movements through commencing actions by one of participants' left or right extremities and receiving a feedback. The other pseudowords did not require actions and were used as controls. Magnetoencephalogram was recorded during passive listening to the pseudowords before and after learning blocks. The cortical sources of the magnetic evoked responses were reconstructed using distributed source modeling.

All of the 24 participants reached successful performance on the task. Phase-locked neural response selectively increased for pseudowords that acquired association compared with control pseudowords. Using data-driven approach, we localized significant differential activation into the left hemisphere, including insula, Broca's complex, intraparietal sulcus and anterior superior temporal sulcus and anterior middle temporal gyrus (aSTS-aMTG). Differential activation started 150 ms after the uniqueness point. These areas can be viewed as both low-tier (aSTS-aMTG), and higher-tier (intraparietal sulcus, temporal pole; triangular gyrus) structures involved in speech processing.



Chapter 1. Literature Review

Words are essential structural elements of any human language. Recognition of spoken words requires that the brain has invariant Gestalt-like representations of complex auditory patterns for each known word ( Griffiths, Warren, 2004; Leaver & Rauschecker, 2010). Yet words belong to a lexicon, and thus lexicality normally requires that words have meaning. Semantics is a basic linguistic feature that allows us to use language as a carrier of information transfer: apparently, words having no meaning would be useless for such a purpose. The Distributed Cohort Model (Gaskell & Marslen-Wilson, 1997) separates the phonological form and the meaning of spoken words. Particularly, detection of a phonological form involves recognition of complex spectrotemporal auditory patterns, i.e. phoneme concatenations (DeWitt & Rauschecker, 2012), while assigning meaning presupposes mapping between phonological forms and representations of objects, events, etc. Although these two aspects of lexicality appear to be tightly interconnected, they may have different neurophysiological and neuroanatomical underpinnings - possibly in parallel with current accounts of two auditory language processing pathways, dorsal and ventral (Dick, Bernal, & Tremblay, 2013; Hickok & Poeppel, 2015; Katzev, Tuscher, Hennig, Weiller, & Kaller, 2013). In order to properly understand word learning both at behavioral and neurophysiological levels, it is essential to keep in mind such a dual nature of word processing. Neuroimaging experiments involving word learning may bring new insights into our understanding of word processing in the brain if they specifically address the existence of these two aspects.

There is well-substantiated point of view that a consolidation period is essential for word learning (Davis & Gaskell, 2009; Gaskell & Dumay, 2003). Yet, there is a significant body of studies evidencing in favor of a `fast mapping' mechanism that makes rapid word learning possible (Borovsky, Kutas, & Elman, 2010; Mestres-Missй, Cаmara, Rodriguez-Fornells, Rotte, & Mьnte, 2008; Sharon et al., 2011; Y. Shtyrov, Nikulin, & Pulvermuller, 2010). Mechanisms subserving word processing immediately after learning and after a consolidation period may be somewhat different, but the difference may be primarily related to the level of word abstraction and its integration with other words: while `fast mapping' may provide isolated concrete word representations, consolidation involving an overnight sleep may be required to integrate learnt words into the language domain - i.e. to integrate with other words of the mental lexicon (Davis & Gaskell, 2009; Gaskell & Dumay, 2003). Since `fast mapping' may be considered an initial stage of word acquisition, which possesses most critical aspects of word learning, many EEG/MEG studies in the area dealt with `fast mapping', and we will mostly consider effects observed within the `fast mapping' approach.

Following a distinction between phonological and semantic aspects of lexicality, studies of word learning can be divided into two groups - those that focused on the phonological aspect, and those that additionally accounted for the semantic aspect of lexicality.

1.1 Phonological word-form learning

A number of studies focused specifically on the phonological aspect of lexicality led by assumptions that the adult brain can learn novel pseudowords as `empty' lexical entries bearing no meaning. Another assumption inherent in these studies is that such word learning can be achieved in the course of passive listening to repeated presentations of pseudowords, without any effort on the part of the participants. Typically, such studies of word learning were aiming to investigate the difference between brain processing of real words and pseudowords.

Several EEG and MEG studies involving passive listening to real words and novel pseudowords demonstrated that the initial ERP responses to real words were stronger than to pseudowords (Kimppa, Kujala, Leminen, Vainio, & Shtyrov, 2015; MacGregor, Pulvermьller, Van Casteren, & Shtyrov, 2012; Shtyrov, 2011). This finding can have at least three complementary explanations: (1) a real word has a well-learned phonological representation (phoneme concatenation stored as a word form), thus presentation of a real word activates appropriate phonological detectors in the brain, while the brain has no such detectors for novel pseudowords; (2) a real word produces greater brain response because it is associated with semantics, thus it's presentation evokes activation in brain areas dealing with semantics - which is not the case for novel pseudowords; and (3) real words may have greater salience - because they are familiar, and because they have certain meaning that may bear certain importance for the individual, while novel pseudowords are presumed to be non-familiar and indifferent and thus non-salient.

The opposite effect was observed when real words and pseudowords were presented repeatedly: Baart and Samuel (2015) reported stronger ERP response to pseudowords compared to words, though they did not analyze temporal dynamics of brain responses in the course of repetitive stimulus presentation.

1.1.1 Repeated presentations of pseudowords and real words

Several EEG studies went further and inquired into the dynamics of brain responses in the course of repeated presentations of pseudowords and real words (Kimppa et al., 2015; Shtyrov, 2011) Both studies reported that the response to pseudowords increased by the end of the session compared with the beginning of the session, while the response to real words either decreased or did not change. Thus, after repeated presentations of pseudowords and real words, responses to pseudowords become similar in amplitude to responses to real words. Authors explained this effect as rapid formation of a neural memory trace for pseudowords.

1.1.2 Oddball paradigm and MMN responses on words and pseudowords

The oddball paradigm involves repeated stimuli presentations, and the studies aiming to reveal contrasts between word and pseudoword processing with the use of the oddball paradigm may be considered in the same vein as ERP studies discussed above (see 1.1.1). In a study by Shtyrov et al. (2010), after repetitive presentations, pseudowords used as rare deviants evoked responses of increased amplitude, while no such dynamics was observed for evoked responses for real words presented as rare deviants. This effect was used as an evidence that a new phonological combination had been learnt to be processed in a way similar to real words. Similar results were later reported by Yue, Bastiaanse, and Alter (2014); this study exactly reproduced the procedure used by Shtyrov et al. (2010), but used completely different stimuli.

The main finding from these studies was that the response to novel pseudowords increased after passive repetitive presentation. This can be explained by formation of a respective phonological template in memory.

It should be noted that both timing of the effects of word learning discussed above (around 100-120 ms after the disambiguation point) and localization (to low-tier perisylvian areas involved in auditory and phonological analysis, as well to articulatory areas possibly involved in phonological loops) is compatible with the explanation that the effects found in these studies were sensory/perceptual in nature, and thus were related exclusively to the phonological aspect of lexicalization (DeWitt & Rauschecker, 2012), while they supposedly did not involve the semantic aspect.

An additional clarification to the studies reviewed above comes from the study by (Jacobsen et al., 2004). While the mismatch negativity (MMN) studies mentioned above used two conditions (pseudowords as deviants vs. real words and vise-versa), Jacobsen et al. (2004) used a full design in which all four combinations of pseudowords and words serving as deviants and standards were used. Although this study did not attempt to study the learning dynamics, the effects observed can be largely attributed to the state that follows passive learning, since the number of repetitions was rather large. The principal finding was that MMN was smallest in the condition of pseudowords vs. pseudowords, and biggest in the condition of real words vs. real words, while the two other conditions (pseudowords vs. words and words vs. pseudowords) produced intermediate strength of MMN. MMN amplitude is known to be directly related to perceptual stimulus discriminability (Kompus & Westerhausen, 2018; Naatanen, 2004; Nддtдnen, Paavilainen, Rinne, & Alho, 2007). Apparently, for real words phonological representations are well established through the life experience, while for pseudowords such representation just begin to emerge during the experiment. Having this in mind, the findings of Jacobsen et al. (2004) can be easily explained: well established phonological representations for real words allow for good discriminability (especially in the condition of words vs. words), while this is not the case for novel pseudowords. The same logic can be applied to interpretation of findings by Shtyrov et al. (2010) and by Yue et al. (2014). Indeed, formation of a memory template for the pseudoword may have increased perceptual discriminability and consequently increased MMN amplitude.

Perceptual phonological nature of the MMN effects is further evidenced by the fact that the MMN effect for the pseudoword/word contrast was localized to the left perisylvian neocortex, within superior temporal and inferior frontal areas (Shtyrov et al., 2010), which are classical lower-tier speech areas responsible for auditory phonological processing (DeWitt & Rauschecker, 2012; Hagoort, 2015; Hickok & Poeppel, 2015). The timing of the effect (around 120 ms after the divergence point) also makes involvement of any semantic mechanisms very unlikely.

1.1.3 Results obtained via words and pseudowords contrasting

Repetitive presentations of sensory stimuli are generally known to induce suppression of responses (Grill-Spector, Henson, & Martin, 2006). Indeed, the decrease of responses at least to real words in the course of repetitive presentations could be expected because the brain already possessed phonological templates (phoneme concatenations) for these words, thus, there is no need for any additional auditory processing, and the responses should simply habituate. Yet actually, in the EEG studies mentioned above repetition suppression was weak or absent even for real words, and the authors explain this by stability of brain representations for real word. These findings do not agree with EEG data obtained by (Cheng, Schafer, & Riddel, 2014), which evidenced repetition suppression both for real words and pseudowords. Tentatively the lack of repetition suppression can be explained by some features of experimental conditions such that habituation was slow and the duration of the experiment did not allow to see it develop. If this is true, then the passive procedure is not sufficiently effective in terms of lexicalization, and allows revealing only the initial stages of brain plasticity that might lead to further lexicalization. This leaves open the question whether such word acquisition procedure is sufficient for formation of true word form representations of real words.

In conclusion, the effects obtained in paradigms involving passive repetitive presentations of pseudowords and real words revealed effects that are essentially sensory in nature and emerge as a result of implicit perceptual learning; essentially, these studies address only neural events related to spectrotemporal and phonological analysis. Indeed, MMN paradigm as well as other paradigms involving passive repetitive presentations could have induced memory formation (Naatanen, 2004; Naatanen, Paavilainen, Rinne, & Alho, 2007) and/or sensory adaptation (May & Tiitinen, 2010). At least the former explanation may essentially mean formation of a spectrotemporal and/or phonological template in lower-tier speech areas.

It is important to keep in mind that the EEG studies mentioned above dealt with just one aspect of lexicality, namely with its phonological aspect, since pseudowords were passively presented and were not assigned any meaning. This process can be regarded as familiarization, or, in broader terms, as implicit perceptual learning. In their exhaustive review, Davis & Gaskell, (2009) underline that in order to assess word learning, both phonological familiarization and establishment of semantic association should be accounted for. Only an interaction between familiarization and semantic lexicalization effects can reveal true experience-dependent plasticity during word learning.

Moreover, in the EEG/MEG studies mentioned above, the two conditions - pseudowords and real words - were unbalanced in terms of subjects' experience preceding the experiment - whether a phonological template exists before the experiment, whether stimuli are salient due to familiarity or due to non-indifferent meaning), and whether they attract attention. As a result of such unbalance, familiarization dynamics would be inevitably different, leaving the interpretation of those findings open to discussion.

1.2 Pseudoword learning via semantic association

Another body of EEG studies attempted to overcome the unbalanced perceptual history inherent to the experiments that directly compared responses to pseudowords and real words. These studies made use of the popular theory that semantics is acquired by words in the course of associative learning (Pulvermьller, 1999). In such experiments, no real words were typically used, and the contrast was studied between pseudowords that acquired semantics during some kind of associative learning, and control pseudowords, that were presented the same number of times without any consistent associative pairings. Thus, these studies addressed both phonological and semantic aspects of lexicality. Unfortunately, there were few studies dedicated to associative learning involving auditory presentations of pseudowords, especially those using EEG/MEG recording.

Hawkins, Astle, and Rastl (2015) aimed to check whether providing semantic information about a novel pseudoword would enhance learning a novel phonological pattern. Pseudowords were presented together with two pictures in such a way, that some pseudowords became consistently associated with one of the pictures, while the other pseudowords were inconsistent paired with pictures and thus no associations were acquired. Importantly, participants had to actively discriminate the pictures and to respond whether the picture was a referent to a particular pseudoword or not. Thus, some of the words acquired meaning through active involvement of subjects, while others did not acquire any. After successful learning, these pseudowords were presented as deviants, while real words were used as standards; the experimental paradigm was similar to that used by Shtyrov et al. (2010) Enhanced MMN was observed in response to pseudowords that had previously acquired association with a picture of an object compared with pseudowords lacking any consistent association. This may be evidence that semantics indeed strengthens the stability of the phonological trace and may be crucial for understanding word learning.

A behavioral study conducted by Savill, Ellis, and Jefferies (2017) also speaks in favor of this “semantic binding hypothesis”. In this experiment, novel pseudowords were familiarized with or without associated semantic information (pictures and control blurred images). Pseudowords with semantic associations were reproduced more accurately than those with no meaning, e.g. for semantically-trained pseudowords there were fewer phoneme ordering and identity errors. These data show that lexical-semantic knowledge improves phonological trace when phonological familiarity is taken into account.

Fargier et. al (2012, 2014) conducted a study, in which pseudowords were associated with videos of grasping movements and abstract animated videos. At the first stage of the experiment, pseudowords and videos were presented simultaneously. During subsequent stages, participants were asked to reproduce hand movements seen on the videos and remember word-action/word-vision association. ERPs were compared for pseudowords associated with abstract videos and for concrete movements (Fargier et. al 2014). This neural changes were traced on the second day of learning, thus passive binding seems to induce slow changes in neural response. While words associated with videos of actions evoked enhanced responses after learning, response to words associated with abstract videos diminished. Important that time interval when these contrast was significant was 100-400 ms after stimulus onset. This interval includes both early phonological word-form analyses (P200 component) and semantic analyses component N400.

ERP studies dedicated to word processing distinguish two components N200 and N400. According to Friedrich and Friederici's review (2010), categorization processes are related to N200 component (150-250 ms), while latency of semantic access for words is 300-600 ms after stimulus onset (Hagoort, Brown, & Groothusen, 1993; Halgren et al., 2002; Kutas & Federmeier, 2010). Both components are sensitive to repetitive presentations of words: the decline in amplitude was observed in the course of repetitive presentations (Dittinger, Chobert, Ziegler, & Besson, 2017). As long as semantic may strengthen on phonological word-form representation it is interesting to investigate both early and late effects induced by acquired semantics.

Franзois, Cunillera, Garcia, Laine, & Rodriguez-Fornells (2017) paired pseudowords embedded into artificial speech stream and compared statistical auditory learning with semantically cued learning. In the audio alone condition, the streams were presented with a stable fixation cross on a computer screen. In the audiovisual condition, four different pictures were synchronously presented with the four words of the stream. N400 component for pseudowords with picture association was revealed while the component was absent for pseudowords, which were learnt to be parced via statistical auditory learning. Unfortunately, this experimental paradigm did not allow revealing the contrast between the early phonological components (P200) for two learning conditions.

The four EEG experiments described above took into account both effects of repetition/familiarization and acquisition of semantics. Actually, they were very close to demonstrating the interaction between the two aspects of lexicality, although the design of these experiments was not sufficient to do that.

1.3 Operant conditioning as word learning booster

In the most studies reviewed above, participants were presented with pseudowords or pseudowords and to-be-associated objects sequentially without any active task set before them - or participants were explicitly instructed to remember the associations (Franзois et al., 2017; Savill et al., 2017).

In the study of (Fargier et. al 2014) participants were asked to actively imitate the movements shown on videos, yet the procedure had little in common with operant conditioning. Only in the study of Hawkins, Astle, and Rastl (2015), which procedurally resembled operant conditioning, participants were actively involved in the process of learning.

Importantly, animal data show that passive learning procedures may be of lesser effectiveness in comparable situations. For example, a series of studies by Blake, Strata, Churchland, and Merzenich (2002); Blake, Heiser, Caywood, and Merzenich, (2006) showed that the cortical plasticity effectively occurred in primates only if an active operant conditioning procedure was used rather than simple stimulus-reward pairing. Thus, an active `trial-and-error' learning procedure may be more effective in the induction of cortical plasticity in adults. Remarkably, there is a lack of word learning studies that use any kind of reinforcement (e.g. feedback) in their experimental paradigm.



Chapter 2. Research Proposal

In the current study, we were aiming to investigate into neurocognitive mechanisms of auditory word learning, involving acquisition of word meaning.

Proceeding from the previous studies mentioned above we designed a behavioral paradigm that involves rapid word meaning acquisition based on trial-and-error learning. We relied on `fast mapping', thus all experimental blocks went in immediate succession within a single experimental day.

Our paradigm utilized only novel pseudowords - thus we eliminated the confounding factor of difference in familiarity and salience between real words and pseudowords. In order to control for acoustical difference between the stimuli, the same phonemes were used to build pseudowords, which were counterbalanced across the types of stimuli used.

In line with the findings of animal studies (Blake, Strata, Churchland, & Merzenich, 2002; Blake, Heiser, Caywood, & Merzenich, 2006), we involved an active trial-and-error learning paradigm that was essentially an operant conditioning procedure. Specifically, participants were to associate pseudowords with particular limb movements during the course of learning -- through commencing actions by one of a participant's left or right extremities; positive or negative feedback was presented on each trial.

In order to successfully perform the task, participants have to differentiate each pseudoword from the others - thus all stimuli required attention. Since stimuli were acoustically counter-balanced using cross-splicing of phonemes in various combinations, the task could not be reduced to simple binary classification on targets and non-targets.

Importantly, only half of the novel pseudowords were associated with specific movements, while the other pseudowords were not assigned to any specific movements and thus can be assumed not to acquire any specific semantics. Thus, we used an experimental design that allowed to directly assess the interaction between effects of stimulus repetition and semantic lexicalization according to Davis and Gaskell (2009).

We used MEG recording, which has excellent time resolution equivalent to EEG, while it offers good spatial resolution allowing finding sources of activity on the cortical surface with sufficient reliability.

Using data-driven approach, we attempted to find time and location of significant events in the brain linked to acquisition of word meaning.

We expected that active learning procedure would provide overlearnt word-form representations of pseudowords. Specifically, due to multiple stimuli repetitions we expected repetition suppression effects to all stimuli. We expected that the effect will be observed within perisylvian brain regions.

We also expected that semantics acquired by `words' stimuli type will influence both early (150-200 ms) and late (250-500 ms) components of responses responsible for phonological word-form recognition and semantic access correspondingly. We predicted that the amplitude of ERP calculated for `words' stimuli type will prevail one on `distractors' within both time windows since we expected that semantics would strengthen word-form representation.

We expected to observe sequential activation within low-tier (beginning from early stages of pseudowords processing) and hi-tier brain regions (at later semantic interval).



Chapter 3. Methods

3.1 Participants

Twenty-four right-handed Russian-speaking volunteers (16 male and 8 female adults with no neurological or psychiatric disorders reported) aged 25 years on average (SD = 7.5) took part in the experiment. All participants were right-handed (the Edinburgh Handedness Inventory, Oldfield, 1971). The study was conducted following the ethical principles regarding human experimentation (Helsinki Declaration) and approved by the local ethics committee. All participants signed the informed consent before the experiment.

3.2 Stimuli

Eight two-syllable pseudowords were used; each pseudoword was built of four phonemes in compliance with Russian language phonetics.

Four pseudowords were assigned to actions (see below), these stimuli will be referred to below as 'words'. The other four pseudowords were not assigned to any actions, and they will be referred to below as 'distractors'.

The first two phonemes formed the syllable `hi' that was identical for all pseudowords used. The next two phonemes varied in such a way that each was included in two words and two distractors (Table 1).

Table 1. Stimulus-to-response mapping

Word	Assigned action		Distractor	Assigned action
hicha	left hand		hichu	none (control)
hishu	left foot		hisha	none (control)
hisa	right foot		hisu	none (control)
hivu	right hand		hiva	none (control)



Each word differed from its respective distractor in the fourth phoneme. The beginning of the fourth phoneme will be referred to below as disambiguation point (Figure 1).

Figure 1. Stimuli design. Black triangle denotes disambiguation point: beginning from this moment, stimuli can be differentiated from each other

Audio recordings of the pseudoword stimuli were synthesized using matlab software. The acoustic contrasts between stimuli were identical, which was achieved by cross-splicing the same phonemes in different combinations, thus ruling out acoustic confounds within each stimulus type.

Two additional non-speech auditory stimuli that differed in modulated spectral composition were used as positive and negative feedback signals. Participants were informed about the meaning of the feedback signal before the experiment.

Stimuli were presented to participants in a quasi-random order binaurally using plastic ear tubes at 50 dB above individual hearing thresholds. Stimuli were presented and participants' responses were recorded using Presentation 14.4 software (Neurobehavioral systems, Inc., Albany, California, USA).



3.3 Experiment design

In the course of the experiment, participants were instructed to learn the relation between unfamiliar pseudowords and movements made by their extremities via receiving external auditory feedback (positive or negative).

The experiment consisted of four blocks: (1) passive listening before learning, (2) active learning, (3) active stable performance, (4) passive listening after learning. Duration of the whole experiment was about 2 hours.

During the experiments, the participants were comfortably seated in the MEG apparatus placed in an electromagnetically and acoustically insulated chamber. A fixation cross was constantly present on a projection monitor in front of the participants at eye level; participants were required to keep their gaze at the fixation cross during the whole experiment in order to minimize artefacts caused by participants' eye movements.

The instruction, which was read orally to the participants before the experiment, offered them to find out the stimulus-to-response mapping on their own, using the positive and negative feedback signals. No additional information concerning the stimulus-to-response mapping was revealed to them, and thus the behavioral procedure was essentially trial-and-error learning.

The participants were required to commit one of the four motor responses (by left hand, or by right hand, or by left foot, or by right foot), or to commit no response at all. In other words, after hearing each of the stimuli participants had to choose between five options: to make one of the four motor responses or to commit no response.

Motor responses were recorded using two hand-held buttons and two custom-made pedals, which interrupted a laser light beam to register the motion onset and record it synchronously with the MEG data.

The feedback stimulus was presented 2000 ms after the end of the pseudoword stimulus (Figure 2). The positive feedback was presented if the participant complied with the task stimulus-to-response mapping - i.e. committed a proper response to a target stimulus or refrained from a response to a distractor (Table 1). The negative feedback stimulus was given if the response was made with a wrong extremity, or if a motor response was omitted after a stimulus that required a response, or if any response was committed after a distractor stimulus.

Figure 2. Experimental procedure. Passive block type is given above. Active block is given below. Blocks were administered in following sequence: (1) passive listening before learning, (2) active learning, (3) active stable performance, (4) passive listening after learning

Interstimulus interval (from the pseudoword stimulus offset till the next pseudoword stimulus onset) was randomly varied within 2000-2500 ms.

3.4 Procedure

The active phase of experiment included two blocks: it started with the `active learning' block followed by `stable performance' block. A short break was offered between the blocks, during which participants were offered to rest while remaining seated in the MEG apparatus. The basic procedure, described above, was identical for the two blocks, except for the number of trials. The learning block ended if a participant reached the learning criterion or if 480 stimuli were presented. Learning criterion was that for each of the eight stimuli there should be at least 4 correct trial outcomes out of 5 latest presentations of any given stimulus; the block was terminated only if the criterion was reached for all of the eight stimuli at a given moment. Twenty-two participants reached the criterion; two participants did not reach the criterion and thus went through a full length of 480 trials in the learning block. Since their overall hit rate during both blocks was well within the range of performance of other 24 participants, these two participants were not excluded from the analyses. Thus, the `active learning' block length varied from 74 to 480 trials (approximately 6 to 40 minutes). The `stable performance' block had a fixed length of 320 trials (approximately 30 minutes).

Two identical passive blocks were administered before and after active ones.

Participants were offered to watch silent movie on the screen meanwhile auditory stimuli were presented to them. Participants were instructed to pay no attention to auditory stimuli.

Interstimulus interval (from the pseudoword stimulus offset till the next pseudoword stimulus onset) was randomly varied within 1500-2000 ms. `Passive' blocks last about 30 minutes each.

3.5 Recording

MEG was recorded using a 306-channel MEG 'Vector View' system (Elekta Neuromag, Helsinki, Finland) at sampling rate of 1000 Hz, a high-pass filter of 0.1 Hz, and low-pass filter of 330 Hz were applied.



3.6 MEG preprocessing

All raw MEG data were first processed using the temporal signal-space separation method to remove biological artifacts and other environmental magnetic sources originating outside the brain. The temporal signal-space separation method as well as movement compensation were achieved with options implemented in MaxFilter software (Elekta Neuromag, Finland). Static bad channels were also detected and excluded from subsequent processing steps.

Trials were rejected if they were heavily contaminated with eye movements or blinks (if the bipolar electrooculogram electrode pairs showed a voltage difference greater than 200 µV), or if the peak-to-peak MEG amplitude exceeded 12,000 fT/cm for magnetometers or 3000 fT/cm for gradiometers in either direction. Artifacts generated by remaining eye blinks were removed using signal space projections, which was performed using Brainstorm software (Tadel, Baillet, Mosher, Pantazis, & Leahy, 2011).

Trials with increased contamination of MEG signals with high frequency muscle activity were excluded by thresholding the mean absolute values of MEG data filtered above 60 Hz in each channel at 5 standard deviations of the mean values averaged across channels.

3.7 Sensor-level analysis

Groups of sensors depicted on Figure 3 were chosen for ERP analysis, separately for the left and the right hemispheres. Such wide ROIs at sensor level were used on the basis of large body of reports evidencing that speech processing effects were mostly observed in perisylvian areas, including insular, temporal and inferior frontal cortex (e.g., Amunts et al., 2010; Hagoort, 2015; Hickok & Poeppel, 2015).



Figure 3. Sensors of left and right hemispheres used for statistical analysis of RMS signals (red and blue respectively)

Root mean square (RMS) signal over gradiometers was analyzed, for the passive blocks before and after learning (Passive 1 vs Passive 2), for the two types of stimuli ('words' vs. 'distractors').

Repetition suppression of the evoked signal (Auksztulewicz & Friston, 2016; Grill-Spector et al., 2006) was expected for both types of stimuli. In order to assess the interaction between the effects linked to acquired semantics and the effects related to word form familiarization in course of repetition Davis and Gaskell (2009), the main statistical analyses both for sensor and source data was performed on the double difference:

double difference = (word2 - word1) - (dist2 - dist1) (1)

where word1 and word2: magnetic evoked responses to 'words', and

dist1 and dist2: magnetic evoked responses to 'distractors' in Passive 1 and Passive 2 blocks correspondingly.

The double difference also cancelled any acoustic variance between stimuli as long as first three phonemes were identical within each word-distractor pair and the last vowels (4-th phoneme) were included into words and distractors in a counterbalance way (see Table 1).

Using t-tests with TFCE correction and permutation procedure (1000 permutation repetitions) to account for multiple comparisons, we determined the time intervals during which the double difference was statistically significant.

3.8 Source-level analysis

The cortical sources of the magnetic evoked responses were reconstructed using distributed source modeling (MNE software). Source estimation was performed using unsigned cortical surface-constrained L2-norm-based minimum norm estimation by using the MNE software suite. A grid spacing of 5 mm was used for dipole placement, yielding 10,242 sources per hemisphere. 'Orientation constraint parameter,' which determines the extent, to which dipoles may deviate from the orthogonal orientation in relation to the cortical surface, was set to 0.4. Depth weighting with the order of 0.8 and the limit of 10 was applied.

Spatial distribution of repetition suppression was analyzed using t-test with FDR correction after time averaging.

To specify the topography of the differential effect between `words' and `distractors' revealed within third time interval we considered time points corresponding to the lowest p-values (< .01) from RMS analysis (Gross et al., 2013). Voxel-wise t-test was performed on double difference calculated according to equation (1).

Cortical areas were obtained as clusters of significant voxels (p < .01). If a voxel had at least one adjacent voxel with significant effect, the voxel was classified as significant. If a voxel had no adjacent voxels with significant effect, the voxel was classified as non-significant. Clusters included less than 20 voxels were not considered.

For illustrative purposes, timecourses of the MEG signal at the most significant voxels averaged with 6 adjacent voxels were plotted.



Chapter 4. Results

4.1 Behavioral data

All participants were successful on the task: average hit rate during the Active session 2 (after active learning block) was 96.3 ± 3.8% (mean ± standard deviation).

4.2 Sensor-level analysis

Visual inspection of the RMS signal timecourses revealed that responses to both types of stimuli underwent strong suppression in both hemispheres, yet the suppression was stronger for 'distractors' compared to 'words' in the left hemisphere only. The analysis produced no statistically significant intervals in the right hemisphere (Figure 4).

Figure 4. Timecourses of the grand-average RMS signal for right hemisphere. Vertical line denotes word disambiguation point (“0”), after which “words” differ from “distractors”. word1 and word2 -- magnetic evoked responses to "words", and dist1 and dist2 -- magnetic evoked responses to "distractors" in Passive session 1 and Passive session 2, correspondingly. The difference between two stimulus types are given below



TFCE-based permutational analysis of the RMS signal in the left hemisphere produced two statistically significant intervals: 144-217 ms and 226-362 ms after the disambiguation point (Figure 5).

Figure 5. Timecourses of the grand-average RMS signal for left hemisphere. Vertical line denotes word disambiguation point (“0”), after which “words” differ from “distractors”. word1 and word2 -- magnetic evoked responses to "words", and dist1 and dist2 -- magnetic evoked responses to "distractors" in Passive session 1 and Passive session 2, correspondingly. The difference between two stimulus types are given below. Green zones indicate the time windows for which the double difference was significant (TFCE permutation analysis)

In order to visualize the direction and dynamics of the effects, topographic maps integrated by 35 ms were plotted along time window obtained after RMS analyses. Significant repetition suppression prevalence was observed for distractors (Figure 6).



Figure 6. Topographic maps of differential gradiometer's signal (Passive2-Passive1) within statistically significant time window for `words' and `distractors' and their difference depicted below

4.3 Source-level analysis

Spatial distribution of repetition suppression was analyzed within three time intervals. The first one was taken from stimulus onset to disambiguation point (-400-0 ms). The second was an exploratory interval 50-150 ms, which was taken in order to check for early effects of word form recognition reported by MacGregor et al. (2012), Kimppa et al. (2015); Shtyrov (2011). The third interval was obtained during sensor-level analysis (150-400 ms).

We reported significant repetition suppression effect for both stimuli type after disambiguation point; no prominent effect was observed right after stimulus onset. We observed significant repetition suppression in a few cortical regions within early exploratory interval 50-150 ms, however the same effect became more prominent and generalized on the main interval 150-400 ms, which was obtained empirically. Widespread suppression was registered within perisylvian regions, Figure 7.



Figure 7. Localization of repetition suppression effect averaged within three time intervals: from stimulus onset to disambiguation point (right), exploratory interval in order to check for ultra-rapid word-form recognition (middle) and empirical interval obtained after RMS analyses (left). Dark blue denote t-test p-value<0.05 (uncorrected), Light blue denote voxels which passed FDR correction on space, note that ** stands for p-value<0.05 (corrected) and *** stands for p-value<0.01(corrected)

To specify the topography of the differential effect between `words' and `distractors' revealed within third time interval, we considered three time points corresponding to the lowest p-values from RMS analysis: 192 ms, 267 ms and 326 ms. Source space signals were integrated by 35 ms around each p-value minimum (Figure 8).



Figure 8. The picture above shows double difference between words and distractors before and after learning within time interval obtained after RMS statistical analysis. Corresponding p-values after TFCE correction are given below. Vertical lines with red dotes denote time points where differential effect is the most significant. Source space signals were averaged around most significant points as showed by purple rectangles.

The differential effect between `words' and `distractors' was found in the following cortical regions, see Table 2.

Table 2. Source space clusters

Cluster localization	Most significant voxel MNI coordinates
Ventral premotor and ventral prefrontal cortex (VPF-VPM)	-35.92	22.32	25.45
Insula and frontal operculum	-35.80	14.27	10.07
Triangular inferior frontal gyrus (triangular IFG)	-40.18	23.12	7.04
Intraparietal sulcus (IPS)	-37.38	-49.22	35.73
Anterior superior temporal sulcus and Middle temporal gyrus (aSTS-aMTG)	-50.68	-21.26	-3.42
Temporal pole	-42.58	11.63	-23.73



For the early time frame (190 ms), the effect was found in several separate, relatively small clusters of insular, prefrontal and temporal cortex. The most notable was the cluster located in the most anterior part of intraparietal sulcus (IPS), Figure 9.

Figure 9. Significant spatial clusters on the left cortical surface obtained for early time frame (190 ms). Top and bottom panels depict differential timecourses. Blue line denotes earning-related activation on words. Red line -on distractors

On later time frame (265 ms) we observed widespread activation within frontal operculum and the posterior insula strongly intersected with ventral prefrontal (VPF) and ventral premotor cortex (VPM), figure 10.



Figure 10. Significant spatial clusters on the left cortical surface obtained for 265 ms time frame. Top and bottom panels depict differential timecourses. Blue line denotes earning-related activation on words. Red line -on distractors

The latest time frame (326) reveals spread of activation from perisylvian regions toward triangular inferior frontal gyrus (IFG) along with activation of temporal pole which goes in connection with the most stable cluster of anterior superior temporal sulcus and middle temporal gyrus (aSTS-aMTG), Figure 11.

Timecourses depicted in figures 8-9, evidence that the effect in most clusters is rather stable within wide time window 150-400 ms, obtained for RMS signal on sensor-level.



Figure 11. Significant spatial clusters on the left cortical surface obtained for late time frame (325 ms). Top and bottom panels depict differential timecourses. Blue line denotes earning-related activation on words. Red line -on distractors



Chapter 5. Discussion

5.1 Statement

Operant conditioning paradigm used in the current study provided successful learning in all subjects: accuracy of at least 90% or better was achieved after the course of learning in all participants (note, that on each trial, a participant had a choice between 5 options, thus chance-level performance would produce accuracy level of 20%). This means that stimulus-to-response mapping was acquired by all participants. Since the procedure involved pseudowords that were associated with actions, and the experiment lasted less than 2 hours, we can say that fast semantic mapping was achieved.

Initially stimuli used in the experiment were totally unfamiliar to participants and carried no meaning. In the course of associative operant learning, four pseudowords were assigned to four movements (referred here as `words'), while the other four pseudowords were used as controls and thus were not associated with any movements (referred here as `distractors'). Strict control for stimulus exposure experience concerning all pseudowords used, and explicit learning procedure enabled us to independently assess the two aspects of pseudoword lexicalization: phonological familiarization and meaning acquisition, as well as their interaction Davis and Gaskell (2009). Specifically, we report pseudoword familiarization effects and the differential effect between familiarized pseudowords with and without acquired semantics.

We compared magnetoencephalographic responses to the two types of stimuli during passive listening before and after learning. Significant repetition suppression was observed for both types of pseudowords, yet the suppression of the response to words was smaller compared with the suppression of the response to distractors. This differential effect was significant only in the left hemisphere within 150-360 ms after the disambiguation point; source localization revealed a number of brain areas, persistent activation of which contributed to the differential effect observed during the time interval obtained. Importantly, all of the cortical areas highlighted by our data-driven analysis are known to be involved in speech processing.

5.2 Repetition suppression

In the current study, a prominent repetition suppression effect was observed for both words and distractors. Generally, repetition suppression is a common phenomenon often observed in various modalities (Auksztulewicz & Friston, 2016; Grill-Spector et al., 2006). Significant repetition suppression was observed in both hemispheres, and its localization was compatible with areas involved in auditory processing, speech processing and motor planning.

Current experimental paradigm implies that participants became well familiarized with stimuli via performance of an effortful active task. As long as the task was to distinguish between pseudowords, the establishment of word-form representations (phoneme concatenation) was essential for successful performance. After word-form representation is formed, it habituates during extensive. This means a diminution effect, which indicates neuronal adaptation and results from a reduction of the size of the neuronal ensemble that reacts to repetitive stimuli (Lцfberg, Julkunen, Tiihonen, Pддkkцnen, & Karhu, 2013).

...