Co-speech gesticulation accompanying verbs of motion in Russian

Размещено на http://www.allbest.ru/

Co-speech gesticulation accompanying verbs of motion in Russian

1. Acknowledgements

I want to thank Ekaterina Vladimirovna Rakhilina for her inspiration and for her ability to motivate me to do research in linguistics. The study of gestures in the School of Linguistics would not have started without Ekaterina Vladimirovna.

I am grateful to my scientific consultant, Daria Ryzhova, who supported my research ideas and edited my writings for the last three years.

I want to express gratitude to the participants of Summer Corpus Linguistics School of the University of Birmingham, the conversation with whom inspired me to explore co-speech gestures.

I would like to thank Asli Цzyьrek for taking the time to discuss the ideas of this work, providing the literature and advice on working with A. Cienki. I also owe a lot to Alan Cienki for being my scientific consultant. He spent a lot of time, providing the relevant literature on co-speech gestures, answering my questions and discussing this topic.

I am grateful for the resources provided for my research. Many thanks to Red Hen Lab team for providing access to NewsScape database. Thanks to Yulia Nikolaeva for answering my questions about working with Pear Stories corpus.

2.Introduction

A large number of studies in linguistics are related to verbs of motion. These studies began with Leonard Talmy's work in lexical semantics [Talmy 1985], where the researcher identified the 4 main components of the motion event (Path, Figure, Ground and Motion) and also Manner. Talmy divided the languages of the world into verb-framed and satellite-framed depending on the part in which the meaning of Path is expressed [Talmy 1985].

Interestingly, typological differences in languages regarding motion events can be studied not only by investigating the speech but also by observing the co-speech gestures. Gesture studies is a relatively new area, started with the works of David McNeil and Adam Kendon [McNeil 1992; Kendon 1972; Kendon 2004].

Many researchers were concerned about how motion components are expressed in gestures. McNeill and Duncan (2000) showed that speakers of Spanish (satellite-framed) and English (verb-framed) express the components of Path and Manner in different ways in gesticulation. Further, [Kita Цzyьrek 2008] showed that at the age of 3, gestures of children, who speak typologically different languages as Turkish and English, are language-specific when expressing situations with verbs of motion.

Russian is considered as a satellite-framed language, except for some verbs in which the semantics of "path" is contained in the root, e.g. padat'' `to fall'. Some of the gesture studies conducted on the material of the Russian language addressed the issue of the expression of verbs of motion in gesticulation. Specifically, Elena Grishina [Grishina 2017] did a corpus study of accompanying gesticulation on the material of the Multimedia Russian Corpus and defined the gestural profiles of verb roots and prefixes for Russian. The author made four general conclusions. First, the accompanying gesticulation differs for all verb prefixes. Further, verbs with roots that have the semantic component "fixed point", e.g. -puskat'/-pustit' `to throw', are accompanied by a gesture that is directed from up to down. Similarly, verbs that have the semantic component "downward movement" in their roots also prefer the top-down direction in gesticulation. Finally, the meaning of the trajectory or the direction of the movement is transmitted by either the movement in the form of a curved arc or by the upward movement of the hand.

However, in the study of E. Grishina [Grishina 2017] verbs were considered separately (without investigation of the context, namely, prepositions or adverbs), and general conclusions about gestural patterns of verb roots with general semantics are presented. We assume that semantics of spatial prepositions and adverbs can have a significant influence on the way gestures are performed, especially for motion verbs. Namely, it may affect the trajectory of gesture. We would like to consider in detail the gestures accompanying the most frequent Russian motion verbs, observing the constructions with spatial prepositions and adverbs too.

In our study, we want to complete the existing paradigm of gestural profiles of verbs of motion in Russian by considering verbs of motion in constructions with spatial prepositions and adverbs. We conduct a corpus study on the data material of several corpora to collect a sufficient amount of data and to determine whether gesticulation differs in experimental data, in the conditions of the stage speech and in spontaneous speech.

Therefore, this study aims is to determine gestural patterns for prefixed and non-prefixed verbs of motion in Russian in constructions with spatial prepositions and adverbs.

Our study hypothesizes that the character of accompanying gesticulation for verbs of motion in the Russian language is determined by the complex influence of semantics of the verb root, semantics of the verb prefixes, and semantics of the prepositional constructions. ?

3. Theoretical background

Section 3 is devoted to a review of the relevant literature. In section 3.1, we give a background of the core studies on verbs of motion and describe the specific characteristics of Russian verbs of motion. Section 3.2 presents the overview of the gesture studies and highlight the basic concepts that are important in the study of co-speech gestures. In section 3.3, we review the relevant studies on co-speech gestures accompanying verbs of motion. Section 3.4. covers the effect of the verb aspect on the character of gesticulation and the gestural patterns of verb roots, verb prefixes, and prepositions, identified in Russian.

3.1 Motion verbs

language typological russian

One of the fundamental researches in the domain of motion verbs is the series of studies by Leonard Talmy [Talmy 1972; 1975] on the typology of lexicalization patterns. Talmy considered the phenomenon of motion to have a strict structure. The author identified the following components of motion event: Motion, Figure, Ground, Path, and Manner/Cause. Motion component represents the presence of motion, Figure is understood as the object which is considered as moving in relation to another object. Ground is the object which is considered as located with respect to which a Figure is considered as moved. Path, meanwhile, indicates the direction of movement. Finally, Manner/Cause is defined as the way an object moves.

Leonard Talmy has shown that the languages of the world lexicalize those components in three combinations: Motion + Manner/Cause, Motion + Manner/Cause, and Motion + Figure. The first combination is characteristic of Romance, Semitic, and Turkic languages. Further, the combination of Motion + Path is common in such languages as English, Russian and German, and the latter is typical for some languages in North America.

In the following study [Talmy 1985] Talmy introduced a typology of motion verbs by opposing verb-framed and satellite-framed languages, using the Path's lexicalization as a delimitation parameter. Path can be expressed by either the main verb (in a verb root) or by a satellite and/or preposition. Thus, verb-framed refers to those languages where the Path component is expressed in the verb root. Examples of such languages are Turkish or Spanish. Examples of satellite-framed languages where Path by a satellite are English (verb particles), Russian and German (verb prefixes). Leonard Talmy's work has significantly predetermined the future direction of research in the field of the lexical semantics and typology of verbs of motion. The existing typology was complemented by Dan Slobin [Slobin 2004], who added the Equipollently-Framed category, where Path and Manner are expressed by equivalent grammatical forms. Examples of such languages are serial-verb languages (Sino-Tibetian) and bipartite verb languages (Algonquian).

There are also many studies proved that the Path component is expressed in a different way in the speech of children who are native speakers of typologically different languages. For example, researchers [Allen, Цzyьrek, et al. 2007] worked on the material of English, Japanese, and Turkish, and showed that children at the age of 3 years use semantic-syntactic mappings preferred by adult native speakers, and express subtle syntactic differences that encode different relationships between Manner and Path.

So how are verbs of motion represented in the Russian language? According to the definition, motion verbs “denote a situation in which X at some point in time occupies the location L1, and at the next moment X occupies the location L2, while L1 is the initial point of movement, and L2 is its endpoint” [Maisak, Rakhilina 1999:53]. As defined in [Avilova 1976], motion verbs are a structural-semantic type of the Russian verb.

Russian is a satellite-framed language. In Russian, the meaning of Path is expressed through prepositions and prefixes [Slobin 1996]. In Talmy's terminology, the meaning of the verb root corresponds to the Manner component. Nevertheless, Russian motion verbs have some distinctive characteristics in terms of aspect, directionality, and Manner. We consider them in more detail below.

Motion events that verbs of motion can express in Russian are divided into static and dynamic. Dynamic motion is determined by the presence of a spatial reference point. In the dissertation of E. Koroleva [Koroleva 2010] it is stated that verbs of motion in constructions with the prepositions are the main way to denote dynamic spatial movements in the Russian language.

Russian verbs of motion also have a category of directionality. All verbs can be divided into determinate (classical understanding of motion as moving an object from one point to another) and indeterminate (denote a random or repetitive movement) verbs, e.g.: begat'-bezhat'; `run' [Zaliznyak, Shmelev, 2000; Pavlenko 2009].

Now we consider the category of an aspect in the Russian language. It is a syntactic category that characterizes the verb in all its forms [Zaliznyak, Shmelev, 2000; Pavlenko 2009]. Each verb in Russian has either a category of perfective indicating that the action has taken place, or imperfective, indicating that the action takes place. Like the verbs of all other semantic groups, Russian motion verbs can be represented in the form of perfect and imperfect, constituting the verb pairs. The specific character of this group of verbs in Russian is the semantic variety of forms of the perfect form. A large set of prefixes can be used for the derivation of verbs of the perfect form, see: yekhat' - vyyekhat', uyekhat', priyekhat', poyekhat', proyekhat' are synonyms of the verb `go', but denote different movement trajectories.

As E.V. Paducheva [Paducheva 2002] emphasized, verbs of motion (without prefixes) have a forward-oriented direction of movement. Russian prefixes are distinguished among other morphemes only by the fact that they are included in the wordlists of all dictionaries of interpretation, and the semantic description of verb prefixes in the Russian language assumes the description of all context meanings of prefixes [Krongauz 2017]. The prefixed verbs of motion are characterized by the presence of a mandatory valency of the direction, which is localized in the prefix: “Where to?” (lative verbs) or “Where from?” (elative verbs). The first group includes prefixed verbs of motion with prefixes v-, vz-, do-, za-, pod-, pri-, and the second group consists of prefixed verbs of motion containing prefixes vy-, ot-, s-, u-. According to Elena Paducheva's classification, the prefixes pro-, pere- and ob- are not spatial.

Next, we consider the classification of spatial prepositions in the Russian language, which are found in constructions with motion verbs. It is possible to divide prepositions into simple and complex (consisting of several words), and also into static and dynamic (similar to situations of movement in which they are used). Some prepositions in Russian with spatial meaning are diachronically connected with spatial prefixes, and their system is similarly located semantically [Sitchinava 2017]. In [Khoruzhaya 2007; Arbuzova 2011] the following meanings of spacial prefixes in Russian are described: vokrug (movement in an enclosed space), v, k, na (motion to the endpoint, “Where to?”) vdol', po, mimo (linear length of movement) do (limit of movement), iz, s, ot (motion to the starting point, “Where from”).

The combination of prefixed verbs of motion with spatial prepositions is regulated by the so-called “law of double - prefix - prepositions” [Vinogradov 1986]. According to Vinogradov's point of view, the spatial preposition repeats the prefixed verb, strengthening its meaning poyekhat' po doroge `to go along the road'. In the other case, the meaning of the preposition after the prefix corresponds to the semantics of prefix, but does not repeat it: poyekhat' vdol' dorogi `to go along the road'. Nevertheless, it is a rather complicated paradigm, because in Russian the prefix and the preposition may not only mark the spatial relationship, but also have a meaning not related to movement, but to the aspect or point of view of the speaker [Pavlenko 2009] zayekhat'/priyekhat'/vyyekhat'/pod"yekhat' k garazhu `to drive up to the garage' each construction has a different PATH meaning.

To sum up, the works of Talmy have initiated a huge number of cross-linguistic studies of verbs of motion. Talmy identified five main components of the motion event: Motion, Figure, Ground, Path, and Manner/Cause. Talmy classified the languages of the world based on the lexicalization of the Path component. In general, Russian is a satellite-framed language, with the exception of some verbs in which the semantics of “path” is contained in the root, e.g. padat' `to fall'. In Russian, the Path component is usually expressed in prefixes, while the Manner component is lexicalized at the root of the verb. Unlike verbs of motion in other languages, all verbs of motion in the Russian language have a category of aspect (perfective/imperfective). Besides, verbs of motion in the Russian language have a category of directionality.

We can conclude that verbs of motion in constructions with the prepositions are characterized as the main way to denote dynamic spatial movements in the Russian language. Verbs of motion in the Russian language have plenty of distinctive features and constitute a complex paradigm. Verbs of motion (without prefixes) have a forward-oriented direction of movement. The prefixed verbs of motion are characterized by the presence of a mandatory valency of the direction which localized in the prefix. The combination of prefixed motion verbs with spatial prepositions is regulated by the so-called “law of double prefix- preposition”.

3.2 Gesture studies

Gesture studies are an interdisciplinary area of study. The area of gesture studies includes the role of gesture in verbal communication, the link between gesture and thought, the recurrence of gestures, and the representation of gestures in culture. Despite the emerging attention to the study of the relationship between speech and gesture, many questions remain to be studied in the field.

The classical study of Efron “Gesture, race, culture” [Efron 1941], the author made a classification of gestures for the first time and distinguished illustrative gestures and gestures that have lost their meaning baton-like gestures), and a study of Ekman and Friesen [Ekman, Friesen 1969] on the repertoire of non-verbal behavior (the authors described patterns of movement of the hands and divided gestures into illustrators, adapters and emblems) provided the base for gesture studies. David McNeill, whose work [McNeill 1992] continued the study of gestures, described gestures as «window onto mind». McNeill [McNeill 1992] believed that gestures and speech emerged at the same time: for the origin of the language, the gestures were necessary.

According to Adam Kendon [Kendon 1972; 1980], gesture is the action of any part of the body when it is used as part of an utterance. As Elena Grishina [Grishina 2017] pointed out, the gesture is a two-way sign that has a signifier (the movement of one or another organ of the human body) and a signified (the meaning of the sign). Different types of gestures, ranging from informal, spontaneous hand movements that often accompany speech to standardized forms of sign language, arranged on the same axis, represent the Kendon continuum [McNeill 1992]: Gesticulation (со-speech gestures) > Speech-Linked (Language-Like Gestures) > Pantomime > Emblems > Sign language. If we follow the given pattern from left to right, we can trace how gestures begin to resemble a separate language: as well as the fact that to continue the full-fledged functioning of the gesture accompanying the gesture with speech activity is not necessary to a greater extent.

Gesture has a structure and the main characteristics. It was first described in [Kendon 1972] and then supplemented in [Kita 1990]. So, the gesture consists of five components: the preparatory phase, pre-stroke hold, stroke, post-stroke hold, and retraction. The main semantic part of the gesture is the stroke phase. The characteristics of co-speech gestures were described in the works of Adam Kendon [Kendon 1972; 1994; 2004]. The co-speech gestures of the speaker are recognized by the speaker himself, gestures are informative (with the help of a gesture the speaker transmits some information), gestures are unselfish (we believe that the utilitarian purpose of the gesture does not exist) and gestures are controlled (gestures are not reflexive). Adam Kendon [Kendon 1994; 2004] also defined the following parameters of distinguishing of co-speech gestures: verbal accompaniment, the degree of regularity of gestures, and the consistency of the language features.

In the classical work on gesture linguistics, David McNeill [McNeill 1992] identified several categories of gestures: deictics, iconics, metaphorics, beats, and cohesives. Deictics indicate the location of an object in time or space. Iconics and metaphorics duplicate or complement words. Deictics correspond to noun phrases or adverbs and iconics and metaphorics correspond to verbs or verb phrases. Iconics most often mark new information important for the recipient [Nikolaeva 2013]. McNeill [McNeill 1992] also divided the gestures into two groups according to the viewpoint at which the gestures are performed: observer viewpoint gestures and character viewpoint gestures. Later, Kendon [Kendon 2004] subdivided the gestures on pragmatic (responsible for the pragmatic aspect of communication, for example, for focus and comment) and substantial, which have an additional meaning to the phrase.

There is a large number of cross-linguistic studies on the differences in gestures among speakers of different languages. One of the first studies was done on the English-French bilingual children aged between one and three years. The study found that the appearance of iconic gestures is related to children's language proficiency, measured by the mean length of utterance [Nicoladis, Mayberry, Genesee 1999]. Further, in the work [Gullberg 2006; 2008], M. Gullberg proposed to consider gestures as part of the language material of the target language. The learning of a new language means learning new words and grammar and, according to the author, also potentially means learning how to gesture in a new language.

At the moment, there are several research centers involved in the investigation of gestures. The research teams are working on the study of gestures within different theoretical approaches and methodologies. Gestures are studying by linguists who work on cognitive linguistics and critical discourse analysis [Becker, Cienki, et al. 2011; Becker, Raymond, et al. 2018], neuropsychology [Lausberg 2019], who investigate the relations of gestures and speech science and Human-Machine-Interaction [Bressem 2013] and address communication naturally produced in various professional setting [Cienki, Iriskhanova et al. 2017].

In most cases, researchers working with gestures conduct experiments for data collection, e.g. in [Alibali et al. 2001; Kita et al. 2007]. Usually, the participants of the experiment are asked to talk about the video they have watched, to describe the picture they have seen, or to talk about a given or free topic in groups of two or more persons. Researchers can also involve people who know each other in participating in the experiment [Becker, Raymond, et al.2018]. There is also an opinion that the experimental environment affects the free character of spontaneous speech and gesturing [Kibrik 2010; Kibrik, Nikolaeva 2016], so scientists try to keep the free spontaneous nature of the conversation during the experiment and also work with data from corpora. An example of a source used by researchers is the Red Hen Lab database [Joo, Steen, & Turner 2017], where a huge number of media broadcasts around the globe are collected (350 000 hours of NewsScape archive).

Currently, many researchers (C. Mьller, J. Bressem, A. Cienki, and others) are working on the identification of recurrent components of co-speech gestures. The researchers believe that the main components of co-speech gestures have similar sets of values and are regularly reproduced [Bressem, Mьller 2014; Mьller 2008]. Many of the described recurrent gestures perform pragmatic functions (mark the type of illocution, focus and comment).

There are several research groups that investigate co-speech gestures in Russian. Researchers study and analyze the discourse, and conduct corpora-based studies.

First of all, we should mention the works of G. Kreidlin, a Russian linguist, a specialist in non-verbal semiotics, and typology of non-verbal acts. Kreidlin emphasizes the importance of studying non-verbal communication, the central issue of which is multimodality [Grigoryeva et al., 2002]. His work focused mainly on etiquette gestures from a semiotic perspective. The author has created a dictionary of Russian body language that includes communicative gestures typical for native speakers of the Russian language [Kreidlin 2001].

Then, a research group at the Institute of Linguistics of the Russian Academy of Sciences, headed by A.A. Kibrik, argues that gestures use a wide enough range of possibilities to transmit pragmatic values, and that the study of accompanying gestures plays an important role in understanding the discourse [Kibrik 2010]. Among the non-vocal communication channels, the most studied one is the gesture channel (namely, manual gestures that accompany speech). The authors consider the verbal component of the conversation, as their research shows that the form of the gesture can vary greatly from case to case. The researchers also take into account such parameters as the number of interlocutors (2 vs. 3+) and the communication environment (specifically created conditions for experiment vs. unprepared environment). The multimodality of communication is understood as the use by the communicating subjects of all possible modes for transmitting and receiving information. As for the theoretical directions, the authors raise the question of how different channels, simultaneously used by the communicant, interact with each other. Other issues are the question of temporal coordination of different channels (verbal, prosodic and gesture), the relationship between the direction of gaze, gestures and speech.

The other group of researchers based on the PoliMod laboratory and headed by A. Cienki, uses a socio-cognitive approach to discourse analysis (taking into account social factors in the basis of linguistic and speech variation, as well as cognitive processes and mechanisms that determine the semantic and structural characteristics of language expressions). The researchers conduct experiments to identify the correspondence between certain features of gesticulation and certain properties of speech (in particular, the boundary of gestures and the aspectuality of verbs).

The most complete theoretical information on the semantics of co-speech gestures' components in Russian can be found in the monograph “Russian gestures from a linguistic perspective: A collection of corpus studies” of Elena Grishina [Grishina 2017]. Elena Grishina created the Multimedia Russian Corpus, MURCO [http://www.ruscorpora.ru/new/search-murco.html], and performed an analysis of different types of co-speech gestures in Russian based on these data.

Elena Grishina considered corpus data as the most reliable material in the study of gestures. Experimental methods in this approach were used only when it is necessary to check the data obtained as a result of the statistical corpus-based study. Grishina stated that the main purpose of her research was to identify repetitive, reproducible components of gestures in the Russian language and to describe the main semantic characteristics of gestural components. The author described Russian deictic gestures that indicate external objects in relation to the speaker or the speaker himself, index gestures, components of gestures having an iconic character, eye movements, and palm configurations [Grishina 2017].

3.3 Gesture studies of motion events

There are several cross-linguistic studies devoted to the gestural representations of events in typologically different languages. To collect data, in all the papers discussed below, researchers showed video clips to the participants of the experiment, and then they were asked to tell about the content of the video to another participant.

One of the first studies in this area is described in [McNeill, Duncan 2000]. The authors studied how the elements of the motion event are expressed in the gestures of speakers of English (satellite-framed language) and Spanish (verb-framed language) in narrations of a cartoon “Canary Row”. In English, Path is encoded in the adjunct of the verb, and Manner is expressed in the verb. In Spanish, Path is expressed in the verb, and Manner is encoded in a gerund, a separate phrase, or clause. When translating texts from English to Spanish, Manner is missing in translation because the text becomes syntactically redundant. As for conclusions on gestural behaviour, the co-speech gestures modulate the lexical system of both English and Spanish. In English, the gesture expresses Manner if it is part of the speaker's focus. When Manner is out of focus, only Path will be encoded in the gesture. In Spanish, the Manner, on the contrary, may appear in the character's gesticulation, even if the Manner component is absented in speech. So, the authors showed that expression of event components in gestures compensate for those expressed in speech and provide extra information if speakers are likely to omit certain elements of motion events in speech.

A cross-linguistic study on co-speech gestures was also carried out by [Цzyьrek et al. 2005] on the material of Turkish (verb-framed) and Japanese (verb-framed) and English (satellite-framed) languages. The participants of the experiment were retelling the content of short animated video clips. The researchers looked at what the gesture would look like in Manner-only and Path-only sentences, whether it would include an expression of additional compensated elements or not. An important conclusion from this study is that when speakers of all observed languages expressed only Path in the speech, they expressed Path in gestures too. When only the Manner component was expressed in the sentence, it was also expressed in gestures. Further, the authors made conclusions about the sentences in which both Path and Manner are expressed. It turned out that English-speaking people express elements of Manners and Path in one gesture. At the same time, Japanese and Turkish speakers are more likely to use separate gestures (one to express Manners and one to represent the Path).

The next study of this research group [Allen et al. 2007] included the identification of gestural patterns in the gesturing of three-year-old monolingual children speaking English, Turkish, and Japanese. The results of this study showed that in general, in expressing Manner and Path in speech and gesture, the same patterns were observed as for adult speakers.

To sum up, it is clear that the way semantic elements of a motion event are mapped onto syntactic structures dramatically influences the gestural representations of events.

3.4 Gesture studies of Russian verbs

3.4.1 Gestural expression of event construal in Russian

In 2011, A. Cienki and R. Becker developed a project on aspect, speech, and gestures of English speakers [Becker 2011]. The initiative to conduct this study based on a broader data, taking into account the grammatical aspect, served as a starting point for the development of a cross-linguistic project, study of the aspectuality across different languages, German, French and Russian [Becker et al. 2018]. The project was based in the Laboratory of Multimodal Communication and Knowledge (PoliMod) at Moscow State Linguistic University.

The authors asked the participants of the experiment, previously familiar to each other, to discuss one or another proposed topic in pairs. One of the objectives of this study was to associate co-speech gestures with the aspectual analysis of verbs in Russian. The results are that both past-tense perfective and imperfective verbs in Russian co-occur significantly frequently with bounded (stroke phase ended with an accentuated stop followed by a brief hold before retraction) gestures. An analysis also showed that the Russian data are very sensitive to lexical and pragmatic features of the discourse, which affect the quality of gestures.

The given analysis of gestures emphasizes the connection between the category of aspect, the lexical meaning of verbs in the Russian language, and the character of the gesture.

3.4.2 Gestural profiles of verbs and prepositions in Russian

The general description of gestural profiles of verbs in Russian is presented in Chapter 7, Gestural Profiles of Verbs and Roots, of the monograph of E.A. Grishina [Grishina 2017]. Based on the Talmy's terminology on the verb-framed and satellite-framed languages [Talmy 1975], the author showed that prefixes and verb roots in Russian tend to certain directions of movement of hands and head relative to the speaker, as well as to certain trajectories of movement of the hand (see Appendixes 1 and 2).

The author assumed that the gestural support of the verb should not contradict its typological characteristics, i.e., since the direction of movement is transmitted by prefixes in Russian, and that verbs with the same prefix will have similar spatial characteristics. However, the gestures that accompany the verb can have not only lexical meaning, but also grammatical meaning (for example, time). When analyzing gestures, the author suggested that they are the result of a choice of grammatical or lexical parameter, essential for the speaker at the moment, so it can be argued that the results of the study will be significant only statistically.

Grishina noted that, according to the corpus study, all prefixes, which are inherent to the verbs in Russian, differ in gestures. As for the study of the roots of the verbs, the corpus is too small, the gestural profiles of Russian verbs were done for groups of verbs with similar semantic component.

In this study, the author considered three groups of verbs of motion: a group of verbal roots including the semantic component “trajectory, distance” (begat' `to run', vezti `to carry', plavat' `to swim', yekhat' `to ride', etc.) verbal roots with the semantic component “fixed point”, initial or final (vstretit'/vstrechat' `to meet', -zhdat'/-zhidat' `to wait'; nayti/nakhodit' `to find', nachat'/nachinat' `to start', -puskat'/-pustit' `to let'), and verbal roots with the semantic component “downward movement” or “being below” (lagat'/-lozhit'/klast' `to lie down', -lezhat'/-lech' `to lay', -padat'/- past' `to fall', -sadit'/-sazhat'/-sazhivat'/-sidet'/-sest' -stavat'/-stat' `to seat', -stoyat `to stand') and analyzed the distribution of gestures along three axes.

The results of Grishina's research are that the idea of a trajectory and/or the distance in the gesture accompanying the roots is marked either by the movement of the curved arc or by the vector movement of the hand (sometimes the head) from the bottom upwards. Further, the author concluded that verbs that include a “downward movement” component in their semantic structure prefer top-down gestures. Verbs that have “fixed point” components in their roots also tend to have a top-down direction of movement. As for the hand configuration, the author noted that the index finger is used for verbs that have a “trajectory, distance, line” semantic component. Finally, no connection to the transverse axis or head movements have been identified. The author claimed that significant corrections to the proposed picture can be made by the study of the gestural accompaniment of prepositions and spatial adverbs, which should be associated with the spatial characteristics of prefixes.

As mentioned (3.2), gestures can be divided into a reflective point of view of the observer (OVPT, observer viewpoint gestures), and reflecting the point of view of the character speech (CVPT, character viewpoint gestures). Speaking about co-speech gestures accompanying motion verbs, the situation is more complicated in the case of observer gestures (OVPT). The roots of the verbs of motion do not convey the idea of the way of movement at all: regardless of the semantics of the root (to go, to run, to crawl, to fly, etc.), the gestures that accompany the roots do not convey Manner, but the fact of movement at some distance. This is done by moving one's hand along the curved arch path or by raising the hand.

Some comments on Grishina's research on gestural profiles of verbs in Russian are presented in the review [Pereverzevа 2019; 60-69]. The reviewer mentioned that Grishina [Grishina 2017; 268] combined into a group of verbs with a similar meaning the reflexive verbs with the irrevocable ones, which is not always justified.

The following conclusions can be drawn from the above study: first, the prefixes that are characteristic of the Russian verbs differ in gestures; second, the idea of a trajectory and/or distance in the gestures accompanying the roots is transmitted either by the movement of the curved arc or by the vector movement of the hand from the bottom to the top. Third, the verbs that include the “downwards movement” and “fixed point” component in their semantic structure prefer a top-down direction in gestures. Finally, the index finger is used for verbs that include a “trajectory, distance, line” component. No correlations were found on the transverse axis and head movements. In conclusion, the study of the gestural accompaniment of prepositions and spatial adverbs may change the picture presented above.

Some work has already been done to define gestural patterns for prepositions in Russian. The report [Ermolaeva et al.2018] presents the results of the study on the material of MURCO, the purpose of which was to establish a connection between the configurations of active gesturing hand (index finger, fist) and prepositions in Russian in constructions with Russian verbs. However, the researchers did not find a connection between these two parameters. The authors suggested taking as the material not the combinations of specific verbs with prepositions, but prepositional and case forms due to the variety of semantic components of prepositions in Russian. This kind of research also requires more data to obtain representative results.

3.5 Summary

In this section, we gave an overview of major studies in the field of gesture linguistics and described the distinctive features of co-speech gestures. We also reviewed the research methodology, the proposed gesture notation system, and the principles of data treatment.

We presented a widely used typology of verbs of motion in the languages of the world. We have considered in detail the specific characteristics of Russian verbs of motion (which may affect the character of accompanying gesticulation). We looked at the existing classifications of Russian verbs of motion and reviewed the semantics of spatial prefixes and prepositions in Russian. We also presented at relevant cross-linguistic studies that show how in typologically different languages elements of motion event can be differently expressed in the gestural representation.

Our literature review showed that Russian co-speech gestures have not been fully investigated. In the previous studies, verbs were considered separately (not in constructions), and the conclusions are rather general for the large group of verb roots. Moreover, the data analysis was carried out only in the framework of a one source, and the number of examples is not sufficient to draw a comprehensive picture for the Russian language.

Gestural representation of motion events in the Russian language requires additional analysis. In our study, we try to describe in a more complex way the gestural patterns for verbs of motion in the Russian language, by combining data from MURCO, Pear Stories Corpus, and Red Hen Lab database.

4. Methodology

Our work consisted of several parts. First, we identified a set of verbal roots with the semantics of motion we examine, based on the classification of Elena Grishina. After that we found available multimedia corpora: Russian Multichannel Discourse Corpus, Multimedia Russian Corpus, NewsScape database (described in Section 4.1.). We chose a corpus with records of Tales and Talks about Pears for the primary analysis of gestures accompanying verbs of motion. To conduct an analysis, we selected the relevant parameters for the data markup, which include annotation of the prefix, the verb root, the prepositional and case forms, the components of the gesture and the presence or absence of semantic roles in phrase (described in Section 4.2.). Then, we carried out a quantitative (using the chi-square) and qualitative (by analyzing the particular verb roots, and the prepositional and case constructions) analysis of the obtained database. Principles of data analysis are described in Section 4.3.

After defining the link between gesture parameters and the semantics of this or that analyzed unit, we compared our results with the findings obtained by Elena Grishina on the Multimedia Russian Corpus material and supplemented them with our results. Then, we looked for examples in Multimedia Russian Corpus and NewsScape database.

4.1 Material

Elena Grishina's classification included three following groups of verbal roots: a group of verbal roots including the semantic component “trajectory”, a group of verbal roots with the semantic component “downward movement” or “finding below”, and a group of verbal roots with the semantic component “fixed point”.

Since our task was to identify some of the main gestural patterns for the prototypical verbs of motion in Russian, we did not include some of the roots that E. Grishina investigated because they express the so-called “fictive” motion (a term of L. Talmy), e.g. smotret'; `to look, to watch'), and here occurs rather a metaphorical expression of motion, which can complicate the description of the gesture. As in the last group of verbal roots with the semantic component “fixed point”, most verbs express this type of motion, from this group we have only considered the root-puskat'/-pustit' `to throw'. In our sample, the only verb with such a root was the verb pair spustit'sya/spuskat'sya `to get down, to descend'. This verb is lexicalized in the Russian language, and has a complex meaning in which we can distinguish the semantic component 'downward movement', so we also included it in the second group - verbs having in their root the semantic component 'downward movement'. Furthermore, we separately observe the reflexive verbs, as reflexivity could affect gesticulation in cases when their gestural patterns were different from non-reflexive verbs.

The following list of verbal roots with the semantic “trajectory” includes the roots we examine: -begat'/-bezhat' `to run', -birat'/-brat'/vzyat' `to take', -vezti/-vozit' `to carry', -vernut'/-vratit'/-vrashchat' `to rotate', -vesti/ vodit' `to lead; to drive', -gnat'/-gonyat' `to chase', -davat'/-dat' `to give', -yezdit'/-yezzhat'/-yekhat' `to drive', -yti/-khodit' `to go', -kidat'/-kidyvat'/-kinut' `to throw', -lezat'/-lezt'/-lzti `to climb', -letat'/-letet' `to fly', -nimat'/-nyat' `to raise', -rasti/-rashchivat' `to grow', -taskat'/-taskivat'/-tashchit' `to pull'.

Among the verbal roots that have the semantics of “downward movement”, we examine such verbal root as -puskat'/-pustit' `to throw', lagat'/-lozhit'/klast' `to lie down', -lezhat'/-lech' `to lay', -padat'/- past' `to fall', -sadit'/-sazhat'/-sazhivat'/-sidet'/-sest'/stat' `to sit', and also a verbal root stoyat' `to stand'.

So, we considered two groups of verbs: a group of verbal roots including the semantic component “trajectory”, a group of verbal roots with the semantic component “downward movement”.

4.2 Corpora

4.2.1 Russian Multichannel Discourse: Tales and Talks about Pears

The research group at the Institute of Linguistics of the Russian Academy of Sciences, headed by A.A. Kibrik, has created a corpus on which they research in the field of multimodal communication. It consists of communicative episodes from the discussion of the adaptation of the film "Tales and Talks about Pears" (created in the 1970s in University of California by a research group led by W. Cheif). Communicators have fixed roles (Commentator, Listener, and Storyteller), but the free character of communication is preserved. The corpus consists of 24 records of about 9 hours in total, made in 2015, and 16 records of about 6 hours, made in 2017, with several types of markup in ELAN (vocal annotation, kinetic annotation, oculomotor annotation) [https://multidiscourse.ru/corpus/].

Speaking about the notation system, the authors created own original technique to annotate hand gestures, which is based on the segmentation of the flow for periods of immobility and individual movements, which then form functional units - individual gestures, adapters, posture changes, and groups of such units. Then, the characteristics of these units are specified. The annotation of head gestures is based on the same principles [https://multidiscourse.ru/corpus/].

For our study, we analyzed three sessions that can be found in the public domain that were recorded in 2015. For each session, there were three recordings of each participant of the experiment. We used records of 9 participants and, accordingly, analyzed 9 video clips lasting 25, 18 and 19 minutes.

4.2.2 Multimedia Russian Corpus (MURCO)

Russian Multimedia Corpus (MURCO) is a subcorpus of the National Russian Corpus, founded in 2010. Most of the multimodal projects belong to the field of psychology or psycholinguistics, and corpora under analysis usually contain the data received from the experiment. However, MURCO is organized differently. MURCO began to be formed precisely with cinematic speech, due to its availability, as well as the fact that full and accurate transcriptions of the movies had already been obtained during the preparation of Corpus of Spoken Russian (a part of National Russian Corpus). Soviet movies are the largest part of the corpus. The second category of the data consists of oral public speech recordings such as television programs, documentaries, reports, lectures, and other samples of poorly prepared oral public speech. The corpus also includes samples of theatrical speech (recording of radio and television performances). The essence of the approach to the analysis of accompanying gestures proposed by the author is that one deals with extensive corpus material, namely, Russian texts, aligned with the corresponding audio and video fragments [http://www.ruscorpora.ru/old/search- murco.html].

The main unit in MURCO is a click: a couple of clips plus the text of the transcript of the speech available in the clip. Clips containing text material are referred to as clicksts (or clipotexts), and video clips containing only gesture material are referred to as clips themselves. A small part of MURCO is marked with different types of speech actions and gestures. As for the MURCO annotation principles, the authors have developed a system of parameters describing the gesture component of the clip and clip-text. One can highlight several characteristics of gestures that will be taken into account in the search: the active or passive body organ, the direction of movement of the body, the multiplicity and type of gesture, the type of communication action, etc. [http://www.ruscorpora.ru/old/search- murco.html].

At the moment, the corpus includes 193,635 clips, 4,540,588 tokens. If one search head gestures, one gets 1437 clips (1598 entries). As for the hand gestures, one can work with 1211 clips (1377 entries).

Initially, we wanted to conduct a primary analysis on the material of MURCO, but the problem with film clips is that often, one cannot see the character at the time of pronouncing the phrase and track his gestures, which to some extent makes it difficult to work with the materials. Besides, there are not so many entries with verbs of motion accompanied by co-speech gestures to make a quantitative analysis of motion verbs in MURCO (that is why Elena Grishina's analysis includes large groups of verb roots). That's why we did the analysis on "Tales and Talks about Pears" and then looked at some examples in MURСO.

4.2.3 Red Hen Lab Database (NewsScape)

The International Distributed Little Red Hen Lab is a community of researchers working in the field of multimodal communication. Red Hen Lab database contains over 500,000 hours of audio-visual files. The original component of the Red Hen Lab database, NewsScape, is the UCLA News Archive, which was originally developed by the UCLA Communications Department.

This archive also includes records of TV news from such Russian channels as TVC, Channel 1, and NTV. Search in news works only by keywords (no search by gestures). However, every news video has transcripts that make it easier to find the context we need. We used the search results of this resource to confirm selected gestural patterns of motion verbs in Russian, because TV programs show the free, unprepared speech of people who present news (which affects the accompanying gesticulation and making it free-flow, as opposed to behaviour in the conditions of the experiment).

4.3 Data treatment

As mentioned before, the first corpus we analyzed was Tales and Talks about Pears. The open data included transcripts of records where elementary discursive units (quanta of oral discourse) were marked. Also, this corpus has a kinetic annotation in ELAN, which also includes co-speech gestures annotation, but we did not use it, as it was organized in a rather complicated way (several levels of annotation), and did not contain all the parameters of interest to us (for example, the trajectory of gesture).

At the first step of work with the data, we have identified all the motion verbs occurrences (in the transcripts), and marked up the grammatical and semantic features of the verb for each occurrence. After that we have watched each video twice. During the first watching of each video we noticed the presence or absence of a gesture in the process of pronouncing a clause containing a verb of motion. In other words, the gestures we considered had to be minimally co-occurring with these clauses. In the second (and further, if necessary) watching, the relevant parameters of the co-speech gesture were annotated.

We encoded the video timeline (e.g.: 22С, 387,55) and full context (e.g.: mal'chik padayet s velosipeda `boy falls off the bike'), a verbal root, as well as presence or absence of specific prefix, preposition or adverb. For each entry, we annotated the grammatical features of the verb, such as aspect, reflexive, transitivity, and the presence of a direct or indirect object in the phrase. In the markup, these parameters looked as follows: aspect(perf/imp), transitivity (trans/intrans), reflexive (yes/no), direct/indirect object (gen/dat/acc/instr/prep). We also indicated verbal root semantics (trajectory/downward movement) as well as explicitly expressed semantic roles in the phrase (Theme/Patient, Recipient, Path, Location, Manner, Container, Goal, Time, Source). Further, we noted the presence or absence of a gesture and specific parameters of the gesture.

Coding and annotation of gestures are usually done in ELAN. The authors use a combination of parameters when annotating gestures. Some researchers [Alibali, Martha W., et al 2001] argue for a double coding (when two researchers annotate the same data) of the material because people may understand and classify gestures in a different way. We used a form-based notation system for gestures which was presented in [Bressem 2013]. Primarily, important components of semiotic analysis of gesture include the form of gesture, its sequential structure, the context in which gesture is used, and the distribution of constructions. The researchers also propose to analyze hand shape, orientation (palm up/palm down/palm vertical/palm lateral), movement pattern, and position in gesture space. Regarding the motion patterns, it is recommended to differentiate straight movement, arced movement, circle, spiral and s-line movements. Next, when describing the direction of the gesture it is important to distinguish between three axes: the horizontal(lateral) axis (right/left), the vertical axis (up and down) and the sagittal axis (away and towards the body). It is also possible to take into account the velocity, the proximity to the centre of the gesture space (in the diaphragm area), the right or left-handed gesture is, one or two hands are involved in the process of gesticulation, as the authors [Bressem 2013] noted.