Co-speech gesticulation accompanying verbs of motion in Russian

For our study, we decided to consider the orientation of the direction of the gesture regarding three axes, the trajectory of the gesture, the orientation of the palm, the direction relative to the gesture space (egocentricity of the gesture), as well as the type of viewpoint of person while performing a gesture (observer or character), the right or left-handed gesture is, and the repeatability of the gesture (the movement was made once or several times). In the markup, these parameters looked as follows: multiplicity (one/mult), gesture space (to/out/neutral), sagittal axis (forward/backward), lateral axis (right/left), vertical axis (up/down), trajectory (arc/straight/other), hand (right/left/both), palm (open palm/iconic/index finger/thumb/fist), viewpoint (CVPT/OVTP).

In general, we focused on the representational gesture stroke, as this part of the gesture contains all the relevant semantic information. So, all gestural patterns that we identify are related to the gesture stroke. We selected only gestures depicting the semantic elements of Path or Manner (i.e. those in which we could identify the trajectory of a gesture and orientation on the three axes). Unclear gestures were excluded from our analysis. Since we were primarily interested in identifying gestural patterns related to the semantics of the units we were investigating, we did not take into account verbs and clauses under negation, as this could affect the character of the gesture.

4.4 Data analysis

Investigating co-speech gestures, the researchers use statistical tools for the analysis of accompanying gesticulation, such as chi-square test [Grishina 2017], ANOVA and t-tests [Kita, O?zyu?rek 2007]. Å. Grishina [Grishina 2017] emphasized that the identification of semantic components of gestures has only a statistical value, as the gesture may be ahead of the corresponding element of speech, or occur after the pronunciation of the corresponding element of speech . Grishina suggested that after a quantitative analysis is carried out, a qualitative analysis should also be conducted and each example should be studied individually. Thus, while processing our data, we also used the chi-test to confirm the relationship between the two parameters (e.g., the trajectory of a gesture and a verb prefix). After that we looked at specific examples and described the particular cases (constructions with spatial and temporal adverbs, semantic parameters of certain verbal roots and the character of an accompanying gesture, and so on).



5. Results

5.1 Pear stories

In this section, we present the results of our work with the corpus of Pear stories. We describe the database we have been working with, and give the conclusions about how gesture parameters relate to the semantics of prefixes of motion verbs, to the semantics of verbal roots, and to the semantics of prepositions (in combinations of prefixed and non-prefixed motion verbs with prepositional and case forms).

5.1.1. Material

After annotating the data, we obtained 468 relevant entries of motion verbs, of which 194 entries were accompanied by gesticulation, and 274 entries were not accompanied or did not match our parameters. Table 1 shows the distribution of prefixed and non-prefixed verbs in our database.

Table 1. Prefixes



As can be seen here, both prefixed and non-prefixed verbs of motion are equally accompanied by co-speech gestures, and the most frequent prefixes in our sample are s-, u-, pod-, so-, za-. Further, we consider the distribution of verbal roots.

Table 2. Verbal roots

Thus, in our sample, the more frequently occurring group of verbs of motion accompanied by gesticulation is the group of verbs with the semantic component of the root “trajectory”, and the less frequent group of verbs of motion accompanied by gesticulation is the group of verbs with the semantic component of the root “downward movement”. Below, we look at the most frequent verbs in our sample (padat'/past', kladyvat'/klast', puskat'/pustit', brat'/birat', yezzhat'/yekhat', idti/khodit') and analyze if there are any gestural patterns for these roots. We also check the patterns previously identified for these groups of verbal roots.

The next two tables concern the presence or absence of a gesture in case the verb is transitive or intransitive and whether it is perfect or imperfect.

Below (Section 5.1.4), we look at gestures in more detail in the following constructions, which have occurred several times:

1. pro- + motion verb + mimo

2. s- + motion verb + s + Gen

3. pod- + motion verb + Acc

4. so- + motion verb + v + Acc

5. so- + motion verb + Acc

6. motion verb + v + Acc (1)

7. motion verb + v + Acc (2)

8. motion verb + na + Acc

9. motion verb + na + Prep

10. motion verb + navstrechu

11. motion verb + k + Dat

12. motion verb + pî + Dat

13. motion verb + Acc

We also analyze in detail examples with spatial adverbs (those adverbs are in bold font in the table) to check if the semantics of adverb is reflected in any of the examined co-speech gestures parameters.

5.1.2. Prefixes

Next, we examine the distribution of prefixes by parameters of co-speech gestures and check if there are any links between the parameters. To determine if there is a relationship between the two parameters, we used a chi-test (which is used to test the hypothesis that the empirical distribution corresponds to the assumed theoretical distribution).



Table 6. Distribution of prefixes by parameter “Multiplicity of a gesture”



The parameters are not bound (p-value > 0.05): Table 6; Table 8.1.; Table 8.2.; Table 10. The parameters are bound and the distributions are accurate (P ? 0.05) for Table 7 (Distribution of prefixes by parameter “Egocentricity”); Table 8.3. (Distribution of prefixes by vertical axis); Table 9 (Distribution of prefixes by trajectories); Table 11 (Distribution of prefixes by parameter “Palm configuration”). In these tables, significant (more than 2.5) deviations from the normal distribution are highlighted in bold. It was not possible to establish a link between the following parameters in our sample: prefix and multiplicity of a gesture, prefix and sagittal axis, prefix and lateral axis, prefix and left/right handed gesture is.

Now we consider in detail the parameters for which the distribution is significant and compare the obtained results with the findings of E. Grishina.

In our sample, the sagittal and lateral axes are poorly represented among the vector characteristics of the prefixes (which corresponds to the findings of Grishina). As the distribution is accurate only for the vector characteristics of the prefixes regarding the vertical axis, we will consider these results below. According to Grishina, prefixes v-, voz-, do-, na-, î-, pod- and pro- are distributed over the vertical axis in their gesture accompaniment. We were only able to draw conclusions about the prefix pod- that has an upward orientation, - and here our data match the findings of E. Grishina. As for the other prefixes, we were unable to identify significant deviation from the standard distribution of data in our sample. Also, the prefix za- has an upward orientation (but in the classification of E. Grishina this prefix does not show any tendencies along the axes. We have also concluded that the verbs with prefixes s-, so- are accompanied by the gesture with downward direction (in the classification of Grishina the prefix s-, is oriented to the left side). Most likely, our findings may be related to the fact that prefixes s-, so- are part of the verb constructions, which have the meaning of downward movement, and the prefix za- is a part of verb constructions which have the meaning of upward movement (and this semantic component should be expressed in verbal root or in the propositional form and it "outweigh" the proposed orientation of the prefix in relation to the vector distribution along the axes). Considering the distribution of gestures towards the vertical axis for non-prefixed verbs of motion, the frequency is the same for downward and upward hand movement, which also shows that not only the semantics of the prefix affects the up or down orientation of the gesture: it may be influenced both by the semantics of the verb root and by the presence of arguments having different semantic roles in the construction with one or another verb of motion.

Next, we look at the distribution of prefixes by trajectories. In the study of Grishina the distribution by trajectories was not significant for most prefixes. However, we were able to identify some of the patterns. Prefix za- is characterized by the straight movement of the hand in a straight line or by other trajectories (e.g., a spiral). For prefixes po-, pod-, s-, so-, a curved arc is a typical trajectory. As for the non-prefixed verbs of motion, they also prefer a curved arc as a way of expressing the trajectory when the verb of motion is accompanied by a gesture.

Regarding the parameter “Egocentricity”, we can conclude that prefixes za-, pod-, pro- are characterized by a neutral position of hands in gesture space, which means they are not egocentric. Further, verbs which have prefixes po, u-, ot-, s- will be accompanied by gestures that will be directed "away from the speaker". This partly contradicts Grishina's conclusions: prefixes za- and s- are characterized as egocentric. The author explained this by the fact that prefixes include a semantic component “self”, which is reflected in the nature of the gesture. As we can see from our data, egocentricity is not always typical for these prefixes (and it seems to depend on the context in which a verb with such a prefix is used). The motion verbs with the prefix so- accompany by co-speech gestures that will be performed towards the diaphragm (Elena Grishina also mentioned that for the prefix so- the typical trajectory is “to the center”). However, most of the examples in our sample with the prefix so- are verbs of motion that have at their root the semantic components of “movement down” and “self”, and it seems that the prefix only complements these meanings, which is expressed in the character of the co-speech gesture.

Finally, regarding the distribution of prefixes by “palm configuration”, we can say that prefixes pro-, s-, so- prefer an open palm configuration, prefix u- and a major part of non-prefixed verbs of motion prefer a straight palm configuration, while prefixes s- and so- are characterized by the expression of an iconic gesture with the palm. Our conclusions do not correlate with Grishina's conclusions concerning gesture patterns of palm configuration. Configurations such as index finger, thumb and fist do not prevail in our sample. This may be because, during the discussion of the film, speakers did not need to point out objects when describing what was happening in the film.

To sum up, we analyzed four parameters for which distributions were accurate in our sample: gesture space, vertical axis, trajectory, and palm configuration. Prefixes in Russian have a lot of meanings. We can make conclusions on gestural accompaniment of the prefixed verbs only for the examples with verbs of motion, where the meaning of the prefixes includes certain semantic components “self ”, “upward motion” and “downward motion”. A large number of our findings do not coincide with those of E. Grishina (in the table below are in bold) - and only the conclusions concerning the upward orientation of verbs of motion with prefix za-, verb orientation with the prefix so- towards the speaker, and the orientation of verbs of motion with the s- prefix in the direction “out of the gesture space” match up. This allows us to conclude that the characteristics of the gesture is influenced not only by the semantics of the prefix. The semantics of the verb root, as well as the presence of certain semantic roles in the construction with one or another verb of motion, may affect the character of the gesture, so it is necessary to consider the constructions, and not the prefixes individually.



Table 13. Gestural patterns for prefixed and non-prefixed motion verbs

5.1.3. Verbal roots

Next, we look at the distribution of verbal roots in relation to the parameters of co-speech gestures (in large groups of verbal roots, sorted by the semantic components of the root).

Table 13. Distribution of verbal roots by parameter “Multiplicity of a gesture”

Table 14. Distribution of verbal roots by parameter “Egocentricity” (gesture space)



The parameters are bound and the distributions are accurate (P ? 0.05) for Table 14 (Distribution of verbal roots by parameter “Egocentricity” (gesture space)), Table 15.3 (Distribution of verbal roots by vertical axis), and Table 16 (Distribution of verbal roots by trajectories). The conclusions are presented in Table 19.1. In these tables, significant (more than 2.5) deviations from the normal distribution are highlighted in bold. The link between groups of verbal roots and the trajectory of the gesture, between groups of verbal roots and the distribution towards the vertical axis, and between groups of verbal roots and relation to the gesture space is determined. Such parameters of gesture as multiplicity, sagittal and lateral axis distribution, right/left handed gesture, and palm configuration are not connected to the semantics of verbal roots, based on our sample.

First, co-speech gestures that accompany verbs with the meaning “trajectory” may equally have an upward or downward direction. Second, verbs with the semantic component “downward movement” tend to the downward direction of the hand. These conclusions correlate with the findings of E. Grishina.

However, some of the verbs with the semantic component “downward movement” at the root are accompanied by a gesture that is directed upwards, and it is particularly interesting to look at these counter-contexts. It turns out that a significant part of such examples is the verb sidet and reflexive verb sadit'sya `to sit down'. The verbal root sadit' has a prototypical meaning 'the object moves to a seating position'. Without a doubt, this root has a semantic component “downward movement” (which can be proved by the large number of figurative meanings that this verb has), but in our examples, it has a meaning “to move to a sitting position from lying down”, and thus the movement of the subject occurs upwards, which is accompanied by a gesture with similar direction. In examples with verb sidet', the construction is accompanied by a spatial adverb: sidet' naverkhu `to sit upstairs'. This allows us to conclude that the character of gesture is determined by the presence of spatial adverbs and the context rather than by the semantics of verbal root.

The typical trajectories of gestures accompanying verbs of motion that have the semantic component “trajectory” at the root are a straight line and a curved arc (there are also examples of other trajectories, such as a spiral, in some contexts, that mark the Manner component). Next, verbs that have the semantic component of the root “downward movement” tend to the trajectory “arc”, and, less often, the trajectory of the hand is a straight line. We will look at different examples and determine what influences the distribution. Apparently, the contexts in which the trajectory is a curved arc are mainly contexts with such verbal roots as kladyvat'/klast'/lozhit, which indicate the displacement of an object into something, i.e. in such contexts a semantic role "Container" is explicitly expressed (for example, polozhit' v korzinu `to put into the basket') - and we may assume that the trajectory “arc” represents the presence of the initial and final point of location of the object being relocated. At the same time, in contexts where the hand is moving downwards by a straight line, the ending point is the horizontal surface which is expressed explicitly or implicitly: klast' na zemlyu, upast' (na zemlyu) `to lay on the ground, to fall (on the ground)'. From this we can conclude that the trajectory of gesture accompanying verbs having in their roots a semantic component “downward movement” is significantly influenced by an argument (implicitly or explicitly expressed), which denotes the ending point at which the moving object appears after the motion event.

Further, a significant number of verbs having semantic component of the root “downward movement”, are accompanied by a gesture directed from the speaker. These examples include, for instance, all entries of the verb padat'. This can be explained by the fact that these gestures occur from the observer's point of view, so there is no “self” component in the semantics of the construction, and these gestures are not “egocentric”. However, it is also interesting to look at the reverse examples where the gesture is self-centered: skladyvat' grushi v fartuk, klast' v karman `to put pears in an apron, to put pears into a pocket'. It is interesting that in such examples gestures occur from the point of view of the character, and Container, which is explicitly expressed in the phrase, is directly related to the Subject (the object of clothing), and hence the semantic component of "self" appears and affects the character of a gesture.

Verbal roots with the semantics of “trajectory” can be accompanied by gestures directed towards the speaker's diaphragm as well as by gestures that are directed from the speaker. We will look at different examples and determine what influences the distribution. Among the verbal roots, that are accompanied by gestures directed towards the speaker's diaphragm, the most frequent are brat'/birat'/sobirat'/vzyat': in these contexts, gestures occur from the point of view of the character, and, furthermore, the prototypical meaning of the root brat' includes the semantic component “proximity to self”, and these two parameters may significantly influence the character of gestures. Further, gestures that are directed away from the speaker may be affected by the following: the presence of prefix s-, for which such a pattern is clearly identified (slezt? `to descend', sletet? `to fly down') or the presence of a spatial adverb that has the semantics of distancing from the starting point (for instance, uyekhat' vdal', poyekhat' dal'she `to drive away'). This pattern is also common for the verb pair dat'/davat', where the initial point (Agent) and the endpoint (Recipient) are marked when the object is being moved.

Accordingly, the following gestural patterns for verbal roots can be identified:

Table 19.1. Gestural patterns of verbal roots

Next, we will look at examples with frequent verbal roots in order to identify what influences the differences in gesture parameters.

1. Padat'/past'

The first verb pair we will consider is padat' /past'. The root contains the semantic component "downward movement". All examples are non-reflexive verbs in sentences with no negation. The gestures are reproduced from the observer's point of view.

In all of the occurrences, non-prefixed padat'/past' and their prefixed derivatives are accompanied by gestures with common characteristics. The following constructions are represented: padat' + na + Acc, padat' + s + Gen, vy + padat' + iz + Gen, u + past'. Gestures in 11 out of 12 entries of our database were performed in the opposite direction from the diaphragm), i.e. these verbal roots are not characterized by the egocentricity of gestures.

Further, all occurrences are characterized by the same top-down direction of gesture along the trajectory of the arc (10 entries) or along the straight line (2 entries), and for all occurrences - the palm configuration is open palm. Interestingly, this verb pair has not been represented in any way with regards to the lateral axis.

2. Klast'/kladyvat'

The next pair of verbal roots root is klast'/kladyvat', that, like padat'/past', contains a semantic component “downward movement”. All examples are non-reflexive verbs in sentences with no negation. In all occurrences, the non-reflexive verb klast' and the prefixed derivatives of -kladyvat' are accompanied by gestures with common characteristics. The following constructions are presented: pere + kladyvat'+ v + Acc, s + kladyvat'+ v + Acc, klast'+ v + Acc, klast'+ Acc. In all examples, co-speech gestures were made from the character's viewpoint. For these verbs, as for the other verbs with the semantic component “downward movement” in the root, the gesture is not represented on the sagittal or lateral axis. Further, the majority of gestures (10 out of 13 entries) are made along the “arc” trajectory. For most (9 out of 13 entries) gestures accompanying verbal roots kladyvat'/klast' the configuration of the palm is an iconic gesture. The palm configuration expresses the size or shape of an object that is placed somewhere, - specifically in our contexts there were pears, and we can assume that this is the analogue of verb agreement with the object expressed in the gesture. Interestingly, almost all of the entries (12 out of 13 entries) were accompanied by a gesture directed from up to down. In the example where the hand moves upwards, the verb perekladyvat' `to displace, to shift, to re-lay' i.e. here the character of gesture is affected by the location of the endpoint.

Most of the gestures (8 out of 13 entries) were directed in the opposite way from the diaphragm, i.e. these verbal roots are also not characterized by the gestures' egocentricity. However, there were 4 entries with these verbal roots where gestures were directed towards the speaker. In such examples, as has already been said, gestures are made from the character viewpoint, and Container, that acts as the end-point, is located "on the character" (the object of clothing, apron or pocket) and explicitly defined by the speaker.

3. Brat'/birat'

Now we turn to verbal roots with the semantic component of “trajectory”. We will consider the verbal pair brat'/birat'. All examples are verbs in sentences with no negation. In all occurrences, a non-prefixed verb brat' and the prefixed derivatives of brat'/birat' are accompanied by gestures which also have common characteristics. The following constructions are represented: na + birat' + Acc, so + birat' + Acc, so + birat' + v + Acc, brat' + Acc + iz + Gen. Most of the examples are gestures reproduced from the point of view of the character (30 entries), and the rest (5) are gestures performed from the point of view of the observer, accompanying the verb zabiratsya `to climb up'). We will look at different examples and determine what influences the distribution.

Gestures are represented on the vertical axis: the movement of the hand can be either downwards (17 out of 35 entries) or upwards (13 out of 35 entries), and the trajectory of a gesture in most of the occurrences (30) is an arc. The direction of movement on the vertical axis can be affected by the Path parameter (the position of the objects with respect to each other, including the start and endpoint). This is illustrated by the presence of the prefix so-, which was characterized by a downward movement (sobirat' grushi `to pick pears'), or the presence of a container indicating the initial (brat' iz korziny `to take from the basket') or the final (sobirat' v korzinu `to collect in the basket') point is accompanied by an upwards gesture.

The verb pair brat'/birat' is the only one considered where the egocentricity of gestures is evident: out of 35 entries, 25 gestures are directed towards the speaker's aperture. This may be due to the fact that most examples with these verbal roots are gestures made from the viewpoint of the character of the action. An example where the hand is directed in the opposite from the speaker is the construction brat'+ Acc + iz + Gen (brat' iz korziny `to take from the basket'), where a gesture marks the starting point expressed by Container.

Like the gestures accompanying the verbal roots klast'/kladyvat, for brat'/birat' (27 out of 35 entries) the typical configuration of the palm was the reproduction of an iconic gesture (the palm configuration expresses the size or shape of an object that is placed somewhere, - specifically in our contexts there were pears or baskets).

There were also three examples of verbs in reflexive form, in the construction za + birat + sya + po + Dat (zabirat'sya po lestnitse `to climb the stairs'). These three examples were characterized by the following parameters: the upward direction of gesture, the trajectory of the spiral, and also a gesture was repetitive: here we see how Manner and Path can be expressed in the character of gesture.

4. Yekhat'/yezdit', idti/khodit'

Next, we will discuss gestural patterns for the two most frequent verbal pairs: idti/khodit' (28 entries) and yekhat'/yezdit' (31 entries) and their prefixed derivatives. All entries of idti/khodit', yekhat'/yezdit' and their prefixed derivatives are accompanied by gestures, which have some common characteristics. All examples are gestures from the observer's point of view. Co-speech gestures that accompany these verbal roots can be both repetitive and single, and no relationship was found for this parameter; however, multiple gestures accompany idti/khodit' more often (9 out of 28 entries), then yekhat'/yezdit' (3 out of 31 entries). Multiple gestures occur in contexts such as yekhat' navstrechu `to go towards somebody/someone', idti v raznyye storony `to go in different directions', prokhodit' mimo `to pass by', idti mezhdu `to go between two objects'- in general, in contexts where Path is explicitly expressed in a construction, and also in contexts where the number of participants and their vectors of motion exceeds one.

Gestures accompanying such verbal roots as idti/khodit' were performed equally by the right hand (11), left hand (10) and both hands (in contexts where there is more than one moving Object and their trajectories differ, for example, idti v raznyye storony storony `to go in different directions'), unlike yekhat'/yezdit', which were characterized by the reproduction of gestures by the right hand (indicates the movement of one Object). For both verb pairs, the most frequent trajectory was a straight line. In the examples where the trajectory of gesture is an arc, we find prefixed verbs with pod- or po- prefixes, for which this trajectory (arc) has been identified as predominant, and also in constructions with a spatial adverb mimo: yekhat' mimo dereva `to ride past the tree', where the spatial adverb has the meaning “past (of movement in relation to an object)”, which appears to be expressed in the trajectory of a gesture.

The following constructions with these verbal roots are represented: pri + yezzhat' + k + Dat, pro + yezzhat'/yekhat'/khodit' + mimo + Gen, pro + yezzhat' + po + Dat, u + yezzhat'/yekhat', po + yezzhat' + na + Prep, yekhat' + navstrechu + Dat, yekhat'/idti + po + Dat, idti + mimo + Gen, po + yti + v + Acc, pod +khodit' + k + Dat.

The interesting fact is that, unlike all the previous verbs considered, co-speech gestures that accompany verb pairs idti/khodit' and yezzhat'/yekhat' are not distributed along the vertical axis, and even the presence of prefixes that are oriented along the vertical axis does not cause it to be expressed in gesture.

Next, we consider the typical palm configurations: open and straight palm are prevailing for idti/khodit' (respectively, 14 and 15 out of 31 entries), and for yezzhat'/yekhat' (respectively, 14 and 9 out of 28 entries) - so here for the palm configuration no special meaning can be clearly identified (as opposed to brat'/birat', where a palm configuration may be used to express the shape or size of the object).

5. Puskat'/pustit'

Finally, we look at the verbal roots puskat'/pustit' (13). In our sample, they were represented by a verb pair of reflexive verbs spuskat'sya `to descend'/ in sentences with no negation. In all occurrences, verbs are accompanied by gestures with common characteristics. In 11 entries, the represented verb is in imperfect, in 2 entries in perfect. The following constructions are presented: s + spuskat' + sya + s + Gen, s + puskat' + sya+ po + Dat. For all entries, gestures are performed from the observer's point of view. Interestingly, all entries are accompanied by a top-down gesture, and this verb is not represented on the sagittal or lateral axis. The most frequent configuration of the palm is the open palm (9 out of 13 entries), which, like for the verbal roots yezzhat'/yekhat' and idti/khodit', seems to represent only a meaning of the moving of the object at some distance, i.e. the fact of movement itself.

We can distinguish the following gestural patterns for particular verbal roots:

Table 19.2. Gestural patterns of verbal roots



5.1.4 Constructions

Here we proceed to consider the most frequent constructions and highlight gestural patterns for them.

1. pro- + verb + mimo, e.g., proyekhat' mimo `to drive past'

The gesture is performed from the observer's point of view. The prefix pro - is neutral regarding the trajectory of the gesture, orientation along axes, and its location in gesture space. This construction is characterized by the presence of the spatial adverb mimo that has the meaning “past (of movement in relation to an object)”, which appears to be expressed in the trajectory of a gesture, a curved arc. As for the vector distribution in the axes for this construction, the movement of the hand is directed left or right. This most likely depends on the direction of movement of the Object relative to the picture in the movie (i.e. the orientation of the gesture depends on where the starting and ending point is from the observer's point of view).

2. s- + verb + s + Gen, e.g., slezt' s velosipeda `to get off the bike'

The gesture is performed from the observer's point of view. Regarding this construction, we can notice the following gestural patterns of the prefix s-: the gesture typically performed by a downward orientation along the vertical axis, and the direction of the gesture is away from the speaker, so this construction is accompanied by a top-down hand movement away from the diaphragm. Most likely, the presence of a paired preposition s reinforces the gestural patterns of the prefix.

3. pod- + verb + Acc, e.g., podnyat' shlyapu `to raise a hat'

The gesture is performed from the perspective of the character. The prefix pod- has an upward orientation along the vertical axis, so this construction is accompanied by an upward hand movement. The typical configuration of the palm is the reproduction of an iconic gesture (the palm configuration expresses the size or shape of an object that is placed somewhere, - specifically in our contexts there were pears or baskets).

4. so- + verb + v + Acc, e.g., sobirat' v korzinu `to collect in the basket'

The gesture is performed from the perspective of the character. The prefix so- has a downward orientation along the vertical axis, so this construction is accompanied by a top-down hand movement. The typical configuration of the palm is the reproduction of an iconic gesture (the palm configuration expresses the size or shape of an object that is placed somewhere, - specifically in our contexts there were pears or baskets). As it has already been said, for verbal roots like brat'/birat', the movement of hands in the direction of the speaker's aperture is typical (since the endpoint of the moved object is connected with the speaker and the gesture is self-centered).

5. so- + verb + Acc, e.g., sobirat' grushi `to pick pears'

The gesture is performed from the perspective of the character. The prefix so- has a downward orientation along the vertical axis, so this construction is accompanied by a top-down hand movement. The typical configuration of the palm is the reproduction of an iconic gesture (the palm configuration expresses the size or shape of an object that is placed somewhere, - specifically in our contexts there were pears or baskets).

6. verb + v + Acc (1), e.g., idti v storonu doma `to go towards the house'

The gesture is performed from the observer's point of view. Regarding this construction, we can notice the following patterns: the gesture is directed from the speaker, and the trajectory of such a gesture would be a straight line. This may be due to the fact that the endpoint is explicitly expressed here.

7. verb + v + Acc (2), e.g., klast' v korzinu `to put in the basket'

The gesture is performed from the perspective of the character. As for this construction, the movement can be directed either from the gesture space or towards the speaker. This may be due to the fact that the endpoint is explicitly expressed here. The typical configuration of the palm is the reproduction of an iconic gesture (the palm configuration expresses the size or shape of an object that is placed somewhere, - specifically in our contexts there were pears or baskets).

8. verb + na + Acc (1), e.g., padat' na zemlyu `to fall on the ground'

The gesture is performed from the observer's point of view. If the construction contains a root that has a semantic component "downward movement", so the gesture accompanying this construction is directed downwards from the speaker. It would be interesting to consider whether the trajectory of the gesture will change for examples of a non-typical fall as “to fall on the back”, “to fall on the side”, where Path changes.

9. verb + na + Prep, e.g., yekhat' na velosipede `to ride a bike'

The gesture is performed from the observer's point of view. The gesture is directed out the gesture space. In this construction, Manner component is explicitly expressed. If the direction of movement is not important, the direction of the gesture is forward and the hand moves in a straight line. The orientation of movement along the lateral axis (left/right) may depend on the direction of movement of the Object relative to the picture (i.e. the orientation of the gesture depends on where the starting and ending point is from the observer's point of view).

10. verb + navstrechu, e.g., yekhat' navstrechu `to drive towards someone/something'

The gesture is performed from the observer's point of view. As for this construction, the movement of two hands from different angles in a straight line on the lateral axis in the direction of the speaker's diaphragm is typical. The aperture here is the "meeting point" for the description of two objects moving simultaneously towards each other.

11. verb + k + Dat, e.g., podoyti k derevu `to go up to the tree'

The gesture is performed from the observer's point of view. As for the vector distribution in the axes for this construction, the movement of the hand is directed left or right. This most likely depends on the direction of movement of the Object relative to the picture (i.e. the orientation of the gesture depends on where the ending point is from the observer's point of view).

12. verb + pî + Dat, e.g., idti po doroge `to go on the road'

The gesture is performed from the observer's point of view. The typical configuration of the palm is the reproduction of an open palm, and it seems to represent only a meaning of the moving of the object at some distance, i.e. the fact of movement itself. The gesture is directed from the speaker, out of the gesture space.

13. verb + Acc, e.g., dat' grushi `to give pears'

The gesture is performed from the perspective of the character. As for the vector distribution in the axes for this construction, the movement of the hand is directed up or down. This most likely depends on the direction of movement of the moving object (i.e. the orientation of the gesture depends on where the starting and ending point is from the observer's point of view). The typical configuration of the palm is the reproduction of an iconic gesture (the palm configuration expresses the size or shape of an object that is placed somewhere, - specifically in our contexts there were pears or baskets).

14. Spatial adverbs

The following gestural patterns are typical for constructions with spatial adverbs: the semantics of the adverb determines the orientation of the gesture with respect to the lateral and vertical axis. Constructions with such adverbs as vverkh/naverkh/naverkhu will be accompanied by a gesture directed upwards, constructions with such adverbs as vniz/vnizu will be accompanied by a gesture directed downwards respectively. Further, for a construction with an adverb mimo, the direction of the gesture will depend on the trajectory of the object being passed by. The construction with the adverb navstrechu is characterized by the movement of two hands from different angles in a straight line on the lateral axis in the direction of the speaker's diaphragm. Gestures may also represent the time axis relations of events (past on the left and future on the right), and this may influence the accompanying gesticulation. With such adverbs as obratno (direction to the left, the meaning - returning to the previous point) and vdal' (direction to the right, the meaning - moving to the next point).



Table 21. Gestural patterns of adverbs

In summary, the following gestural patterns for frequent constructions are identified below.

Table 22. Gestural patterns of constructions



5.1.5. Summary

We can conclude that the semantics of the verb root in Russian influences whether the gesture is performed from the point of view of observer or from the point of view of character. In these two situations, the presence of verb arguments with different semantic roles affects the character of the gesture. The presence of spatial adverbs and prepositions influences the character of gesticulation and can overlap the patterns allocated for prefixes and roots.

Both prefixed and non-prefixed verbs of motion are equally accompanied by co-speech gestures. Verbs of an imperfect form are more typically expressed by the accompanying co-speech gesture. Perfectives are rather characterized by a lack of gesture. Whether a verb is transitive or not, the presence or absence of a gesture accompanying it does not depend in any way.

We analyzed four gesture parameters for which distributions were accurate in our sample of verbal prefixes: gesture space, vertical axis, trajectory, and palm configuration. We clearly identified the following gestural patterns of prefixes: the upward orientation of verbs of motion with prefix za-, the orientation of gesture accompanying the verb with prefix so- towards the speaker, and the orientation of verbs of motion with the s- prefix in the direction “out of the gesture space”. The semantics of the verb root, as well as the presence of certain semantic roles in the construction with one or another verb of motion, may affect the character of the gesture, so it is necessary to consider the constructions, and not the prefixes individually.

Then, the link between groups of verbal roots and the trajectory of the gesture, between groups of verbal roots and the distribution towards the vertical axis, and between groups of verbal roots and relation to the gesture space is determined. Further, the following gestural patterns are common for verbs of motion that have at their root the semantic component “downward movement”: the orientation of the gesture on the vertical axis from top to bottom, the trajectory of the gesture in the form of a curved arc directed from the speaker. Verbs of motion with the semantic component “trajectory” at their root are also characterized by the vertical axis distribution: the gesture can be directed upwards or downwards, and the trajectories of motion can be either a straight line or a curved arc, depending on the semantics of the prefix, spatial adverbs and the presence of verbal arguments with certain semantic roles.

The trajectory of gesture accompanying verbs having in their roots a semantic component “downward movement” is influenced by an argument (implicitly or explicitly expressed), which denotes the ending point at which the moving object appears after the motion event. The orientation of gestures accompanying verbs with the semantic component “trajectory” in verbal root towards the horizontal axis depends on where the starting and ending point is from the observer's point of view. This allows us to conclude that the character of gesture is determined by the presence of spatial adverbs and the context, and not only by the semantics of verbal root and prefix.

Finally, the following gestural patterns are typical for constructions with spatial adverbs: the semantics of the adverb determines the orientation of the gesture with respect to the lateral and vertical axis. Constructions with such adverbs as vverkh/naverkh/naverkhu will be accompanied by a gesture directed upwards, constructions with such adverbs as vniz/vnizu will be accompanied by a gesture directed downwards.

5.2 MURCO and NewsScape

After identifying some general trends for gestural accompaniment of verbs of motion in Russian, we looked at MURCO and NewsScape data to confirm our conclusions. We also examined constructions that we had not observed in our initial data analysis.

We have confirmed the patterns we have identified for prefixed verbs of motion in Section 5.2.1. Further, we looked at how the palm configuration (Section 5.2.2.) and the trajectory of the gesture (Section 5.2.3.) would differ for gestures accompanying various constructions with causal motion verbs. When analyzing Pear stories recordings, we did not find many examples where Manner was expressed in a gesture, so we described such examples in Section 5.2.4. We also examined constructions where verbs with the semantic component "downward movement" in the root, in constructions where the trajectory and type of downward movement are different, are accompanied by gestures, in Section 5.2.5. Finally, we looked at the gestural accompaniment of constructions with verbs of motion and spatial adverbs (Section 5.2.6.).

5.2.1. Prefixed verbs

The following gestural patterns were clearly identified for the verb prefixes: the prefixed verbs containing prefix pod- are characterized by a gesture directed upwards the prefixed verbs containing prefix s- are accompanied by a gesture directed downwards, and the prefixed verbs containing prefix u- are characterized by a gesture directed from the speaker. As we can see in Figures 1-4, the character of the co-speech gesture is affected by the semantics of these prefixes.

5.2.2. Caused motion: palm configuration

While analyzing Talks and Tales about Pears, we have found that the gestures accompanying verbs such as brat'/birat' and klast'/kladyvat' are characterized by an “object agreement”, expressed in that the palm configuration expresses the shape or size of the object being moved (as opposed to configurations such as open or straight palm, typical of other verbs of motion). Therefore, we decided to look at the contexts with caused motion verbs, for example khvatat' `to grab', tashchit' `to lug', polozhit' `to put', postavit' `to put', zasunut' `to push in', vytashchit' `to pull out', vydernut' `to pull out', tolkat' `to push', tyanut' `to pull'. We assumed that the palm configuration of gestures accompanying such verbs of caused motion will also express the shape or size of the object to be moved.

We have found some interesting examples in the corpora. All gestures are performed from the perspective of the character. Figure 5 shows an iconic gesture that accompanies the construction khvatat' rybu `to grab a fish', Figures 6a and 6b present an iconic gesture that accompanies the construction polozhit' rubilo `to put down a hand axe', and Figure 7 shows the configuration of the palms of a gesture accompanying the construction skhvatit' bombu `to grab a bomb'.

As it is clearly seen in the illustrations, the configuration of a palm transmits the characteristics of three different objects: the size of a fish, the shape of a hand axe, and the size and shape of a bomb.

So, the palm configuration of gestures accompanying verbs of caused motion may also express the shape or size of the object to be moved (when the gesture is performed from the perspective of the character). The given examples demonstrate the variability of gestures accompanying Russian motion verbs in terms of this parameter of gesture.

It seems interesting to see further on how gesticulation will change in contexts such as “bird grabbed branch”, “tiger grabbed his prey”, where the Ñontainer will be the mouth of the animal, and the gesture will be made from the observer's point of view.

5.2.3. Caused motion: expression of Path

We have found a tendency that gestures accompanying verbs such as brat'/birat' and klast'/kladyvat' are oriented towards or from the speaker depending on the starting point and the endpoint of the moved object, and we want to check it with other examples from corpora. In MURCO and NewsScape, we looked for different constructions with verbs of caused motion, where the position of the endpoint will be different (for example, polozhit' na stol `to put on a table', polozhit' pod stol, `to put under a table' polozhit' v yashchik stola `to put in a drawer'). We assumed that the trajectory of the gesture will also depend on the presence of an explicit starting and ending point.

We found several examples that confirm our assumption. All gestures are performed from the perspective of the character. Figure 8 shows the gesture accompanying a construction polozhit' pod ramku `to put under the frame'. Here, the speaker talks about the frame under which the document he is talking about should be placed. The frame is located typically on the wall: the trajectory of the gesture indicates the location of the endpoint. Figure 9 illustrates the accompanying gesticulation of a construction polozhi na mesto! `put it on its place!'. As it can be seen here, the man's index finger is directed towards the original location of the object. Figures 10a and 10b show a gestural accompaniment of a construction polozhit' v chemodan `to put in a suitcase'. As we can see, the trajectory of the gesture is an arc directed downwards: the suitcase (a final point) acts as a container in which objects are put together. Finally, Figure 11 accompanies a construction polozhit' v kholodil'nik na pravuyu polku `to put in the refrigerator on the right shelf'. The gesture is oriented forward and to the right: the trajectory of the gesture changes because the end point of the object being moved is the refrigerator shelf on the same level as the speaker.

Therefore, the trajectory of the gesture accompanying caused motion verbs may also depend on the presence of an ending point of the moved object (when the gesture is performed from the perspective of the character). The given examples demonstrate the variability of gestures accompanying Russian motion verbs in terms of this parameter of gesture.

5.2.4. Expression of Manner

Further, in Pear stories we did not find many examples where Manner would be expressed by a co-speech gesture. (only in zabirat'sya po lestnitse `to climb the stairs'), but it is interesting to see examine the gestures accompanying motion verbs with different semantic components of Manner at their root would differ, for example, uyti `to go away', ubezhat' `to run away', upolzti `to crawl away', uprygat' `to jump away', ukatit'sya `to roll away'. We assume that the trajectory of the gesture will be different for such verbs.

In MURCO and NewsScape, we have also not found any representative examples of co-speech gestures accompanying such verbs as ubezhat' `to run away', upolzti `to crawl away', uprygat' `to jump away', ukatit'sya `to roll away'. However, as it turned out, MURÑO contains not only movies, lectures and performances, but also cartoons where we found two relevant examples with verbs ukatit'sya `to roll away' (Figures 12a and 12b and upolzti `to crawl away' (Figures 13a and 13b). Both examples are repeatable (multiple) gestures. As can be seen from the illustrations, the gestures accompanying verbs of motion in the speech of the cartoon characters differ in their trajectory and configuration of the palm. The configuration of the palm of the first gesture expresses the form of a rolling object, and the second gesture is performed more slowly than the first

So, the trajectory of the gesture accompanying motion verbs with different semantic components of Manner at their root may differ. The given examples demonstrate the variability of gestures accompanying Russian motion verbs in terms of this parameter of gesture (Manner).

5.2.5. Downward movement: expression of Path

Our analysis of Talks and Tales about Pears also showed that verbs of motion having at the root a semantic component “downward movement” and prefixed motion verbs having the prefix s- are characterized by the orientation of the gesture on the vertical axis downwards. However, it is interesting to see how specific types of endpoints will affect the trajectory of the gesture. To do this, we analyzed the gestural accompaniment of the verbs of falling, that express not a prototypical act of moving to a lower position under the effect of gravity, but other types of falling, for example, upast' na ego golovu `to fall on his head', upast' na spinu `to fall backwards', upast' na ego nogu `to fell on his leg', etc. We assumed that the direction of movement of a falling object may affect the gesture of such construction, and the movement will occur on specific trajectories.

In MURCO and NewScape we found several relevant examples. Most gestures are performed from the observer's point of view. Figures 14a and 14b show a co-speech gesture accompanying the construction upast' na mashinu sverkhu `to fall on top of the car'. As it can be seen from the illustration, the gesture is made with a straight palm, which seems to indicate that the object fell on top of the car, flat. Further, Figure 15 shows the gesture accompanying a construction upast' v yamu `to fall into the pit'. Figures 16, 17a and 17b show the expression of a construction upast' na koleni 'to fall on the knees'. It is interesting that there are several examples in MURÑO where this construction was expressed not by the hand movement, but by the full reproduction of this motion by the speaker: this is because most of the MURCO clips are theatre performances, which shows once again the reinforcement of the character of the accompanying gesticulation on stage. And finally, Figure 18 show a co-speech gesture that accompanies the construction upast' na golovu 'to fall on my head'. It can be clearly seen here that the position of the gesture in the gesture space changes and the gesture is reproduced in the area around the speaker's head.

...