Identification of emotional facial expressions in a lab and over the Internet

Размещено на http://www.allbest.ru/

Размещено на http://www.allbest.ru/

Identification of emotional facial expressions in a lab and over the internet

M. Vesker, D. Bahn, F. Dege

Abstract

Collecting data over the internet is an approach that allows researchers to vastly expand the possible sample sizes of their studies, and enables the study of populations that may otherwise be difficult to access. However, to ensure that data collected over the internet is of the same level of quality as data collected in a lab, the comparability of internet-collected data with lab-collected data must first be assessed for individual areas of research and experimental approaches. To answer the question of whether internet data collection is suitable for experiments involving facial expressions, we conducted a deliberately difficult facial emotion-identification experiment where participants completed the same task either under supervision in our lab, or at an unsupervised location over the internet. Stimuli consisted of sad faces that participants were asked to identify as resembling either anger, fear, or disgust. Regardless of belonging to either the group tested in the lab or over the internet, participants showed highly similar response distributions, while differences between the groups were non-significant and of very low magnitude. We can therefore conclude from our findings that internet data collection is a viable method for experiments requiring the identification of emotional facial expressions, being able to produce similar results to those which can be obtained in a lab.

Keywords: internet, online, verification, faces, emotion, identification.

emotional facial expressions

Introduction

With the emergence of the internet, researchers immediately seized upon the potential for internet-based remote data collection as a tool of enormous potential. Data gathering over the internet can give researchers access to sample sizes in the hundreds and even thousands with relative ease, while allowing for data gathering from participants of very specialized populations. Despite these apparent advantages, questions were soon raised about the compatibility of data collected remotely over the internet with standard laboratory experiments. Were participants truly performing similarly regardless of the administration method, or could the lower degree of control over the testing environment in the case of internet experiments produce different results?

This question of the comparability of data collected in the lab and over the internet is crucial if one wishes to conduct an experiment over the internet while maintaining compatibility with existing lab-based experimental literature in a given area of research. Thus far, several reviews of this topic broadly seem to indicate that internet data collection is a viable tool for psychological experiments (Birnbaum, 2004; Coles, Cook, & Blake, 2007; Hewson, 2015; Krantz & Dalal, 2000). However, several studies (even those that ultimately conclude in favor of internet and in-lab testing interchangeability) do show differences with respect to some measures.

In the realm of behavioral experiments, Linnman et al. (2006) found that response times were significantly slower on the web compared to the in-lab condition while attempting to replicate the Stroop effect in an internet experiment. Germine and colleagues (2012) meanwhile found age- and sex-matched subject showed significantly lower accuracy scores in the internet versions of the VPAM (Verbal Paired Associates Memory) and FDS (Forward Digit Span) tasks relative to participants tested in the lab. Thus, it is necessary to individually verify the suitability of internet testing for various areas of experimental psychology. To collect data over the internet without prior verification that this data possesses similar qualities to data collected in the lab could result in doubts being cast on the resulting findings in terms of fitting into the existing literature.

Experimental tasks involving social aspects such as emotion recognition from human faces may be particularly influenced by aspects of the testing environment such as the presence of others in the room, the familiarity of the testing location, etc. Thus, we were interested in whether such an experiment could produce truly similar results, regardless of being conducted under supervision in a laboratory, or in the comfort of one's home.

Faces expressing negative emotions are also especially interesting due to the links between attentional biases towards such expressions and disorders such as depression and various anxiety disorders (Gotlib, Krasnoperova, Yue, & Joorman, 2004; Mogg & Bradley, 2002; 2004). Some studies have also demonstrated results which show promise in using tasks involving negative face perception as tools for potential identification of patients at risk of depression (Joormann, Talbot, & Gotlib, 2007), and even as a therapeutic technique for the treatment of social anxiety disorder (Schmidt, Richey, Buckner, & Timpano, 2009). Being able to provide such therapies remotely over the internet could greatly increase access for patients, but as noted previously, one must first verify the comparability of emotion perception abilities for human faces when assessed remotely from uncontrolled locations versus the lab.

To our knowledge, the only experiment which attempted to verify the suitability of internet data collection for the identification of emotions in a part of the face was the “Reading the Mind in the Eyes” experiment carried out by Germine et al. (2012) as part of their battery of tests. This task, based on the paradigm of BaronCohen et al. (2001), involves participants being asked to identify emotions having only seen the eye-region of a target face stimulus. Although Germine et al. (2012) did not find significant differences between the data of subject who performed this task in the lab versus online, the specificity of the task with regards to the eye- region, as well the lack of other studies which could confirm their findings make it difficult to conclusively say that internet testing is universally suitable for emotion-identification experiments involving whole faces. To further clarify this question, we conducted an emotion identification experiment where participants were shown complete faces, and were asked to identify the emotion out of three possible choices (fear, anger, and disgust). The task was made deliberately difficult not only by the possibility of three responses instead of the usual two, but even more so by showing participants only sad faces, effectively forcing participants to choose the second most obvious expression, given that sadness was not available as a response. By using a deliberately difficult task, our goal was to create a situation where even slight influences of effects such as the presence of an experimenter in the lab condition would produce noticeable differences in our data if they were present. Negative faces were chosen for this task due our aforementioned interest in their clinical importance, as well the greater variety of primary emotions possessing a clear negative emotional valence (sadness, anger, fear, and disgust) relative to positive faces. This crowding of basic emotions in the negative emotional valence category naturally produces greater rates of mistakenly identifying one negative facial expression for another (Calvo & Lundqvist, 2008; Loughead, Gur, Elliot, & Gur, 2008; Palermo & Coltheart, 2004). This, in turn, may allow for easier perception of negative emotions as “compound” emotions bearing resemblance to more than one emotion at once (Du, Tao, & Martinez, 2014), a necessary component of our method.

Method

Participants

Stimuli

Our stimuli consisted of 56 images of human faces (frontal view, 31 female, 25 male) from the Caucasian category of the Tarr Lab face database (see Figure 1)¹.

The faces were selected from a larger set of 92 faces which were categorized as sad faces in the Tarr data base, and wore no glasses or headgear. In a preliminary test with 4 adult participants we presented the 92 faces and asked participants to indicate which of the faces had expressions resembling other negative emotions in

Images courtesy of Michael J. Tarr, Center for the Neural Basis of Cognition and Department of Psychology, Carnegie Mellon University, http://www.tarrlab.org/ Funding provided by NSF award 0339122 addition to sadness. Fifty-six of the 92 faces were judged as expressing fear, anger, or disgust in addition to sadness by at least 2 of the 4 participants in this preliminary test, and these faces were selected as the stimuli faces for our main experiment.

Figure 1

Examples of Female (A) and Male (B) Facial Stimuli Used for the Current Study

Procedure

Stimuli faces were presented on a computer monitor in a browser window using the Ilias internet platform. Participants were asked to identify 56 sad faces as expressing the emotions of anger, disgust, or fear by clicking on one of three text labels for those emotions presented below each face on the screen. Thus, our participants were forced to identify not the primary emotion (sadness), but rather the additional/lesser emotional characteristics of each face, increasing the task difficulty, and allowing for a highly sensitive comparison that would not suffer from ceiling effects.

For the internet-testing condition, participants completed the task through the Ilias internet platform at the University of Marburg from their personal computers at uncontrolled locations. For the in-lab testing conditions, participants completed the same task at the University of Giessen during supervised individual appointments (again using the Ilias platform) on a laboratory-provided laptop computer with a 15.6 inch screen.

Results

Selection Percentage Analysis

To examine the possible effects of data-collection methods on our participants' selection of one of the three emotion categories, a 2X3 (Condition [lab, internet] X Emotion [anger, fear, disgust]) item-based repeated-measures ANOVA was performed on the selection percentages across all subjects of each emotion for each face (i.e. what percentage of participants selected each of the three possible emotion categories for each face, see Figure 2). The order of stimuli presentation was added as a co-variate. The focus of our analysis was the interaction between the two main factors of Condition and Emotion, which would be an indicator if the testing condition influenced the resulting selection of emotion by our participants. The results revealed no significant interaction of condition and emotion, F(2, 108) = 0.128, p =.88, f² =.002, with a calculated power of 0.946 according to a post- hoc analysis in G*Power 3.1.9.2 (Faul, Erdfelder, Lang, & Buchner, 2007), indicating no differences across testing conditions. No other factors or interactions showed any significant or noteworthy results, indicating similar selection percentages for all faces, regardless of testing condition (see Figure 3).

Figure 2

Note. Box-A highlights an example of a tied leading selection for the internet condition, Box-B highlights an example of a single leading selection in lab conditions, while Box-C highlights an example of a mismatch in selections between the lab and internet conditions. Box-D shows an example of a face stimulus with a matching leading emotion selection across both conditions, but with varying margins between the percentages of the most commonly selected, and the second most commonly selected emotion.

Matching the Most Common Selections of Emotion Categories across Testing

Next, we examined how closely the final selection of one of the emotion categories for each face matched across our two conditions (internet and lab). In other words, how often were participants in the two testing conditions in agreement with each other about which emotion was expressed by each face. To answer this question we first had to score each face stimulus in terms of which emotion emerged as the most commonly selected one across all participants in a given test condition (internet, lab). However, given the subjective nature of the task, we anticipated that some of the faces may produce situations where more than just one emotion would emerge as the preferred choice across participants. Therefore, in terms of absolute agreement in the choice of one of the three emotions, a clear choice for each face stimulus was counted only if the emotion with the highest selection percentage among participants was at least 5% higher than the next closest selection. If the two highest selections for a stimulus were less than 5% apart, the two choices were counted as a tie. For example, in the lab condition, face number 15 (see Figure 2, Box B-solid grey lines) was determine by 60% of participants to be displaying fear, while another 30% selected anger, and the remaining 10% selected disgust, fear was therefore determined to be the leading selection amongst those participants. Meanwhile, face number 6 was described in the internet condition as showing fear by 48% of participants, anger by 46% of participants, and disgust by the remaining 6% of participants, the leading selection among participants was thus determined to be tied between fear and anger (see Figure 2, Box A-dotted black lines). A clear mismatch between the lab and internet conditions was registered for any face for which there was no match between the emotions selected as either the leading choices, or tied for the lead choice. For example, face 35 had disgust emerged as the most common selection in the lab condition, while in the internet condition anger emerged as the most commonly selected emotion at 54%, leading to a mismatch classification for this face between the two conditions (see Figure 2, Box C).

Figure 3

Note. Error bars represent standard error.

This comparison showed a total of only 6 mismatches among our 56 stimuli (faces 7, 10, 35, 42, 44, and 48), giving a disagreement rate of just 10.7%.

Selection Certainty across Participants in Each Condition

Finally, we examined whether the testing condition would influence the degree of certainty or agreement among subjects' selections even if the highest selected emotion was matched overall between the two conditions. For instance, for face number 39 (see Figure 2, Box D) 65% of participants in the lab condition selected disgust as the emotion on display and 30% selected anger, while in the internet condition only 59% of participants selected disgust and 33% selected anger. Thus, disgust would be the most common selection under both conditions. However, the margin between the selection of fear and anger for this face still differed between the two conditions, which could still show an influence of the testing condition on the degree of certainty in participants' choices. We therefore analysed this degree of certainty among our participants in the 50 matching selections across both testing conditions. For this comparison we used the differences between the selection percentages of the highest selected emotion and the second-most commonly selected emotion (for each stimulus within each condition). A t-test showed a high degree of similarity between the two conditions in this comparison, f(98) = 0.297, p =.767, (see Figure 4), and thus the certainty of our participants in their selection choices was very similar under both testing conditions.

Discussion

Utilizing internet-based data collection for psychological research can provide obvious benefits in terms of the sheer volume of possible subject participation in psychological experiments, as well as improving access to potential diagnostic and therapeutic techniques involving face perception. The crucial downside of this approach is the lack of control over the testing environment compared to the lab. Experimenters can of course provide their participants with instructions, but have not real way of insuring that the environment for each subject contains no sensory distractions that may impact the experimental performance. Various domains of psychological research may be differently impacted by this lack of environmental control (Germine et al., 2012), and thus experimental approaches must first be verified for suitability under these conditions. Tasks involving the identification of human facial emotions are inherently social in nature, and thus may be especially sensitive to differences in the testing environment, particularly the presence of other persons. We therefore decided to perform a short verification experiment to determine if data collected over the internet for such tasks could be used as a viable substitute or complement for lab-collected data. Our results demonstrated the suitability of data collection over the internet in tasks involving the identification of facial expressions by showing no effects of testing method when comparing data collected over the internet to that collected in the lab.

Figure 4

Average Differences in the Selection Percentages between the First and Second Most Common Choices

Up to this point, the only related published evidence concerning a part of the face that we are aware of came from the work of Germine et al. (2012). The authors asked subjects to perform the “Reading the Mind in the Eyes” task (Baron-Cohen et al., 2001) in the lab, while others performed the same task over the internet. However, while the eyes do convey a lot of emotional information, the mouth can also play an important role in the identification of certain emotions such as disgust and anger, and thus it is necessary to conduct an internet-lab comparison also for whole faces as in the present study (Bassili, 1979; Kohler et al., 2004; Yuki, Maddux, Masuda, 2007).

We therefore decided to conduct an experiment using a more difficult 3-choice emotion identification task using whole faces. The increased difficulty would also produce data that would be more sensitive to differences resulting from subtle influences in the internet- or lab testing environment, and help avoid ceiling effects. This increased sensitivity would make it easier to detect differences between the testing conditions, and thus would make a more convincing argument for the interchangeability of testing methods should no differences be found as in the study by Germine at al. (2012). Additionally, our usage of whole faces would increase the generalizability of our results to other whole-face paradigms.

In agreement with the findings of Germine at al. (2012), the analysis of the response distribution in our study showed that the testing condition had no significant effect on the participants' selections for each stimulus, with the interaction between condition and emotion-selection being extremely weak and showing no significance. In terms of the most common responses to each face, our results showed that despite the task difficulty, participants were able to reach a very high agreement level between the two testing modalities, disagreeing only in 6 cases out of 56. Our final analysis examined whether the testing condition had any influence on the degree of agreement or selection-certainty among participants within each individual condition for the 50 stimuli where participants agreed on the final selection in both conditions. We averaged differences between the selection percentages of the first most common, and second most common choices for each stimulus, and compared those averages between the two conditions. Once again, we found no effect of testing condition, indicating that participants demonstrated highly similar levels of certainty in their selections across both testing methods. Our results thus demonstrate that even in a difficult emotion-identification task where the increased noise of uncontrolled testing environments could theoretically influence the outcome of the experiment, participants tested over the internet still demonstrated highly similar outcomes compared to those tested under controlled laboratory conditions.

Our study was limited in scope to the agreement in emotion-selection between participants, and did not measure response times. An extension of this study based on measuring response times would be necessary to investigate whether the choice of internet- or lab-based data collection might influence the processing speeds of such mental tasks. Furthermore, although the ability to collect larger data sets is one of the primary benefits of online data collection, a larger sample collected in the lab from a population more closely matched to the online sample would be a useful additional aspect of further studies examining the comparability of data collected in the lab versus online.

Conclusions

In summary, we conclude from our study that collecting data over the internet for tasks requiring participants to identify emotional facial expressions yields highly similar data in terms of the participants' selections compared to data collected under laboratory conditions. Thus, internet-based data collection can serve as a useful method for expanding the scope of experiments focused on emotional faces, as well as diagnostic and therapeutic approaches to depression and anxiety disorders.

References

Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., & Plumb, I. (2001). The “Reading the Mind in the Eyes” Test Revised Version: A study with normal adults, and adults with Asperger syndrome or high-functioning autism. Journal of Child Psychology and Psychiatry, 42(2), 241-251.

Bassili, J. N. (1979). Emotion recognition: The role of facial movement and the relative importance of upper and lower areas of the face. Journal of Personality and Social Psychology, 37(11), 2049-2058. doi:10.1037/0022-3514.37.11.2049

Birnbaum, M. H. (2004). Human research and data collection via the Internet. Annual Review of Psychology, 55, 803-832. doi:10.1146/annurev.psych.55.090902.141601

Calvo, M. G., & Lundqvist, D. (2008). Facial expressions of emotion (KDEF): Identification under different display-duration conditions. Behavior Research Methods, 40(1), 109-115. doi:10.3758/BRM.40.1.109

Coles, M. E., Cook, L. M., & Blake, T. R. (2007). Assessing obsessive compulsive symptoms and cognitions on the internet: Evidence for the comparability of paper and Internet administration. Behaviour Research and Therapy, 45, 2232-2240. doi:10.1016/j.brat.2006.12.009

Du, S., Tao, Y., & Martinez, A. M. (2014). Compound facial expressions of emotion. Proceedings of the National Academy of Sciences, 111(15), E1454-E1462. doi:10.1073/pnas.1322355111

Faul, F, Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175-191. doi:10.3758/BF03193146

Germine, L., Nakayama, K., Duchaine, B. C., Charbis, C. F., Chatterjee, G., & Wilmer, J. B. (2012). Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments. Psychonomic Bulletin Review, 19, 847-857. doi:10.3758/s13423-012-0296-9 Gotlib, I. H., Krasnoperova, E., Yue, D. N., & Joormann, J. (2004). Attentional biases for negative interpersonal stimuli in clinical depression. Journal of Abnormal Psychology, 113(1), 127-135. doi: 10.1037/0021-843X. 113.1.127

Hewson, C. (2015). Research methods on the Internet. In L. Cantoni & J. A. Danowski (Eds.), Communication and technology (pp. 277-302). Berlin: De Gruyter Mouton.

Joormann, J., Talbot, L., & Gotlib, I. H. (2007). Biased processing of emotional information in girls at risk for depression. Journal of Abnormal Psychology, 116(1), 135-143. doi:10.1037/0021- 843X.116.1.135

Kohler, C. G., Turner, T., Stolar, N. M., Bilker, W. B., Brensinger, C. M., Gur, R. E., & Gur, R. C. (2004). Differences in facial expressions of four universal emotions. Psychiatry Research, 128(3), 235-244. doi: 10.1016/j.psychres.2004.07.003

Krantz, J. H., & Dalal, R. (2000). Validity of Web-based psychological research. In M. H. Birnbaum (Ed.), Psychological Experiments on the Internet (pp. 35-60). San Diego, CA: Academic Press. Linnman, C., Carlbring, P., Ehman, E., Andersson, H., & Andersson, G. (2006). The Stroop effect on the internet. Computers in Human Behavior, 22(3), 448-455. doi:10.1016/j.chb.2004.09.010 Loughead, J., Gur, R. C., Elliott, M., & Gur, R. E. (2008). Neural circuitry for accurate identification of facial emotions. Brain Research, 1194, 37-44. doi:10.1016/j.brainres.2007.10.105 Mogg, K., & Bradley, B. P. (2002). Selective orienting of attention to masked threat faces in social anxiety. Behaviour Research and Therapy, 40(12), 1403-1414. doi: 10.1016/s0005-7967(02)00017-7 Mogg, K., & Bradley, B. P. (2004). Selective attention to angry faces in clinical social phobia. Journal of Abnormal Psychology, 113(1), 160-165. doi:10.1037/0021-843X.113.1.160 Palermo, R., & Coltheart, M. (2004). Photographs of facial expression: accuracy, response times, and ratings of intensity. Behavior Research Methods, Instruments, and Computers: A Journal of the Psychonomic Society, 36(4), 634-638. doi:10.3758/BF03206544 Schmidt, N. B., Richey, J. A., Buckner, J. D., & Timpano, K. R. (2009). Attention training for generalized social anxiety disorder. Journal of Abnormal Psychology, 118(1), 5-14. doi:10.1037/a0013643 Yuki, M., Maddux, W. W., & Masuda, T. (2007). Are the windows to the soul the same in the East and West? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. Journal of Experimental Social Psychology, 43(2), 303-311.

Размещено на Allbest.ru