Semantic priming effect on survey results

Размещено на http://www.allbest.ru/

Semantic priming effect on survey results

Olha Yu. Filonik,

student, «Ostroh Academy» National University

Svitlana Yu. Winters,

PhD in Linguistics, University of Calgary

This article presents the findings of an experimental study focusing on the effect of semantic priming on survey respondents. The study involved manipulation of survey questions so that one version included priming triggers and the other one did not. The two versions of the survey were tested on two groups of Canadians (50 respondents each). The results confirmed the authors' hypotheses, as they demonstrated that the inclusion of the priming triggers activated the relevant concepts in respondents' minds and, as a result, they included concepts similar to those triggers in their responses to open-ended questions. To be precise, respondents who were exposed to priming triggers “one”, “first” and “three”, as well as “saving on food”, were significantly more likely to say they shop one or three times a week and to recall the grocery store “Save-On-Foods” in an unaided recall qustion. The findings in this study have theoretical and empirical significance and should be taken into consideration by all the researchers who design questionnaires in their research projects. Based on this research, one can conclude that a researcher who designs questionnaires should be cautions and make sure to sequence questions in a way that would minimize the priming effect on questions following priming triggers.

Key words: semantic priming, questionnaire design, survey bias.

Вплив семантичного праймінгу на результати опитувань

Філонік Ольга Юріївна,

студентка, Національний університет «Острозька академія»

Уінтерз Світлана Юріївна,

кандидат філологічних наук, Університет Калгарі

У статті представлені результати експериментального дослідження, що фокусується на впливі семантичного праймінгу на респондентів. Дослідження включає маніпулювання питаннями опитування таким чином, що одна версія опитування включала праймінг тригери, а інша - не включала. Дві версії опитування були протестовані на двох групах канадців (по 50 респондентів у кожній групі). Результати підтвердили гіпотези авторів, оскільки було продемонстровано, що включення тригерів активує від-повідні концепти у свідомості респондентів, і, як наслідок, респонденти включили у відповіді на відкриті запитання поняття, по-дібні до тих, що служили тригерами. В даному випадку, респонденти, які бачили тригери «one», «first» і «three», а також «saving on food», значно частіше казали, що вони купують продукти харчування оди або три рази на тиждень і називали продуктовий магазин «Save-On-Foods» у своїх відповідях на запитання опитування. Результати цього дослідження мають теоретичне та практичне значення, і їх слід враховувати всім дослідникам, які розробляють анкети опитування у своїх дослідницьких проектах. Виходячи з цього дослідження, можна зробити висновок, що дослідник, який розробляє анкети, повинен бути обережним і ство-рювати таку послідовність питань, яка б мінімізувала ефект праймінгу на відповіді.

Ключові слова: семантичний праймінг, розробка опитувань, упередження в опитуваннях.

Designing a questionnaire is an integral part of any research project. There is sufficient literature on how to sequence questions in a questionnaire (see, for example [6-7] and others). However, little attention has been paid to the priming effect that questionnaire verbiage or sequencing of questions can exert on respondents, therefore biasing the results (see brief discussions in [1, 5, 9]).

Semantic priming has been defined as “the phenomenon by which hearing or reading a word partially activates other words that are related in meaning to that word, making the related words easier to recognize in subsequent encounters.” [8, p. 324] This definition can be extended by saying that the related words also become more accessible to a language speaker and they are more likely to use them in an answer to a question following the activation. This research study aims to look at the effect that the use of certain words (primes) exerts of respondents in a market research survey.

Background

semantic priming survey result

In the present experiment, the goal is to study the effect of verbiage and question sequencing on survey results. The following specific research questions have been addressed: (i) Does the presence of specific numerals (namely, “one” and “three”) in questions affect answers to those subsequent questions which require a numeral in the answer? (ii) Does the presence of a phrase (namely, “saving on food”) trigger respondents to think of a brand with the name similar to the phrase (namely, “Save-On-Foods”) when answering the subsequent question?

Our hypothesis is that due to priming effect, respondents who are exposed to numerals “one” and “three”, as well as the phrase “saving on food”, will be more likely to say “one”, “three” or “Save-On-Foods” in their answers to subsequent open-ended questions. If, however, the study reveals that there is no significant difference in responses of those who were exposed to the triggers above and those who were not, one could conclude that there has been no priming effect and that a researcher can have freedom to use whatever words and phrases they like before open-ended questions.

Design and Materials

The experiment described here involved conducting two online surveys with two groups of Canadians: a test group and a control group (one survey per group). The survey was programmed in SurveyMonkey, an online survey development cloud-based software. The respondents were informed that the questions would focus on their grocery shopping habits and expectations. This topic was selected merely due to the fact that most Canadians would find it relatable. The surveys contained five screening questions (to identify respondents' age, gender, province they reside in, English language proficiency and whether they purchased groceries in the past month) and ten main-body questions. In order to target only those respondents who could enable us to answer the research questions above, the survey was programmed to terminate those who did not do any grocery shopping in the past month and those who lived outside of the provinces where the grocery store chain “Save-On-Foods” is located. While the screening questions were identical in the two surveys, the main-body questions were manipulated with the research questions in mind. Table 1 presents the differences between the two surveys (the differences are presented in bold type).

Table 1. Survey Questions

The differences in bold type present the triggers which are expected to prime respondents to answer questions in a certain way. To be precise, the triggers “three”, “one” and “first” in the questions about attributes of shopping experience (Q1), recall of grocery stores (Q2) and grocery stores shopped at most often (Q3) are expected to increase the frequency of responses “one” and “three” to the question about the frequency of grocery shopping (Q4). Furthermore, the phrase “saving on food” (Q1) is expected to facilitate the recall of the grocery store brand “Save-On-Foods” in “unaided recall” question (Q2).

Subjects

The sample was supplied by SurveyMonkey (Contribute and Rewards Panels). One hundred Canadians participated in the experiment: 50 in the test group and 50 in the control group. Data was examined for quality: all answers were found to be of high quality and no respondents' answers had to be removed. 43 of the 100 subjects were male and 57 female. The average age was 47 years (range: 23 to 68). Most subjects evaluated their English language proficiency as “Excellent” or “Good” and merely 4 as “Okay”. 35 respondents reside in Alberta, 42 in British Columbia, 12 in Manitoba, and 10 in Saskatchewan. As mentioned above, the survey targeted only residents of those provinces in which “Save-On-Foods” stores are located to ensure there is a non-zero probability that respondents are familiar with this grocery store brand.

Results

Results of the study confirmed the original hypotheses. Overall, questionnaire type (questionnaire containing primes vs. one not containing primes) had a significant effect on the results. First, Table 1 shows the results for question 2 (Q2. Please list THREE grocery stores that first come to your mind. / Please list any grocery stores that come to your mind). The percentages in the table reflect the proportion of respondents who listed each of the brand as one of their three or more options and, therefore, they do not add up to 100%. As demonstrated in Table 2, when question 2 was preceded by the version of question 1 that contained the attribute “saving on food”, respondents were considerably more likely to recall “Save-On-Foods” when asked to list grocery stores (the upward arrow indicates statistically significant difference at 95% confidence interval). The difference can be attributed to the priming effect exerted by “saving on food” in question 1.

Second, the presence of the primes “three”, “one” and “first” affected the answers to question 1 (frequency of grocery shopping). The test group overall provided lower numbers when asked about the frequency of their weekly grocery shopping, with answers “one” and “three” being significantly more frequent. The control group provided skewed higher when answering the question about the frequency of their weekly grocery shopping (see Table 3).

Table 2. Unaided Recall of Grocery Stores

Table 3. Frequency of Shopping

Conclusions and Directions for Future Research

The results of the study provides the empirical evidence showing that the presence of triggers like numerals “one”, “first”, “three” and “saving on food” activates the relevant concepts in the mind of a survey respondent and, a result, when answering an open ended question before the activation dies down, the respondent is likely to provide answers that are semantically related to the triggers “one”, “first”, “three” and “saving on food”. This finding has significance for the theory of questionnaire design. It suggests that a researcher who designs questionnaires should be cautions and make sure to sequence questions in a way that would minimize the priming effect on questions following triggers.

There is a number of promising directions for future research related to this topic. For example, even though some attempts have been made to investigate how long the activation lasts from the temporal point of view, no conclusive answer has been arrived at as of yet. Another direction for research could be to study whether any profiling variables affect the strength of the priming effect, for example, whether the province a respondent lives (and the number of grocery stores and popularity of a brand in that area) or respondents' language proficiency (native speaker vs. non-native speaker) could affect the strength of the priming effect. This would require a larger sample size.

References:

1. Ariely D. Predictably Irrational. NY: HarperCollins, 2008. 304 p.

2. Burke D., Yee P. Semantic priming during sentence processing by young and older adults. Developmental Psychology. 1984. № 20. P. 903-910.

3. Colston H. Using figurative language. Cambridge: Cambridge University Press, 2015. 275 p.

4. Fussell S., Krauss R. The effects of intended audience on message production and comprehension: Reference in a common ground framework. Journal of Experimental and Social Psychology. 1989. № 25. P. 203-219.

5. Kahneman D. Thinking fast and slow. NY: Farrar, Straus and Giroux, 2011.

6. Malhotra N. Marketing Research. An Applied Orientation. NJ: Pearson, 2010. 897p.

7. Questionnaire Design. Lectures in Principles of Market Research Certificate Program, the University of Georgia. 2020.

8. Sedivy J., Carlson G. Sold on language: How advertisers talk to you and what this says about you. Chichester, 2011. 330 p.

9. Thaler R., Sunstein C.R. Nudge. Improving decisions about health, wealth and happiness. London: Penguin Books, 2009. 312 p.

10. Winters S. Lexical blending in Ukrainian: System or sport? University of Calgary Dissertation. Calgary, 2017. 329 p.

11. Zwitserlood P., Schriefers H. Effects of sensory information and processing time in spoken-word recognition. Language and Cognitive Processes. 1995. №10. P. 121-136

12. Черниш Л. З., Кошлань М. В. Антитеза, як семантична особливість стилю Йозефа Рота у трансформації понятійних одиниць трагізму та краси. Наукові записки Національного університету «Острозька академія»: серія «Філологія». Острог: Вид-во НаУОА, 2020. Вип. 9(77). С. 84-86.

Размещено на Allbest.ru