1. Topic description, problem statement and research questions The chapter was part of the Project Proposal for Final Thesis, Yarovaya (2020)

Regardless of which topic a sociologist, psychologist, political scientist or any other researcher chooses for his or her study, one of the work components remains consistently significant. It is a methodology. Without proper developed methods and valid research tools, any results and efforts of the author can be negated. Of course, using the right methodology and having a good understanding of how different tools can influence the research results remains quite a popular topic in the scientific environment. For physics - the crucial parts of the methodology are measurements and calculations. For historian - the set of reliable literature and proper analysis of different sources. For sociologist, one of the main concerns is the development of tools and instruments to get “clean” and valid data. For instance, one primary data collection technique is a survey. Since this method involves a certain interaction between the respondent and the researcher (for example, as opposed to content analysis or collection of some secondary data, where researcher does not connect to any respondent), there is a very high probability of receiving an error not only while building the questionnaire but also from the side of a survey participant, while he or she is responding to a certain question. Due to the ongoing relevance of the chosen subject and the possibility of making practical conclusions from the results of the study, the theme of the influence of question wording on the respondent's answer was chosen as the main topic of the diploma theses.

Emphasizing the focus specifically on a sociological research, survey remains the most frequently used method of obtaining the primary data, as was said above. There are a lot of ways and suggestions on how to avoid occurring errors when collecting information from a respondent. Most of them are aimed at changing the form and the content of the questionnaire itself, which may make it easier for participants to understand the questions and balance possible mistakes - for the researcher to get more valid data. These "recommendations" contain information on how to properly create a questionnaire, which questions to include and which to remove, how to form a statement and choose a scale of answers.

One of the directions in which researcher can analyze the immediate impact of the applied methods on the work results is the study of the survey methodology. Since this topic is very broad, it is worth to specify the research interest for this work. The majority of studies in sociology and neighbor areas are based on measuring one's own feelings, perceptions and opinions about the object of research. Therefore, a sociologist learns the respondent's thoughts through a questionnaire. Nevertheless, despite the clear logic and seeming simplicity of this method, a sociologist may obtain completely invalid results. For example, if the respondent is not interested in the subject of the questionnaire, he or she answers the questions randomly without reading. Or if the participant did not understand the question right, he or she responds in guesswork trying to find the “expected” or “right” answer. The overall personal tendency to answer on questions (some authors name this tendency as “style”) also matters - one participant usually picks extreme values ("Very satisfied", "Not at all satisfied"), while the other one selects values close to the middle of the scale ("Rather yes", "Neither yes or no", “Rather no”). Another reason, apart from enumerated, may be the influence of the wording of the survey question itself on the respondent's answer. Many authors studied the specifics of this parameter - the linguistic characteristics of the wording, the psychological aspect of perceiving one or another phrase, and the already mentioned "style" of answering. Therefore, the fundamental research question to be answered in this diploma theses can be formulated in the following way: Is the answer on the question influenced by the wording of that question? This is the issue I have decided to concentrate on in my diploma.

To conduct a valid meaningful study on the influence of the questionnaire wording to respondent's answer, a topic of motivation was selected. Motivation will play a role of context or, in other words, background frame of the study. The topic of motivation has always been in my personal research interest, as it has a significant impact on school education, company work, and the scientific environment. The school setting was chosen as the research field for the diploma. The topic of students' motivation can be studied both from the side of sociology and from the side of psychology, economics and educational management. On the other hand, the very methods of exploring children's motivation at school are also interesting - how exactly do researchers ask children? What tools have been developed to measure motivation? What mistakes can be made when questioning children? A full-fledged research paper can be written for each of these questions. Developing instruments and understanding main biases that research with children has, would help to decrease an error rate and misleading conclusions of such type of studies. This is the reason why I became interested in the methodology of research, precisely, in the school environment. Last year the teacher-student relationship and teaching styles were discussed in the coursework. I am going to mention some aspects of that research in the literature review, as two out of five used scales are actualized and updated for the diploma thesis as well. This year the focus of research has been shifted to the methodological part, as well as to the motivation of the pupil and not to the relationships with the teacher.

Moving back to the central topic of the research, the question formulation should be studied in more precise way, being one of the parameters that can affect the results of high school surveys. Depending on what words are used to ask child's opinion, he or she will give different (often significantly varying) answers. Thus, the hypothesis of the thesis is that the wording of the question in survey affects the responses given by the pupil.

The diploma thesis is divided into key chapters indicated in the content section. The first part includes Introduction, research question and hypotheses. These aspects have already been mentioned above. Further, a theoretical framework and a detailed literature review will be provided. The theoretical framework also contains the operationalization of main terms, used in the study and builds a bridge from the idea of the thesis to concrete literature basement. The next chapter includes a description of the methodology, questionnaire construction and methods used in the study. After that, a description of the sample and the field will be presented in the subsequent paragraphs. The part with results will describe the analysis of the data received and its outputs. The final part of the thesis is the review of findings and drawing conclusions, returning back to the research question and hypothesis of the diploma thesis.

2. Theoretical framework and literature review The chapter was part of the Project Proposal for Final Thesis, Yarovaya (2020)

This chapter will give a general description of the theoretical framework within which the study was carried out. The main concepts used in the thesis will also be explained. The chapter includes the operationalization of terms and logically leads to the detailed literature review presented in the chapter.

2.1 Survey, question formulation and wording

Survey methods (that include interview and/or questionnaire) are the most commonly used tools of sociological research. They can be used both before the study (for pre-analysis of the research field and testing of the questionnaire) and right after the research (for example, as post-hoc analysis after the experiment or during formulation of conclusions - to ascertain the correctness of existing findings).

Many authors have focused their attention on studying this particular method of sociological research - survey. Indeed, the results of the whole work strongly depend on how the sentence was formulated, what scale of answers was offered, whether specific terminology was used and even what design was implemented to the answers sheet. That's why it's crucial to analyze how the survey method itself and the subtleties of framing can affect respondents' answers in all possible ways. While it's intuitive to know that depending on how a person asks a question, a respondent will give divergent answers, it's important to statistically show this effect.

To find articles on the topic of survey design, question formulation and wording techniques, the present query in Web of Science platform was used: (quest* OR survey*) AND (word* OR fram* OR formulation*). Since there were too many results, the literature was filtered by the relevant research fields - psychology, sociology, pedagogy and related areas.

Writing a competent questionnaire that is not too big, but still does not lose in the content; that is understandable and at the same time hassles all the necessary for the research topics is a very challenging task for the sociologist. The questionnaire should be a tool that allows to obtain sincere responses from the participant without producing any additional noise or mistakes. Not only is this difficult to do when respondents are adults, but it is even more complicated when working with children (for instance, pupils or students). Although high school students are a very interesting social group to study opinions on a variety of issues, there is a number of obstacles that any researcher will have to overcome on this path. In addition to very significant barriers in accessing the field, the sociologist encounters challenges in linguistics (since teenagers speak a language distinct from the adult one), in their spare time, in understanding the methods of collecting results (for example, allowing children to fill out a questionnaire via an online form or create a special quiz) - all these issues need to be addressed at the planning stage of the study. Even when all requirements of the proper survey have been met and methodology was formulated clearly, researcher may face low response rate among participants. Touched topics are widely and actively discussed in the literature.

Survey fatigue

In the article “Student Engagement: The Key to Improving Survey Response Rates” (Nair et al., 2008), authors have studied response rate and falling engagement in the research with the college students (Nair et al., 2008). Former pupils were asked to share their opinion on provided courses, educational process and facilities of the college. It was found, that students were more engaged in such kind of activity as opinion sharing only when they were sure, that their responses truly matter to the researcher or stakeholder (for instance, if the results of such study are going to facilitate the implementation of new standards and policies). This factor is highly connected with the idea, that students nowadays are overly surveyed (Coates, 2006). This tendency is called “survey fatigue” and may influence the results of any survey, with students as respondents specifically. Survey fatigue may lead to the random responding and (as a result) to the skewed data. Unfortunately, this is not the only factor that can make the results inaccurate. For example, respondents often skip the invitation letter that describes the research project and move straight to the stage of completing the questionnaire without properly understanding the key idea of the study and the aim of the survey (Mavletova et al., 2014). Furthermore, when the survey is conducted online and the researcher sends the link by email, respondents may simply forget about the required questionnaire and occasionally miss it. This factor also may significantly reduce the number of completed questionnaires and increase the attrition rate.

Answer scales

Another stumbling block is connected with the question formulation and presentation of the answers. In the work “Using Survey Methods for Design and Evaluation in Child Computer Interaction” (Read & Fine, 2005) authors provided a short description of methods that are suitable for the research with children (Read & Fine, 2005). Sociologists have to be careful when asking yes/no questions - children tend to be giving positive answers on such questions. Another point is also concerning answer scale - children may lack the understanding of the range of answers (such as Likert scale, scale from 1 to 10 etc.). To prevent misunderstanding and biased results, it is suggested to use schemes, emoticons, or pictures to demonstrate what each answer of the scale means (Read & Fine, 2005).

Language

One more difficulty which should be taken into consideration is language. Not only children may struggle in understanding questions, that are not clearly formulated - such problem may also occur with adult respondents. However, this point is more crucial for studying children. For instance, even when you ask a direct question about school field trip, pupils may be got confused because they use other words or slang to describe the same activity (as described in the article “Stability of Children's Survey Responses” (Vaillancourt, 1973)). It is also suggested to make questions as simple and short as possible - a narrow, concise and comprehensive question is the gold standard to which all surveys aspire. The child may have troubles with understanding questions that use metaphors or that are formulated in the first-person manner ("I often feel...", “I am successful”). Such question may be misinterpreted. On the other hand, it is better not to ask children about specific things, for instance, the family budget or the parents' place of work. The child may simply not possess this information, which will result in increased errors in the completed questionnaires (Borgers et al., 2000).

Invitation

In the article “Invitation Design Elements in Web Surveys” (Mavletova et al., 2014), authors discussed how different texts may attract participants or vice versa (Mavletova et al., 2014). Sociologists mentioned few key features of invitations and preface texts: they are the topic, appeal to respondents, sender's title, expected time of filling the survey and possible remuneration. The professional title or position of the person who sends the survey to participants may increase the number of finished questionnaires, if the position or the status is high (for example, professor in the University). By the topic of the survey, authors understand the short introduction or invitation sentence with brief explanation of the research. The appeal to the respondents may be translated through the emotions. For instance, authors have compared results for such prefaces as “Please, participate in the research and help your company to become a better workplace” and “Share your opinion with the company” (Mavletova et al., 2014). In this example, number of attracted participants is larger in the first case, than in the second. Additionally, when the title of the survey contains specific characteristics of the recruited participants (“Survey for Key employees”, “Survey for IT-specialists”) number of respondents increases. As suggested in another article with using more formal and personal appeal to the respondent, the share of filled questionnaires may grow up (Pearson J., Levine R. A., 2003). The crucial point in any survey is the time spent on one questionnaire. Even if the survey is time consuming, it is always better to write shorter approximate estimated time (if twenty minutes is required to fulfill the questionnaire, it is better to write eight-ten minutes). Finally, adding some remuneration for participants enlargers answer rate, however, it is quite costly for scientists.

Framing

Another way to influence responses is via framing effects. Framing, as its name implies, encloses a general narrative, question or phrase in a specific context. It focuses the attention of the reader, or in our case the respondent, on the particular aspect, point of view or position. Moreover, frame may help to make emotional appeal to the receiver of the information. Thus, framing is often used in the media and culture, helping to highlight the necessary touch on a common background for the viewer. These are many different types of framing that are used not only by researchers, but also by the journalists, politicians, writers and activists.

Comparison of in-group and out-group characteristics is the first type of it. This method can be used in surveys related to ethnic inequality - something that is beneficial to one side may have the opposite effect on the other (Londen et. al 2010).

The next option to use framing is the effect of selection. For example, in such questions when the respondent is asked what he or she would have done in the place of government/judge/hypothetical character. In these formulations, the researcher often offers many answer options that need to be underlined or, conversely, the unsuitable ones should be crossed out (Levin et al. 2000).

The third way or application of framing is to influence the wording of a question. Specifically, the wording may contain such adjectives as "better", "more productive", "more positive". In this example a positive framing (wording) is used, while questions containing "not" as well as " less", "worse" have negative frames (Levin et al 1998). This type of framing may be considered as the classical one. In one of the works, the authors studied the influence of framing on the attitude to the evaluation system among students (Bies-Hernandez, 2012). Framing consisted of how exactly the grade was calculated - as a loss (the beginning of the course assumed 100% of all students; then each student "lost" points during the module) or as a gain (a classic system of cumulative grade). The experiment showed that negative framing led to lower results than the positive ones. Thus, we can conclude that the presence of the effect of framing on further results is almost indisputable.

Wording

Continuing the topic of the question wording, few more aspects have to be highlighted. The usage positive and negative items, which are polar opposites, can have a significant impact on the responses received. The general trend is to include both directions of the wording in the one questionnaire, so the a mixed-wording survey is built. Such technology is required to ensure that the respondent does not answer the same type of questions automatically, and to track these cases (for example, when a respondent answers in the same way on opposite statements without reading questions). On the other hand, when analyzing the impact of wording on the response, the individual "style" that prevails for a particular respondent is of great importance. For example, someone may often choose extreme scale values, while another respondent almost always answers "Difficult to answer", "Rather yes" or "Rather no".

In the dissertation “Effects of Negative Keying and Wording in Attitude Measures: A Mixed-Methods Study” (Coleman, 2013) the author describes the methodology of how to elaborate the questionnaire (Coleman, 2013). He distinguishes between positive or negative direction of the question and positive or negative wording. Thus, for example, the direction is responsible for the reversed question. Even the statement "I am a happy person" can occur to be negative item in the scale where all other questions are oriented in the opposite way (i.e. they measure the level of sadness and apathy). In such statements, the higher the answer (e.g., the "Fully agree" answer) will lead to a lower average score on the "apathy" scale. On the other hand, "positively directed" is considered to be the statement which it directly counts, in other words, is co-directed with the whole scale or construct. Wording, in turn, is quite different in its essence from the direction of the item. Semantically, this means the use of "no", "not" particles that give the polar opposite shade. However, using "no"/"not" particles is not the only way to " flip" the question. Also, the word opposites can be helpful in formulating the reverse question. Thus, there are two angles or ways to look at the item. First is its direction, regarding the overall scale (construct). Second one is the formulation itself, which may be positive or negative, reflected through the used words. The last one implies the usage either of negative particles or antonyms.

In the article "The impact of item phrasing on the validity of attitude scales for elementary school children" (Benson & Hocevar, 1985) the authors examine the impact of negative, positive and mixed questionnaires on the respondents' answers (Benson & Hocevar, 1985). Likert scale was used as an answer scale, which allowed to compare the results of responses to each of the listed questionnaire forms. The study showed a significant difference between positive and negative response rates. As could be logically suggested, a negative answer to a negatively formulated question equals a positive answer to a positive question in terms of the meaningful content. However, this trend was not visible in the survey results. These conclusions support the necessity to study the impact of question wording on the answer. Since respondents involuntarily break the logic illustrated above, it is worthwhile to follow up on the differences between the response rates to the polar questionnaires.

Despite the fact that the method of using balanced or mixed questionnaires remains the most popular and frequently used approach, there are also hypotheses about the low effectiveness of this method. For example, such idea was presented in the article "Ineffectiveness of Reverse Wording of Questionnaire Items" (Sonderen et al., 2013). In this paper, the authors also wanted to measure the impact of positive, negative formulation, as well as mixed-type questionnaires on respondents' answers. However, the authors were trying to challenge the opinion of previous sociologists about the correctness and necessity of using two types of questions formulation (positive and negative) in one questionnaire. In the conclusions of the analysis, it was assumed that if the correlation coefficient between opposite formulations of the same question is high enough, then the polar formulations do measure "the same" and do not differ. Thus, disagreeing with the negative item is the same as agreeing with the positive item. Finally, the authors received such results of correlation analysis that it is impossible to conclude about the confirmation of this hypothesis.

The article "Positively and negatively worded items in a self-reported measure of dispositional optimism" (Mook et al., 1992) also examines how respondents understand and respond to opposing language (Mook et al., 1992). In the results, the authors report that the mean response rates to negative wording is significantly higher than the mean response rates to positive wording. In this experiment, the researchers used the following logic to create positive and negative formulations: the authors did not formulate statements using "not" particles, but with antonyms and linguistic techniques to create negative wording. Thus, the key conclusion that can be drawn from this article is that there is a distinction between responses to positive and negative language. In other words, it cannot be argued that opposing formulations, firstly, can be applied as "equal" questions, and secondly, it cannot be expected that even when creating diametrically opposed formulations, the respondent will answer them in the same diametrically opposed manner.

Another paper with the similar narrative, written by Schriesheim, C. A., Eisenbach, R. J., and Hill, K. D. in 1991, suggest the similar hypothesis. The authors of article "The Effect of Negation and Polar Opposite Item Reversals on Questionnaire Reliability and Validity" (Schriesheim et al., 1991) conducted an experiment to determine the effect of the wording of the question on the response rate (Schriesheim et al., 1991). The paper used five different questionnaires, which consisted of positively formulated questions, positively formulated questions with negative definitions and words, negatively formulated questions using a "no" particle, negatively formulated questions using antonyms and a mixed questionnaire. After data collection and analysis, the authors found that the modification of wording affected the reliability and validity of the questionnaire scales. However, there were also pairs of questions that were not attempted when the wording was changed. This discussion and the contradictory findings create a field for new research and for obtaining own results. The hypothesis of a diploma thesis can be either confirmed or refuted, which will make it possible to refer the conclusions to one or another side of the given discussion.

In conclusion of the literature review, a brief description of such a method as experiment, which is the main method of research in this thesis, would be helpful. The book "Bit by Bit" (Salganik, 2017) is a fundamental literary source, which describes methods of research with big data and not only. From a methodological point of view, it is necessary to define the main parameters of the experiment. In the book "Bit by Bit" Salganik not only defines this research method, but also describes the main examples, difficulties and approaches to conducting the experiment (Salganik, 2017). Thus, the author identifies four main pillars on which the social experiment is based: participants, randomization of " treatment", measurement of results, the presence of a controlled parameter. In other words, in order to conduct a minimum experiment, participants must be hired, randomly distributed by a parameter over which the researcher has control, and a monitor of the results. As could be seen, there is no clause in this definition about the necessary existence of a control group to conduct the experiment. This is due to the fact that there are several approaches to the design the experiment. For example, a within-sample experiment does not imply that there are two groups to compare the action of a parameter. In this design, the results of pre-treatment and treatment performed on the same group of participants are compared. Mechanisms that "bind" cause and effect imply the presence of a theory, hypothesis or variable through which the influence of cause on effect occurs. In this research, the experiment suggests a randomized distribution of questionnaires between identical (by socio-demographic parameters) two groups of students. The mechanism that leads to the distinction between response rates is the formulation of the question.

2.2 Motivation-measuring scales

In order to create a questionnaire on the basis of which the results of positive and negative formulations will be compared and analyzed, it is required to properly study the ways of measuring motivation.

Two approaches were used to explore the articles on motivation. The original pool of literature was found through the Web of Science site. The following enquiry was used: motivat* AND learn* AND (teach* OR school*). Then, after reading the articles on the topic connected to my research, the method similar to "snowball" was used - the lists of references of these articles were looked through and the most relevant literature resources were found.

Motivation towards learning and performance

Students motivation is always strongly connected with the environment in the class, maintained by the teacher and goals, set by the teacher during the studying process. In the article “Liking or Disliking the Teacher: Student Motivation, Engagement and Achievement” (Montalvo, Mansfield, & Miller, 2007) authors tried to test the hypothesis about the link between personal sets of learning objectives of a student and the fact that the teacher, present in the class, is liked or disliked by children (Montalvo, Mansfield, & Miller, 2007). To give explanations for this research, the light has to be shed on the theoretical background of that study. When working in the classroom environment, students set themselves directives, goals or (in other words) motivations to learn material, work on exercises and participate in discussions. The main two types of this motivation can be defined in the following approach: a learning goal-orientation and a performance goal-orientation (Ames & Archer, 1992).

The learning goal-orientation provides motivation for understanding the material, asking, clarifying questions, working with additional material, but not concentrating on the highest possible marks. The resulting score is not so critically important for the student. He or she wants to dig into the subject, fill all existing gaps in understanding and seek for explanations, answers for additional questions because such type of behavior is evidently maintained by the teacher. Another approach, performance goal-orientation, presumes getting only good marks, which may underline guessing, remembering (but not understanding) the material and trying to get pure positive feedbacks. Students who adopt this approach may have developed learnt helplessness, seeing all exercises and questions to the audience as tests of their competences, which they are extremely afraid to fail. These children are also more likely to suffer from low self-esteem, which affects their continued success in learning and future life. Furthermore, this approach towards learning process is connected with the atmosphere projected by the teacher - for instance, when the tutor gives tests too often, assesses the success of the student only by his or her grades and scolds those who do not keep up with the overall pace, the motivation for learning may significantly decrease.

The concept of liking or disliking the teacher was developed from the idea, that children have rather performance goal-orientation or learning goal-orientation mainly because of teacher's behavior and personal attitudes towards the studying processes, and not because of child's traits of character. These orientations may be inconsistent. For example, student may have the learning goal-orientation in one classroom environment (during one lesson) and the performance goal-orientation in another one, which argues the idea that such objectives are the consequences of the temper. The analysis found that the difference between liked and disliked teacher by measured parameters was significant.

A learning goal-orientation and a performance goal-orientation were consequently changed into other scales in further works. The focus of Montalvo, Mansfield, & Miller study, which was mainly concerning teacher's role in the formation of the studying approaches, has been shifted to other dimensions. Kaplan and Maehr in 2007 defined the same scales a little bit differently - as mastery orientation and performance orientation (Kaplan & Maehr, 2007). A student who adheres to the first type of motivation is determined to understand the material, deepen his or her knowledge and work hard regardless of the feedback received (regardless of whether it is affirmative or negative). While the performance goal orientation provides willingness to be better than others, have best marks (but not necessarily via good knowledge of a subject).

Motivation for avoidance

The third scale - motivation to avoid failures - was described in the article “Approach and avoidance achievement goals and intrinsic motivation” (Elliot & Harackiewicz, 1996). Pupils who have this prevailing form of motivation try not to answer the teacher's questions publicly, not to go out to the blackboard. Any test for such students is a big challenge, as their main goal is not to be worse than other students and not to demonstrate their incompetence. Combining their theory with the concepts, described by Kaplan, researchers have received four categories of students (see table 1)

Table 1

	mastery orientation	performance orientation
approach motivation	self-development difficult tasks desire for understanding	aptitude ambition the desire to be better than others
avoidance motivation	preservation of existing knowledge	desire to be no worse the ambition to hide incompetence

Table 1 provides a matrix describing the key characteristics for each type of student in terms of motivation. As can be seen, each direction is quite clearly distinguished from the others. Thus, for example, a student with master-goal orientation and approach motivation will focus on the self-improvement, enhancement of knowledge. In turn, the diametrically opposite description - performance orientation and avoidance motivation is more related to the desire to be no worse than others, to keep up with the pace of the class. It should be noted that this classification does not assume that any of these descriptions corresponds to an "excellent student", and the other one - to an "F-student". The grades in this case are not important at all. An "excellent student" may have the motivation for avoidance, while an "F-student" may have the motivation to multiply his or her knowledge. The links between grades and motivation prerequisites are not taken into account in this case.

Confidence in teacher and teacher's support

In addition to the scales that have already been described above, measuring the student's approach to the educational process (i.e. self-assessed indicators), two other scales are used in the questionnaire of this diploma. An equally important part and prerequisite for motivation level (high or low) lays in the teacher-student relationships. The student may have different forms of motivations in each lesson. First of all, it depends on the teacher's behaviour in this class. The second cause may be found in the personal attitude towards studying and traits of character. However, two other dimensions, which constitute the teacher-student relationships, may also influence the pupil's interest towards learning and the degree of the applied efforts.

Confidence in the teacher is reflected in the extent to which a relationship of trust between teacher and child is built. For example, in a healthy environment, a pupil feels the tutor's believe in abilities and success of the class. The teacher is available for conversation and gives support to the child's development. Teacher support is close to the previous scale in terms of meaning, but in this case the slope is directed towards emotional support, teacher's assessment through praise and feedback (Evertson & Weinstein, 2012). These two scales are used to measure not only the self-reflected level of involvement in the school environment, but also the perceived teacher-student relationships, that are established both with the class and with each particular student.

Thus, five scales were chosen to measure motivation, which will allow to build a complete questionnaire. The survey forms will contain identical (meaningfully) questions on the five scales: motivation towards learning, motivation towards achievement and motivation towards avoidance, teacher's support and confidence in teacher.



3. Methodology, data and methods of the research Parts of the chapter were included in the Project Proposal for Final Thesis, Yarovaya (2020)

In my work I will try to analyze how exactly a question affects the answer in researches with students and how the reliability of the scale is connected with polar wording (positive and negative).

For this purpose, two questionnaires are developed, that have identical semantic content but different wording of questions. These questionnaires will be randomly distributed among the participants (students), so that the response rates for two research groups will be analyzed. In plain words, by means of this research it will be possible to clearly see the difference between students' answers to the questionnaires with positive and negative wording.

As has become clear, this study is not a simple survey of students. It is an experiment. The research has four key features of the experiment: selected participants (1); one differentiating, researcher-controlled parameter (2); random distribution of this parameter among groups of participants (3); and analysis of results (4) (Salganik, 2017). Since all the parameters corresponding to the experimental method are included in my research, it is possible to assert the experiment as the main research method of my theses.

3.1 Sample and distribution of the questionnaires

The experiment is conducted with the help of pupils of one St. Petersburg Gymnasium. Two types of questionnaires are distributed to 10th and 11th grade students by variants. This method stands for the classical distribution of tasks among students sitting at one desk. The pupil, sitting by the left hand from the teacher's point of view is called “first variant” and on the right hand - “second variant”. The distribution of the questionnaires is believed to be random, because of the two reasons. First of all, pupils sit at the desks not by their free will (for example, with their friends) - they sit according to the teacher's decision. because the teacher, when placing children at desks, wants to achieve a fair distribution of children - not only the condition of quantitative balance is observed, but also the fact who sits with whom (the teacher divides friends at different desks, tries to take into account the gender of the child etc.). Thus, in both samples there will be an equal number of students sitting on the back desks (which may be considered as not attentive enough to the learning process) and those sitting on the first desks, the distribution of gender will be close to the equal numbers. Secondly, all working classrooms where the "class hours" are hold, have a classic desk arrangement (having an equal number of rows and tables by three columns). Due to the fact, that the survey will be performed during these lessons, a unified approach to the distribution of questionnaires "by variants" can be used. “Class hours” are special lessons which are hold once a week and are the part of the pupils' curriculum. These sessions are used for discussions of holidays, events, future exams, existing problems and concerns.

Due to insurmountable circumstances, an emergency and self-isolation regime was introduced in spring 2020. Educational institutions, including schools, were sent on vacation and then to distance learning. Owing to this fact, the data collection method for this thesis has been changed. Request to distribute links to the surveys was sent to class teachers of 11th and 10th grades through the social network "Vkontakte". A message with information about the initial data collection method and the new approach was sent to the teachers. The key question that arose when changing the data collection method was the distribution of questionnaires among students. The advantages of using a "by variants" distribution method have already been highlighted above. Since the student seating is constant throughout the year in all lessons, students know exactly which variant they belong to.

In an email to class teachers, two messages were prepared for distribution in class dialogues in Vkontakte: the first message was addressed to the first variant, respectively, a link to the positive questionnaire was attached to this message. The second message concerned the second variant - and the negative questionnaire. Thus, two messages from the class teacher were sent to each class conversation. The message contained: an indication of the need to take the survey in the shortest possible time, the need to respond to only one appropriate version of the questionnaire, a link to the questionnaire.

3.2 Construction of the questionnaires

Moving on to the description of the questionnaires that are used in the experiment, it is necessary to indicate the final set of scales for measuring the motivation of the student. These scales have already been described in the chapter "Theoretical Framework and Literature Review". The final set of scales in the questionnaire equals five: motivation towards learning, motivation towards achievement and motivation towards avoidance, teacher's support and confidence in teacher. Each scale consists of five questions - statements to be answered using the Likert scale from “Fully disagree” to “Fully agree”. A full list of questions in its original and adapted wording can be found in the appendix (Annex No. 1). The general logic behind using these two types of questionnaires is to compare the median response rates on each used scale for two groups, and to conduct the CFA analysis. Firstly, the exploratory data analysis (EDA) is provided to get the overall understanding of how students have responded. Secondly, the results of EDA are compared for two groups to see differences. Lastly, the confirmatory factor analysis (CFA) may help to check, whether the questions are “united” in the scales correctly, prescribed by the theoretical frame. Once these methods have been applied, it will be possible to draw an unambiguous conclusion about the effect of question wording on the answer.

For a more detailed description of the two questionnaires, it is worth reconsidering Coleman's dissertation on the use of balanced scales and mixed questionnaires. A key tool in this approach is the use of polar opposite pairs of questions. The two types of questionnaires do not differ in their semantic content and contain the same number of scales and questions. The statements themselves also have the identical meaning. The difference between the questionnaires is that the formulation is diametrically opposed. As the main approach that has been used in the construction of the two types of questions, the following plan has been developed. First, the scales used by various authors to measure studying objectives and motivation prerequisites were found. Thus, an initial set of questions and scales for questionnaires were defined. Then the stage of preliminary translation and adaptation of questions into Russian was half-seeded. At this stage, the items were chosen so that they can be correctly translated and are relevant for Russian school environment. After this step, the sample of questions was drastically reduced, but yet this was not the final phase of filtering the questions. The third stage was to translate the items in a more detailed manner. It should be noted that at this step, the first questionnaire began to form, consisting of only positive formulations - in the process of translation of statements into Russian language, a common logic was preserved. Non-negative adjectives and particles "no", "not" etc. were not used. Thus, the first questionnaire was formed. The next step was to adapt the existing questions to the negative formulated survey. It was necessary to "reverse" each item, without violating the meaning of the content. Since complex constructions, the presence of linkages with compounded sentences, the " either-or" format could not be adapted to the negative wording, it was necessary to re-filter the set of questions once again (and this time - finally). As a result, two questionnaires were created, each containing five scales. "Positive" formulation of the survey was developed for the "first variant". The "second variant" suggests mirror-like language. The more the student with the second variant agrees with the statement, the lower is his/her level of motivation and involvement in the learning process.

The use of the experiment in my study provides an opportunity to clearly compare the results of the survey with different types of wording. Since only one survey parameter will be changed, and the respondents will be randomly distributed between the first and second variants, it will be possible to conclude with certainty about the impact of question formulation on the responses.

3.3 Data analysis

In this paragraph the data analysis description will be provided, along with results and conclusions. After data collection, it is necessary to analyze the obtained material. The R-studio software is used for this purpose. First, it is planned to give descriptive statistics in order to have an overview of the obtained data. The exploratory data analysis (EDA) is provided to get the overall understanding of how students have responded. Further, visualization will be used to analyze differences between the results of the two questionnaires. Secondly, the results of EDA are compared for two groups to see differences. The next step is to perform statistical tests to analyze how significant the difference between the results is - specifically a t-test, which can show the level of the contradiction between two groups of data (two variants of the survey). It will be possible to see how differently students are responding to the same questions formulated in opposite directions. If the analysis reveals the statistical significance, then my hypothesis will be confirmed - wording influences answers on questions. Thirdly, the exploratory factor analysis will be conducted to see the factors, formed in the real data regardless the theoretical assumptions that are provided in the work. Lastly, the confirmatory factor analysis (CFA) helps to check, whether the questions are “united” in the scales correctly, prescribed by the theoretical frame. Once these methods have been applied, it will be possible to draw an unambiguous conclusion about the effect of question wording on the answer.

Theoretical model with factors

While conducting the research, authors usually construct special scales, factors which combine formulated parameters. Once the researcher receives real data from respondents, he or she moves on to analysis, assuming that the scales and factors that were formulated within the hypothesis are valid. In the case of this diploma, the theoretical model is shown in picture No. 1. It is identical for both questionnaires (both positive and negative). Its key feature is that each factor consists of 5 authentic items (survey questions).

Размещено на http://www.allbest.ru/

Picture 1 Exploratory data analysis

The total number of completed questionnaires equals 305. Since all questions were marked as mandatory, there are no data gaps and the questionnaires are filled in completely. The distribution between the 10th and 11th parallels is almost equal - 168 pupils in 10th grade and 137 pupils in 11th grade (Table No.2). The larger amount of data relates to the socio-economic program. Since there are more socio-economic classes in parallel than other programs, the number of students from this profile is over 60 (Table No.3).

Table 2

Summary of the data by grades
10th grades	168
11th grades	137

Table 3

Summary of the data by programs	N
Гуманитарный	38
Информационно-математический	47
Cоциально-гуманитарный	40
Cоциально-экономический	74
Физико-математический ...

Страница:

•  1 
•  2 
•  3 

дипломная работа "Measuring student motivation: how the formulation of a question affects a response" скачать

Подобные документы

• Rise of sociology as an intellectual tradition. Classical tradition in sociology of the XIX century

  The essence of social research communities and their development and functioning. Basic social theory of the XIX century. The main idea of Spencer. The index measuring inequality in income distribution Pareto. The principle of social action for Weber.
  
  реферат [32,5 K], добавлен 09.12.2008
  

• Why societies are divided into classes

  Four common social classes. Karl Marx's social theory of class. Analysis the nature of class relations. The conflict as the key driving force of history and the main determinant of social trajectories. Today’s social classes. Postindustrial societies.
  
  презентация [718,4 K], добавлен 05.04.2014
  

• Effects of employment on student academic success

  Studies to determine the effects of fulltime and parttime employment on the academic success of college students, on time to graduation and on future earnings. Submission of proposals on how a university student employment offices may utilize these data.
  
  статья [62,1 K], добавлен 23.02.2015
  

• Problems of race discrimination of the USA in the XX century

  Race discriminations on ethnicity backgrounds. The Globalization and Racism in Media Age. African American writers about racism. Comparative analysis of the novel "To Kill a Mockingbird" Harper Lee and story "Going to Meet The Man" by James Baldwin.
  
  дипломная работа [135,9 K], добавлен 29.03.2012
  

• Migration? What does it mean?

  The nature and content of the concept of "migration". The main causes and consequences of migration processes in the modern world. Countries to which most people are emigrating from around the world. TThe conditions for obtaining the status of "migrant".
  
  презентация [4,8 M], добавлен 22.03.2015
  

• Social institutions

  The concept, definition, typology, characteristics of social institute. The functions of social institution: overt and latent. The main institution of society: structural elements. Social institutions of policy, economy, science and education, religion.
  
  курсовая работа [22,2 K], добавлен 21.04.2014
  

• Drug trafficking - a global challenge of the XXI century

  Description situation of the drugs in the world. Factors and tendencies of development of drugs business. Analysis kinds of drugs, their stages of manufacture and territory of sale. Interrelation of drugs business with other global problems of mankind.
  
  курсовая работа [38,9 K], добавлен 13.09.2010
  

• Social structure

  Social structure as one of the main regulators of social dynamic. The structure of the social system: social communities, social institutions, social groups, social organizations. The structure of social space. The subsystem of society by T. Parsons.
  
  презентация [548,2 K], добавлен 06.02.2014
  

• The American family: Past and Present

  American marriage pattern, its types, statistics and trends among different social groups and ages. The reasons of marriage and divorce and analyzing the statistics of divorce and it’s impact on people. The position of children in American family.
  
  курсовая работа [48,3 K], добавлен 23.08.2013
  

• Social provision in Russia

  The need for human society in the social security. Guarantee of social security in old age, in case of an illness full or partial disability, loss of the supporter, and also in other cases provided by the law. Role of social provision in social work.
  
  презентация [824,4 K], добавлен 16.10.2013
  

• Word demographic problems

  The study of human populations. Demographic prognoses. The contemplation about future social developments. The population increase. Life expectancy. The international migration. The return migration of highly skilled workers to their home countries.
  
  реферат [20,6 K], добавлен 24.07.2014
  

• Gender issues in the USA

  The concept and sex, and especially his studies in psychology and sociology at the present stage. The history of the study of the concepts of masculinity and femininity. Gender issues in Russian society. Gender identity and the role of women in America.
  
  дипломная работа [73,0 K], добавлен 11.11.2013
  

• What is Hot with the Young Generation

  Study the opinion of elderly people and young people about youth culture. Subculture as a group of people with the same interests and views on life. Passion for today's youth to heavy music, computers, dance parties and special styles of clothing.
  
  презентация [654,6 K], добавлен 28.10.2014
  

• The mechanisms of transition from on-line public to the off-line public, as well as from local to global level

  The concept of public: from ancient times to era of Web 2.0. Global public communication. "Charlie Hebdo" case. Transition of public from on-line to off-line. Case study: from blog to political party. "M5S Public": features and mechanisms of transition.
  
  дипломная работа [2,7 M], добавлен 23.10.2016
  

• Methods of Lexicological Analysis

  The process of scientific investigation. Contrastive Analysis. Statistical Methods of Analysis. Immediate Constituents Analysis. Distributional Analysis and Co-occurrence. Transformational Analysis. Method of Semantic Differential. Contextual Analysis.
  
  реферат [26,5 K], добавлен 31.07.2008
  

• The motivation as training to foreign language

  Motivation to learn a foreign language in Kazakhstan. Motivation in the classroom. The role of games on language lessons. Examples of some games and activities which had approbated on English language lessons. Various factors of student motivation.
  
  курсовая работа [25,0 K], добавлен 16.01.2013
  

• Research methods to collect primary empirical information

  Research methods are strategies or techniques to conduct a systematic research. To collect primary data four main methods are used: survey, observation, document analysis and experiment. Several problems can arise when using questionnaire. Interviewing.
  
  реферат [16,7 K], добавлен 18.01.2009
  

• Different styles of teaching

  This article suggests specific ways in which college teachers can foster relationships with students that promote motivation and satisfaction. Fostering personal relationships with students. Motivating students to work. Handling interpersonal issues.
  
  статья [18,6 K], добавлен 10.05.2014
  

• Frame analysis

  Defining cognitive linguistics. The main descriptive devices of frame analysis are the notions of frame and perspective. Frame is an assemblage of the knowledge we have about a certain situation, e.g., buying and selling. Application of frame analysis.
  
  реферат [324,4 K], добавлен 07.04.2012
  

• Traffic indicators investigation

  Construction of zone and flight plan. Modeling of zone in experimental program "Potok". Analysis of main flow direction of modeled airspace. Analysis of modeled airspace "Ivlieva_South" and determination of main flow direction, intensity, density.
  
  курсовая работа [2,0 M], добавлен 21.11.2014
  

Другие документы, подобные "Measuring student motivation: how the formulation of a question affects a response"

• главная
• рубрики
• по алфавиту
• вернуться в начало страницы
• вернуться к началу текста
• вернуться к подобным работам