Factors affecting MOOC retention rates

Размещено на http://www.allbest.ru/

FACTORS AFFECTING MOOC RETENTION RATES

E. Sorokina, Y. Gambeeva,

A. Glotova, L. Litvak



Аннотация

Факторы, влияющие на результативность освоения МООК

Е.И. Сорокина, Ю.Н. Гамбеева, А.В. Глотова, Л.Д. Литвак

В условиях активного применения электронного обучения и формирования новой образовательной среды особую актуальность приобретают вопросы трансформации, модернизации и совершенствования системы повышения квалификации кадров. Развитие современной системы повышения квалификации продиктовано необходимостью внедрения новых стандартов обучения, использующих результаты новейших научных достижений, инновационные инструменты и методики обучения, а также модели и технологии непрерывного обучения.

Цель работы -- проанализировать факторы, от которых зависит результативность освоения МООК в рамках повышения квалификации работников образовательных организаций по дистанционному курсу «Массовые открытые онлайн-курсы (МООК) в образовании», реализованному в формате МООК на платформе «Университет без границ».

В ходе исследования использовались методы системного анализа, синтеза и обобщения. Описан эксперимент по разработке и апробации онлайн-курса в условиях временных ограничений и без них.

Ключевые слова: массовые открытые онлайн-курсы (MOO^), онлайн-курсы, онлайн-обучение, среда обучения, профессиональное развитие, учебный курс, инструменты онлайн-обучения.



Annotation

learning environment educational lifelong

Implementation of eLearning and the development of the new learning environment encourage reformation of professional development courses. Introduction of new educational standards incorporating the latest scientific breakthroughs, new technologies and learning tools requires a thorough reconsideration of professional training. Online courses offer new ways to promote lifelong learning and continuous professional development.

This research paper examines factors affecting MOOC retention rates and overall performance ofparticipants enrolled in the “MOOC in Education” course. The course was designed for educators as an online professional development course delivered via the “University without borders” platform.

The study was based on the system-oriented approach, synthesis and generalisation methods. The article also contains a description of the experiment including design and beta-testing of a fixed- schedule MOOC and a course with flexible assessment dates.

Keywords: massive Open Online Courses (MOOC), online courses, online learning, learning environment, continuous professional development, training courses, online learning tools.



Introduction

Fast-evolving information and communications technologies (ICT), such as the Internet, virtual cloud servers, mobile applications, virtual reality, etc., as well as other social trends, stimulate changes in the understanding of professional development.

A shift from centralised learning to online activities and networking proves the importance of technologies and digital skills (e. g. searching for information, data processing, online interaction, etc.). Modern professional development courses are aimed at designing a flexible, mobile and hybrid learning environment that encourages individual learning approaches.

Number of MOOCs

Fig. 1 Quantity of MOOCs in the world



Fig. 2 Number of new learners on platforms

Flexible models of eLearning in professional development courses address needs of students by providing ongoing access to open educational resources. When enrolled in a MOOC, learners interact with a lecturer, tutors, mentors and other students via online activities. They are engaged in group discussions through chats, forums and webinars. MOOCs are usually self- paced and easy to fit into students' schedule. Participants complete numerous tasks, explore new ways of learning and share their experience with others by interacting on the course forum [1].

Integrating MOOCs into a curriculum helps to design a flexible and efficient professional development programme.

The origin of MOOCs dates back to 2008; however, this learning tool started gaining popularity in 2012, when it became widespread in the education sector. MOOCs provided new possibilities and benefits for learning. Best universities all over the world are interested in the MOOC design and distribution, which was revealed by the analysis of online courses market, taking into account the number of designers, providers and participants. From 2012 till 2018, the number of courses offered via various platforms grew 37 times. Similar trends are observed in the statistics showing the number of participants registered on the online platforms, which increased up to 40 times. The highest peak was in 2016-2017 [5].

MOOCs are effective and popular tools used for digital transformation of the learning process and further professional development.

A traditional online course is represented by a fixed-schedule MOOC (6-10 weeks long, as a rule) comprising various assignments with progress and final assessment tests. This model requires weekly time commitment from participants. The average amount of working hours is set in the lead-in section of a course.

One of the challenges for MOOC is actual retention rate, revealing small number of individuals who completed courses successfully. According to statistical sources, their number varies from 5 to 15% [2; 8; 9; 11]. Such a high dropout rate is the result of the MOOC model, which, on the one hand, enables fast information search and selection [3], but, on the other hand, requires being motivated and self-organised within deadlines. This fact explains the popularity of MOOC models with a flexible schedule (e. g. “on demand” MOOC, “self-paced” MOOC, “anytime” MOOC).

MOOC types with no deadlines for progress assessments are more convenient for designing personal learning plans. They are preferred by working participants and those pressed for time. A flexible MOOC format is popular in continuous professional development, as time is of the essence in addressing the needs of such learners.

Interaction is vital in the learning process. Traditional interaction forms are based on the communication of participants. When enrolled in a MOOC, students may organise their work individually, as many online courses are self- paced. However, interaction facilitation and engagement in group discussions are of primary concern in eLearning. Numerous research papers are dedicated to the features of MOOC learning procedures [6; 7; 10]. These works focus on certain student categories that are grouped based on students' professional field, nationality, age, sex, level of education, etc. However, it is difficult to assess data and make assumptions based on the results of the previous research.

The current study analyses the correlation between active interaction and the performance of MOOC participants, who are mostly represented by university lecturers from the Russian Federation.

Objectives:

• to design an experimental learning environment with further analysis of MOOC retention rates and the performance of in-service training professionals enrolled in professional development courses;

• to examine the relationship between a well-organised interaction environment and the course progress of participants;

• to study the relationship between the engagement of learners in their interaction and their course performance.

Methods

Course design

The implementation of the “MOOC in Education” course proved the viability of MOOC theoretical fundamentals in practice. It was designed as an online course in the framework of professional development courses for educators. The course was launched on the “University without Borders” platform in the autumn of 2017 (https://distant.msu.ru/course/view. php?id=1164).

The course is intended to prepare professionals for the integration of MOOCs into the curriculum of educational institutions. It also provides assignments to develop the skills necessary for online course design and maintenance. Moreover, the project reveals certain psychological, pedagogical and organisational aspects of course design. Learning materials also focus on MOOC quality standards and ways of their assessment.

The target audience of the course is mostly represented by professionals from various institutions of higher education and vocational training, specialists in professional development, teachers, trainers and students from pedagogical universities.

The participants signed up on the “University without Borders” website designed by the Lomonosov Moscow State University (distant. msu.ru) and were provided access to the original learning content: video lectures, presentations, auto-graded quizzes and additional materials.

The course has a three-stage structure:

• MOOC theoretical fundamentals;

• consolidating assignments;

• final assessment.

The course was provided for free and delivered online on a voluntary basis.

Beta-testing of the online course revealed numerous benefits and opportunities of eLearning, with unlimited number of potential enrolments via the offered MOOC format.

More than 3100 participants completed the course from the launch of the course on 1 November 2017 till 1 March 2019. The course participants belong to various institutions of higher education and vocational training from all federal districts of the Russian Federation. The average age of students is 43 years.

A variety of mixed research methods and approaches were applied in the course of this study:

1. Mathematical methods (quantitative analysis, grouping and recording variables, categorisation). The study of interaction patterns is based on the overall course performance and final test results. The significant part of the research focused on testing the hypothesis of an relationship between successful assignment performance and the unlimited number of attempts to pass course tests. Another crucial task was to analyse the relationship between interaction patterns used on the platform and the overall course progress of the participants. This approach also indicates the degree of participants' engagement in discussion forums and their interaction with other members and moderators.

2. Content analysis was used to outline the most appealing topics in discussion sections for course members.

Experiment description The control group of 736 participants started the course on 1 November 2017. The course was designed with specified start and end dates (with 31 December 2017 as a starting date) and scheduled for 2 months (8 weeks). 411 participants completed the course successfully, which constitutes 55% of the total number.

Fig. 3 The comparative chart of training effectiveness for the fixed-schedule MOOC and the “on demand” MOOC

Fig. 4 Completion dates for the “on demand” MOOC

The experimental group of 199 participants started learning on 1 December 2017. The course was designed as an “on-demand” MOOC with 31 December 2019 as the end date (2 years long). Further statistical analysis revealed that 149 users stayed on the course till the final assessment phase, which is 74.8% of the total number of the participants in the “on-demand” MOOC.

The diagram provides an overview of statistical data showing that 89.32% of the participants passed the course within 3-5 months of the specified schedule. One individual completed the course in a year and a month.

Evidently, when choosing a MOOC type for professional development courses, it is important to take into account the needs of the learners. If we focus on a short period of training, it is recommended to set time limits with fixed deadlines to stimulate students' course performance. More than a half of the participants passed the fixed-schedule course during the period of 8 weeks, while only 2% of the participants completed the course with a flexible schedule during the same time period. On the other hand, “on- demand” courses are preferable if we are interested in the unlimited enrolment of working professionals. Three quarters of the signed-up users completed the course with a flexible assessment schedule.

The correlation between interaction patterns and progress

Table 1

Topics of the online course forum

Topics	Number of messages	% of the total message number
Certification (registration, learning process)	22	23.66%
Getting a certificate	28	30.11%
Technical issues	29	31.18%
Evaluation (tests and essays)	6	6.45%
Other	8	8.6%

One of the research objectives was to determine whether younger participants pass the course within a shorter period of time and demonstrate better performance compared to older members, who might be less efficient with digital technologies and online networks. Another hypothesis suggested that professionals under the age of 40 could be more active in discussion forums since they are accustomed to similar practices, such as using social networks, messengers, etc. on a daily basis.

Quantitative research indicators:

• progress tests grades (maximum 190 points);

• final assessment grades (maximum 150 points);

• writing an essay (maximum 20 points);

• number of tests attempts (unlimited);

• number of forum messages;

• age.

Qualitative indicators:

• sex;

• message content.

It should be noted that 72% of the participants were women, while 28% were men; possibly, due to the predominance of female professionals working in the field of higher education (about 80% of the teaching staff in institutions of higher education are women, based on the 2015 statistics) [4]. 34% of the participants are younger than 35, while 66% are older.

The recorded data shows that only 41 course members out of the 200 who signed up participated in forum discussions, which comprises 20.5% of the total number. The number of female participants was 3 times higher than the number of male ones.

60 140 * I

0 10 20 30 40 50 60 70 80 90 100

Fig. 5 Comparative gender analysis of communicative activity on MOOC

They wrote 93 messages on the course forum. Major topics are shown in Table 1.

Lead-in procedure issues and technical specifications were among the most popular topics; these messages were not addressed to other course members. The enrolled participants almost never interacted with each other. The posts are short (about 2-3 sentences) and indifferent.

The other messages (8.6%) are divided into two groups. The first part is devoted to personal information about registered applicants and their professional background. The second category comprises medium-specific discussions. Results of the content analysis showed that participants provide feedback on various topics but do not interact directly with each other. The most informative and direct responses belong to men, while messages intended to clarify misunderstandings are written by women.

The current study demonstrated participants' poor interaction on the course forum. The majority of the course members are older than 35 years. This fact disproves the initial research hypothesis (46.34% female and 19.51% male members).

A correlation was found between the participants' overall performance and their age with the number of test attempts and engagement in interaction. The following graded evaluation marks were set in the study: “satisfactory” (up to 300 points), “good” (301-345) and “excellent” (346-360).

Table 2

Effectiveness of learners with a “satisfactory” grade

Up to 300 points
Age	Number of learners (% of the total number)	Average number of additional attempts	Number of the forum participants
Up to 35	7 (3.5%)	13	1
Up to 50	5 (2.5%)	20	1
Over 50	10 (5%)	7	0
Total	22 (11%)	40	2

The results of the research show that one tenth of all participants passed the course with a “satisfactory” grade. Moreover, they used additional attempts to complete course tests after receiving poor results for the assignments. Only two individuals from this group were forum users.

More than a half of the course participants fall into the large group who received “good” grades. Most students belong to the youngest group, who showed better results (about 1.5-2 times better). 16 members were active forum users, which is 8 times more than in the previous group with poor results. The average number of additional test attempts is less than that recorded among the participants with a “satisfactory” grade. The following table demonstrates the statistical data.

Table 3

Effectiveness of learners with a “good” grade

301-345 points
Age	Number of learners (% of the total number)	Average number of additional attempts	Number of the forum participants
Up to 35	47 (23.5%)	8	5
Up to 50	30 (15%)	6	6
Over 50	27 (13.5%)	7	5
Total	104 (52%)	21	16

Performance rates of professionals who passed the online professional development course with an “excellent” grade are shown in Table 4.

Table 4

Effectiveness of learners with an “excellent” grade

346-360 points
Age	Number of learners (% of the total number)	Average number of additional attempts	Number of the forum participants
Up to 35	33 (16.5%)	19	11
Up to 50	19 (9.5%)	13	3
Over 50	22 (11%)	12	9
Total	64 (32%)	44	23

This group comprises 32% of the participants who scored the highest results due to their strong motivation. The average number of additional test attempts is much higher than in the previously described groups.

Professionals under 35 years obtained the best course progress grades; however, other age groups received almost the same grades, therefore refuting the original research hypothesis. It should be noted that 23 members were active forum users (56% of the group).

Results

The case study examined the experimental learning results of 900 course participants enrolled in a professional development course with integrated MOOC types. The study also focused on individual learning approaches of students and their course progress with various time limits for completion.

MOOC retention rates depend on various factors that were revealed in this research. Furthermore, the experiment provided analysis of the students' performance and interaction on the course forum. Therefore, the research results serve to assess the effect of MOOC on the ongoing learning process.

The research results suggest that online course progress does not depend on the participants' sex and age. However, participants that engage in the forum interaction tend to score higher passing grades.

Discussion

The study provided the following significant results.

1. Fixed-schedule MOOCs with specified deadlines have a positive effect on adults enrolled in short-term professional development courses and generally motivate them. More than a half of the participants completed the course within the 8-week term compared to 2% of members taking a MOOC with a flexible schedule. Assessment with deadlines affects students' overall performance.

2. The opportunity to complete a course with a flexible schedule without deadlines for assessment has positive effect on adults pressed for time (three quarters of the enrolled members passed the course). The length of the course ranged from one to two years, providing a better learning environment.

3. In terms of interaction, it was found that only 20% of the course participants post on discussion forums, which is quite a small number demonstrating low student engagement (80%). The majority of topics are focused on the platform technical issues, the assessment procedure or earning a certificate of completion. Feedback and opinion posts demonstrating students' engagement in the course are scarce.

No predominant age group was revealed in the course of the study. It proves that forums are user-friendly in facilitating discussion for all categories of professionals.

4. The results of the study confirm a direct correlation between the course performance, the number of additional test attempts and the degree of engagement in forum discussions. The students with an “excellent” grade were active forum users, as opposed to those that received “good” and “satisfactory” grades.



Conclusion

Integrating MOOCs into professional development courses proved to be efficient. This format facilitates continuous lifelong learning and addresses important learners' needs, such as accessibility, large-scale potential enrolment, flexibility and exposure.

The models of MOOC professional development presume the following.

• Flexible schedule. Self-paced MOOCs provide opportunities for the development of personal learning plans and strategies.

• Feedback. Course members are engaged in interaction via a discussion forum, webinars and chats, thus showing better performance.

* Transparent assessment. Auto-graded quizzes and peer-reviewed assignments determine the overall course progress.

The most significant conclusion is that retention rates do not depend on the course design. To increase retention rates and the number of members who complete the course successfully, it is vital to focus on the primary needs and learning objectives of students while choosing a MOOC format and designing the interaction environment. It is recommended to offer a fixed- schedule MOOC type for short-term professional development courses with reliable course maintenance and experienced tutors. “On-demand” MOOCs with a flexible schedule may also increase the number of potential participants but require proper interaction arrangements.



Список литературы

1. Гущина О. М., Михеева О. П. Массовые открытые онлайн-курсы в системе подготовки и повышения квалификации педагогических кадров // Образование и наука. 2017. Т. 19. № 7. С. 119-136.

2. Двоеглазов Д. В., Дешко И. П., Кряженков К. Г. Массовый открытый онлайн-курс по требованию: опыт реализации и результаты // Интернет-журнал «Мир науки». 2016. Т. 4 № 2. С. 1-10.

3. Кузьминов Я. И., КарнойМ. Онлайн-обучение: как оно меняет структуру образования и экономику университета. Открытая дискуссия Я. И. Кузьминов -- М. Карной // Вопросы образования. 2015. № 3. С. 8-43.

4. Пугач В. Ф. Гендерный состав преподавателей российских вузов // Высшее образование в России. 2015. Т. 24. № 12. С. 78-88.

5. Семенова Т. В., Вилкова К. А., Щеглова И. А. Рынок МООК: перспективы для России // Вопросы образования. 2018. № 2. С. 173-195.

6. Friedman L. W., Friedman H. H. Using social media technologies to enhance online learning // Journal of Educators Online. 2013. Vol. 10 (1). P. 1-21.

7. Hashizume A., Kurosu M., Kaneko T. The choice of communication media and the use of mobile phone among senior users and young users // Proceedings of the 8th Asia-Pacific Conference on Computer Human Interaction. 2008. P. 427-436.

8. Khalil H., Ebner M. MOOCs completion rates and possible methods to improve retention -- A Literature Review // Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications. 2014. Р. 1236-1244.

9. Koller D., Ng A., Do C., Chen Z. Retention and intention in massive open online courses // Educause Review Online, June 3, 2013. [Electronic resource]. URL: https://er.educause.edu/articles/2013/6/retention-and-intention- in-massive-open-online-courses

10. LiN., Himanshu V., Skevi A., Zufferey G., Blom J., DillenbourgP. Watching MOOCs together: Investigating co-located MOOC study groups // Distance Education. 2014. Vol. 35 (2). P. 217-233.

11. Reich J. MOOC completion and retention in the context of student intent // Educause Review Online, December 8, 2014. [Electronic resource]. URL: https://er.educause.edu/articles/2014/12/mooc-completion-and- retention-in-the-context-of-student-intent

References

1. Guschina O. M., Miheeva O. P. Massovyie otkryityie onlayn-kursyi v sisteme podgotovki i povyisheniya kvalifikatsii pedagogicheskih kadrov // Obrazovanie i nauka. 2017. T. 19. No. 7. S. 119-136.

2. DvoeglazovD. V., DeshkoI. P., KryazhenkovK. G. Massovyiy otkryityiy onlayn-kurs po trebovaniyu: opyit realizatsii i rezultatyi // Internet-zhurnal “Mir nauki” 2016. T. 4. No. 2. S. 1-10.

3. Kuzminov Ya. I., Karnoy M. Onlayn-obuchenie: kak ono menyaet strukturu obrazovaniya i ekonomiku universiteta. Otkryitaya diskussiya Ya.I. Kuzminov -- M. Karnoy // Voprosyi obrazovaniya. 2015. No. 3. S. 8-43.

4. Pugach V. F. Gendernyiy sostav prepodavateley rossiyskih vuzov // Vyisshee obrazovanie v Rossii. 2015. T. 24. No. 12. S. 78-88.

5. Semenova T. V., Vilkova K. A., Scheglova I. A. Ryinok MOOK: perspektivyi dlya Rossii // Voprosyi obrazovaniya. 2018. No. 2. S. 173-195.

6. Friedman L. W., Friedman H. H. Using social media technologies to enhance online learning // Journal of Educators Online. 2013. Vol. 10 (1). P. 1-21.

7. Hashizume A., Kurosu M., Kaneko T. The choice of communication media and the use of mobile phone among senior users and young users // Proceedings of the 8th Asia-Pacific Conference on Computer Human Interaction. 2008. P. 427-436.

8. Khalil H., Ebner M. MOOCs completion rates and possible methods to improve retention -- A Literature Review // Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications. 2014. P.1236-1244.

9. Koller D., Ng A., Do C., Chen Z. Retention and intention in massive open online courses // Educause Review Online, June 3, 2013. [Electronic resource]. URL: https://er.educause.edu/articles/2013/6/retention-and-intention- in-massive-open-online-courses

10. LiN., Himanshu V., Skevi A., Zufferey G., Blom J., DillenbourgP. Watching MOOCs together: Investigating co-located MOOC study groups // Distance Education. 2014. Vol. 35 (2). P. 217-233.

11. Reich J. MOOC completion and retention in the context of student intent // Educause Review Online, December 8, 2014. [Electronic resource]. URL: https://er.educause.edu/articles/2014/12/mooc-completion-and- retention-in-the-context-of-student-intent

Размещено на Allbest.ru

...