Assessing the quality of the didactic process on the base of its monitoring with the use of ICT

Размещено на http://www.allbest.ru/

Размещено на http://www.allbest.ru/

Assessing the quality of the didactic process on the base of its monitoring with the use of ICT

Pawet Plaskura

The article deals with the problem of evaluating and ensuring the quality of the didactic process with the use of modern technologies, which makes it possible to solve the problem of objectivity in the assessment of the didactic process. The current problem is to use the proprietary platform QUELA for managing the learning process and the Dero microsystems simulator used to model individual learning process for each student. The implementation of this problem requires an interdisciplinary approach and is an integration of knowledge in the fields of pedagogy, psychology, computer science, electronics and ergonomics. The best results are obtained by mixing traditional methods with innovative technologies, which leads to mixed education (b-learning). The article focuses on the issue of monitoring the didactic process, ensuring fair assessment of students' knowledge and prevention of poor quality of the didactic process.

If the didactic process is carried out in an educational environment using ICT оп the platform Quela, it is possible to identify the quality of students' knowledge based on the analysis of activity data, i.e.:

frequency of using materials;

length of time of using materials;

type of materials that have been used.

If there is a discrepancy between the predicted and received assessment, an analysis of the causes of this phenomenon should be made. The reasons can be:

unethical behavior of the student during the exam (copying);

using other sources outside the system;

initial knowledge, which the student did not disclose during the preliminary test.

In the case when the lecturer is not sure about the quality of assessment, he should use knowledge verification mechanisms, including, for example, additional verification of knowledge - a repeated test or a narrative diagnostic interview. The results of the additional verification of students' knowledge ensure an objective approach to their assessment and improve the quality of education. The use of the ICT on the platform Quela system enables an objective implementation of an individual approach to monitoring the level of knowledge of students and preventing the negative effects of the learning process. In the sense of the ethics of behavior of students and lecturers, the exam assessment isn't accidental but predictable and depends on the effort attached by the student. It is fair and implements the democratic principles of education.

Key words: monitoring, didactical process, modeling, quality, assessment, objectivity, democratization, ICT. dero quela pedagogy

У статті розглядається проблема оцінювання й забезпечення якості дидактичного процесу з використанням ІКТ на основі платформи QUELA та симулятора мікросистем Dero, що дозволяє забезпечити об'єктивність під час визначення рівнів засвоєння студентами знань і вмінь. Реалізація цієї проблеми вимагає міждисциплінарного підходу та інтеграції знань у галузі педагогіки, психології, інформатики, електроніки й ергономіки. Найкращі результати отримують шляхом інтегрування традиційних методів з інноваційними технологіями, що веде до змішаної освіти (b-learning). У статті схарактеризовано приклади моніторингу дидактичного процесу під час викладання у вищій школі, шляхи забезпечення обєктивної оцінки знань студентів та попередження низької якості дидактичного процесу.

Ключові слова: моніторинг, дидактичний процес, моделювання, оцінка, якість, об'єктивність, демократизація, ІКТ.

В статье рассматривается проблема оценки и обеспечения качества дидактического процесса c использованиeм ИКТ на основе платформы QUELA и симулятора микросистем Dero, что позволяет обеспечить объективность при определении уровней усвоения студентами знаний и умений. Реализация этой проблемы требует междисциплинарных подходов и интеграции знаний в области педагогики, психологии, информатики, электроники и эргономики. Лучшие результаты получают путем интеграции традиционных методов с инновационными технологиями, что ведет к смешанному образованию (b-learning). В статье охарактеризованы примеры мониторинга дидактического процесса во время преподавания в высшей школе, пути обеспечения объективной оценки знаний студентов и предупреждения низкого качества дидактического процесса.

Ключевые слова: мониторинг, дидактический процесс, моделирование, оценка, качество, объективность, демократизация, ИКТ.

Introduction. The article presents the problem of evaluation of the didactic process with the help of ICT system to manage the didactic process using the platform Quela [21, 28] and the simulator of the microsystems Dero [20] that enables description of the didactic process. The system makes it possible to solve the objectivity problem of assessing the didactic process. The relevant problem is the using of ICT as the tools of modeling and monitoring the quality of the learning process. The implementation of this problem requires an interdisciplinary approach and is an integration of knowledge in the fields of pedagogy, psychology, computer science, electronics and ergonomics. The best results are obtained by mixing traditional methods with innovative technologies, which leads to blended education (b-learning). Modeling process requires the use of advanced technologies. Monitoring of the didactic process is a very current problem in terms of ensuring its quality. The monitoring process in traditional terms is a time-consuming process. It requires involvement of people and resources. Modern information technologies can help in the quality monitoring process; they can give it ergonomic features. The most important issues of monitoring the teaching process are:

ensuring fair assessment of students' knowledge;

preventing the poor quality of the teaching process.

Analysis of relevant research. Problems related to the modeling of the didactic process and remote education were highlighted in the writings of Polish scholars, among others: B. K^dzierska [15; 14] (competences), P. Rozewski, E. Kusztina, R. Tadeusiewicz, and O. Zaikin [24] (distribution of competences in the education system), P. A. Wozniak and E. J. Gorzelanczyk [26; 27], (SuperMemo material repetition algorithm), W. Furmanek [5], B. Polak [22], Cz. Kupisiewicz [13]. In the world this subject was dealt with, among others S. S. Buckingham and

R. Deakin [1], M. A. Chatti, A. L. Dychhoff, U. Schroeder, and H. Thus [2], M. K. DiBenedetto and H. Bembenutty [3], G. Gray, C. McGuinness, P. Owende, and A. Carthy [7], A. Jonsson, J. Johns, H. Mehranian, I. Arroyo, B. Woolf, A. Barto,

Fisher, and S. Mahadevan [11], R. J. Mislevy, J. T. Behrens, and K. E. Dicerbo [16], Z. A. Pardos, R. S. J. Baker, S. M. Gowda, and N. T. Heffernan [19], J. Herszage and N. Censor [8], M. J. Jaap, J. M. J. Murre, and J. Dros [10], D. C. Rubin, S. Hinton, and A. Wenzel [23], J. R. Whitlock, A. J. Heynen, M. G. Shuler, and M. F. Bear [25], G. Fioretti [4], M. C. F. Panadero, A. Pardo, J. F. Panadero, and A. M. Lopez [17], W. Horton [9], R. M. Gagne et al. [6], A. Pardo [18], S. Juszczyk [12]. These issues have been dealt with by the author of the article for over 10 years. This topic requires further deepening.

The aim of the article is to show the use of information technology in solving the problem of ensuring the quality of the didactical process in higher education.

Research methods. To achieve the set goal, theoretical methods of research were used: systematic analysis, comparison and generalization of the relevant results of research in the international space for studying the approaches to monitoring the didactic process; systematization of pedagogical experience using ICT for model and simulation students' learning activity. The practical testing of the efficiency of the described pedagogical approaches to the assessing students learning activity using ICT in the process of training of the future employees in the sphere of national security have been conducted with the help of pedagogical observation, experimental research and interpretation of the obtained data.

Results and discussion. The approach presented in the work enables the implementation of generalized stages of monitoring in terms of individual cognitive processes of students. The project designed and implemented by the author is a system that combines the didactic process, modern technologies and cognitive processes in terms of monitoring issues to ensure high quality of education. The issues of modeling and simulation of the didactic process can be used in order to:

determine the effects of course education;

design the course or student's activity in terms of maintaining the appropriate level of knowledge;

monitor the effects of education.

The article presents real examples of simulations of designed courses and individual monitoring of the didactic process. For the needs of the above mentioned tasks it is required: having an electronic system for monitoring the didactical process (Quela), introducing the electronic course structure to the system, describing individual materials with appropriate sets of parameters, having the competence of the lecturers in the use of the system and analysis of the results, students having competence in the field of information technology and use of the Quela system to analyze the results obtained.

Examples of monitoring of the quality of the didactic process using the Quela system are presented. Two courses for students of faculty of social sciences specialty "National security" at the Jan Kochanowski University (Poland) conducted in the years 2012-2014 were analyzed. The result of the exam (on a scale of 0-100 points) was converted into a grade according to the scale contained in the Table 1.

Table 1 Points to grades converter

The Quela [21, 28] system was used to monitor the didactic process for 377 students. In the following, the actual results of the didactic process for three selected students for two different courses carried out with an interval of one year will be presented.

The Information Technology (TI) course was run on the first semester of studies in the academic year 2012-2013. The course consists of eight practical lessons (C1.8), to which didactic materials in electronic form were prepared. At the end of the course, a knowledge test, consisting of 10 questions was carried out. The questions covered all topics. The Table 2 presents the organization of the designed real course for one of the student groups.

Table 2 Designed organization of the TI course

The default parameters of the element models describing the learning process were adopted. The minimum score of the exam was assumed at the level of 0.51, i.e. 51 pts (3.0 score - Table 1). The simulation results of the designed course are shown in the Figure 1. The figure shows the course of changing the level of knowledge over time related to forgetting (TI - Simulated Knowledge). Based on the results of the simulation, it can be seen that at the moment of the exam the expected level of knowledge of the student should reach the level of about 0.77, i.e. 77 points, which gives the assessment of 4.0 (Table 1). Achieving a lower level by students means that the workload was insufficient, there was absenteeism or the material was not repeated. A course simulation for three sample students was carried out. Initial knowledge of students was tested. None of the students had initial knowledge of the material covered by the program. The simulation results are shown in the Figure 1.

Figure 1. Results of simulation of the didactic process for the TI course.

The simulated course of the level of knowledge and the level checked at the moment of the exam is presented. The results of the pass were converted to the normalized level. Changes in the level of knowledge are related to the effect of learning (rapid increases) and forgetfulness (gentle decreases). At the end of the course, the credit presented in the figure as a vertical bar took place. The evaluation of knowledge was planned on January 14, 2013. In the next class, on January 21, 2013, the exam was discussed and the selected material was repeated.

The Figure 1 shows a short replay of material by the student KD. Theoretically, the repetition should raise the level of simulated knowledge, but it did not happen. In this case, the predicted and actual levels of knowledge are almost identical. Although the simulated result is slightly lower than the result achieved, the student received the rating 3.0 (Table 1). The results of simulation of the didactic process for the student SJ (Figure 1) do not match the real level of knowledge. The difference is 2 grades. You can't see any additional student activities outside of the classroom. There is no activity from mid- December to completion in January. Before the completion of the test, a short repetition of the material was made. Nevertheless, the student received the grade 5.0 (Table 1). This indicates the use of other sources or knowledge of issues, which was not confirmed in the input test.

The learning activity of the student TW (Figure 1) began about a month after the course starting. Visible is the lack of activity during the last month - from mid-December to mid-January. In spite of this, TW student received the grade 5.0. This indicates the use of other sources or knowledge of issues, which was not confirmed in the input test. Before the completion of the test, a material repetition was made. In this case, the predicted and actual levels differ by one rating. Based on the simulation, the student should receive the 4.0 grade (Table 1), while the student received the 5.0 grade.

The Information Security (BI) course was run in the sixth semester of studies in 2014. The course consisted of seven lectures (W1.7) and five practical lessons (C1.5). The training and lecture materials as well as relevant tasks to be carried out were available in electronic form. At the end of the course, a knowledge test was carried out. The result of the test was converted into a grade according to the scale from the Table 1. The organization of the actual course is presented in the Table 3.

Table 3 Designed organization of the BI course

The default parameters of the element models describing the learning process were adopted. The result of the final exam was adopted at the level of

0.51, i.e. 51 pts (3.0 score - Table 1). The simulation results of the designed course are shown in the Figure 2. The Figure 2 presents changes in knowledge level in time related to forgetting (Simulated Knowledge). Based on the results of the simulation, it can be seen that at the moment of the exam the student's expected level of knowledge should reach about 0.73, i.e. 73 points, which gives the assessment of 4.0 (Table 1).

The courses were simulated for three of the same students who had previously completed the TI course. Initial level of students' knowledge was tested. None of the students had initial knowledge of the material covered by the program. The level of initial knowledge was zero.

Although the course started in March, the first activity was registered for the student KD in May (Figure 2). Thus, the period of study was shorter but more intense. Unfortunately, student activity was not registered before the exam itself. This may indicate that the student used materials previously downloaded from the platform or from his own notes. Unfortunately, you can't tell for sure. Some indication is the slightly better result of the exam in relation to the simulated value. A short repetition of the material took place before the exam. Unfortunately, the student was not able to use the repetition of knowledge. He received the previously passed 3.0 rating (Table 1).

Student SJ works regularly from the beginning of the course (Figure 2). There is no activity in the last period before the exam, which may indicate that you use other materials, because it is unlikely that the student could pass this course without repetition of the material before the exam.

Based on the results of the simulation, it can be seen that at the moment of the exam the expected level of knowledge almost coincides with the achieved result of 4.5 (Table 1).

The TW student's activities are recorded from the beginning of the course (Figure 2). There is no activity in the last period before the exam, which may indicate that the student used other materials. As in the case of the student SJ, it is unlikely that the student could pass the exam without repetition of the material. Based on the results of the simulation, it can be seen that at the moment of the exam the student's level of knowledge should reach the level of about 0.67, i.e. 67 points, which gives the rating 3.5 (Table 1), but in reality, it reached the level of 0.95, i.e. 95 points which gives the rating of 5.0.

Conclusions

If the didactic process is carried out in an educational environment using ICT оп the platform Quela, it is possible to identify the quality of students' knowledge based on the analysis of activity data, i.e.:

frequency of using materials;

length of time of using materials;

type of materials that were used.

If there is a discrepancy between the predicted and received assessment, an analysis of the causes of this phenomenon should be made. The reasons can be:

unethical behavior of the student during the exam (copying);

using other sources outside the system;

initial knowledge, which the student did not disclose during the preliminary test.

Figure 2. Results of simulation of the didactical process for the BI course.

In the case when the lecturer is not sure about the quality of assessment, he should use knowledge verification mechanisms, including, for example, additional verification of knowledge - a repeated test or a narrative diagnostic interview. The results of the additional verification of students' knowledge ensure an objective approach to their assessment and improve the quality of education. The use of the e-platform Quela system enables an objective implementation of an individual approach to monitoring the level of knowledge of students and preventing the negative effects of the learning process. In the sense of the ethics of behavior of students and lecturers, the exam assessment isn't accidental but predictable and depends on the effort attached by the student. It is fair and implements the democratic principles of education. You can compare classical methods (without the use of ICT) and methods using modern ICT to evaluate the didactic process - Table 4.

Table 4

Comparison of evaluating methods of the learning process

Comparing these two approaches, it can be concluded that the use of modern technologies makes it possible to equip the participants of the learning process with knowledge enabling the improvement of the effectiveness and quality of the didactic process.

At the stage of system implementation, it is required to acquire competence in the use of the system and interpretation of results. For this reason, a positive effect will give appropriate training for lecturers.

Prospects of further research. In the future it is planned to describe the content and methods of training pedagogues of higher school to use the developed technologies in practice.

References

1. Buckingham, S. S., Deakin, C. R. (2012). Learning dispositions and transferable competencies: Pedagogy, modelling and learning analytics. Second International Conference on Learning Analytics and Knowledge.

2. Chatti, M. A., Dychhoff, A. L., Schroeder, U., and Thus, H. (2012). A reference model for learning analytics. International journal of technology enhanced learning. Special Issue on State of the Art in TEL, 318-331.

3. DiBenedetto, M. K., Bembenutty, H. (2013). Within the pipeline, self-regulated learning, self-efficacy, and socialization among college students in science courses. Learning and Individual Differences, 23, 218-224.

4. Fioretti, G. (2007). The organizational learning curve. European Journal of Operational Research, March 2007. ISSN 1375-1384. doi:10.1016/j.ejor.2005.04.009.

5. Furmanek, W. (2014). Wspomaganie rozwoju kompetencji tworczych, chapter Innowacyjnosc w pracy wyzwaniem spoteczenstwa wiedzy, 8-32. Wyd. FOSZE, Rzeszow, 2014. ISBN 978-83-7586-106-8.

6. Gagne, R. M. et al. Principles of Instructional Design. Thomson/Wadsworth, Belmont, CA, 5th edition, 2005. ISBN 978-0534582845.

7. Gray, G., McGuinness, C., Owende, P., and Carthy, A. (2014). A review of psychometric data analysis and applications in modelling of academic achievement in tertiary education. Journal of Learning Analytics, 1 (75-106), 219-245, ISSN 1929-7750.

8. Herszage, J., Censor, N. (2017). Memory reactivation enables long-term prevention of interference. Current Biology.

9. Horton, W. (2011). E-Learning by design. Pfeiffer.

10. Jaap, M. J., Murre, J. M. J. and Dros, J. (2015). Replication and analysis of

11. Ebbinghaus' forgetting curve. Plus One, 2, 396-408, July 2015.

12. Jonsson, A., Johns, J., Mehranian, H., Arroyo, I., Woolf, B., Barto, A., Fisher, D., and Mahadevan, S. (2005). Evaluating the feasibility of learning student models from data. Technical report. American Association for Artificial Intelligence.

13. Juszczyk, S. (2002). Edukacja na odlegiosc, Kodyfikacja pojqc, regui i procesow. Wyd. Adam Marszatek, Torun, Poland.

14. Kupisiewicz, Cz. (2005). Podstawy Dydaktyki Ogolnej. WSiP, Warszawa, ISBN 8302093106.

15. K^dzierska, B. (2007). Kompetencje informacyjne w ksztaiceniu ustawicznym. Instytut Badan Edukacyjnych, Warszawa.

16. K^dzierska, B. (2007). Przedsiqbiorstwo w rozwoju zawodowym pracownikow, chapter Kompetencje informacyjne w ksztatceniu zawodowym, 131-138. Instytut Badan Edukacyjnych, Warszawa.

17. Mislevy, R. J., Behrens, J. T., and Dicerbo, K. E. (2012). Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining. Journal of Educational Data Mining, 4 (1), 11-48.

18. Panadero, M. C. F., Pardo, A., Panadero, J. F., and Lopez, A. M. (2002). A mathematical model for reusing student learning skills across didactical units. 32nd ASEE/IEEE Frontiers in Education Conference, November 6-9 2002. ISSN 0-7803-7444-4/02.

19. Pardo, A. (2002). A platform for Parametrized Exercises in Web Based Education. Proceedings of SITE.

20. Pardos, Z. A., Baker, R. S. J., Gowda, S. M., and Heffernan, N. T. (2011). The sum is greater than the parts: Ensembling models of student knowledge in educational software. SIGKDD Explorations, 13 (2), 37-44.

21. Plaskura, P. (2013). Symulator mikrosystemow Dero v4. Metody i algorytmy obliczeniowe, modelowanie behawioralne, przykiady. (Microsystems simulator Dero v4. Computational methods and algorithms, behavioral modeling, examples). AIVA, ISBN 978-83937245-1-2.

22. Plaskura, P. (2016). Quela - a platform for managing the didactical process, 337340. Poltava V. G. Korolenko National Pedagogical University, Pottawa, Ukraine, ISBN 378.016:5-028.31(063).

23. Polak, B. (Ed.) (2013). Podstawy teorii ksztaicenia. Szczecinska Szkota Wyzsza Collegium Balticum, Szczecin, ISBN 978-83-61082-81-1.

24. Rubin, D. C., Hinton, S., and Wenzel, A. (1999). The precise time course of retention. Journal of Experimental Psychology: Learning, Memory, and Cognition, 1161-1176.

25. Rozewski, P., Kusztina, E., Tadeusiewicz, R., and Zaikin, O. (2011). Intelligent Open Learning Systems: Concepts, Models and Algorithms. Intelligent Systems Reference Library. Springer Berlin Heidelberg, ISBN 9783642226670. Retrieved from: https://books.google.pl/books?id=Mnz4zY9T2CIC.

26. Whitlock, J. R., Heynen, A. J., Shuler, M. G., Bear, M. F. (2006). Learning induces

27. long-term potentiation in the hippocampus. Science, 313 (5790), 1093-1097, ISSN 00368075. DOI:10.1126/science.1128134. Retrieved from: http://science.sciencemag.org/ content/313/5790/1093.

28. Wozniak, P. A., Gorzelanczyk, E. J. (1998). Hypothetical molecular correlates of the two-component model of long-term memory. The 7-th International Symposium of the Polish Network of Molecular and Cellular Biology UNESCO/PAS, June 1998.

29. Wozniak, P. A., Gorzelanczyk, E. J., Murakowski, J. (2005). The two-component model of long-term memory. Symbolic formula describing the increase in memory stability for varying levels of recall. Cybernetic Modelling of Biological Systems.

Размещено на Allbest.ru