Evaluation of the MOOCs Quality and Its Effectiveness for Teachers’ Training in the Field of Digital Competences and Their Use in Education: A Case Study
Abstract
This study presents the research results obtained after the assessment of the digital competences of the pre-service and in-service teachers after their completion of the MOOC “Contemporary ICT Tools and Innovative Methods of Creative Education”. The paper provides a short description of the MOOC, requirements to pass the course and analysis of the learning outcomes through the students’ self-evaluation and feedback. The MOOC was developed in Polish and English within the project “MOOCs for Sciences of Education” and hosted on the Polish MOOCs platform Navoica (www.navoica.pl) within the framework of the competition, initiated by of Ministry of Education and Science of Poland and National Center for Research and Development (NCBR ‒ Narodowe Centrum Badań i Rozwoju) on “Direction to the MOOC”. Keeping in mind that Massive Open Online Courses (MOOCs) have developed into a mainstream for universities, education reformers and startup companies, especially in the time of the COVID-19, the study is believed to contribute to the development of the MOOC pedagogy, and address the question of the MOOCs effectiveness for students’ learning outcomes and satisfaction. The experimental MOOC “Contemporary ICT Tools and Innovative Methods of Creative Education”, which was aimed at enhancing teachers’ digital competences, contained 8 modules, and was attended by more than 90 students through its 1st edition in 2020‒2021. The conclusions of the MOOC in focus and overall recommendations on enhancing the MOOCs effectiveness for formal education and learning outcomes have been evaluated based on the research data and provided accordingly. The selected statistical analyses and the data comparisons were made using Wilcoxon’s test at the significance level alpha = 0.05. The normality of the distributions of the studied variables was checked using the Shapiro Wilk test.
Keywords
MOOCs; education; digital competences; teachers; ICT-tools in education
References
Aldahmani, S., Ali Al-shami, S., Adil, H., & Sidek, S., (2020) A Review Paper on MOOCs Development Stages, Types, and Opportunities and Challenges. Systematic Reviews in Pharmacy, 2020; 11(12): 172‒179, https://doi.org/10.31838/srp.2020.12.28
Cabero-Almenara J, Gutiérrez-Castillo J-J, Palacios-Rodríguez A, & Barroso-Osuna J. (2020) Development of the Teacher Digital Competence Validation of DigCompEdu Check-In Questionnaire in the University Context of Andalusia (Spain). Sustainability. 2020; 12(15):6094. https://doi.org/10.3390/su12156094
Cabero-Almenara, J., Romero-Tena, R., & Palacios-Rodriguez, A. (2020). Evaluation of Teacher Digital Competence Frameworks Through Expert Judgement: the Use of the Expert Competence Coefficient. Journal of New Approaches in Educational Research, 9(2), 275‒293, https://doi.org/10.7821/naer.2020.7.578
Condruz-Bacescu, M. (2018) Free Online Education - The Future of a Better World? In Roceanu I., Topor, S., Holotescu, C., Radu, C., Nitu, F., Grosseck, G., Radoi, M. (Eds.) Elearning Challenges and New Horizons, (pp. 303‒310) https://doi.org/10.12753/2066-026X-18-257
Cong, R., Tago, K. & Jin, Q. (2022) Measurement and verification of cognitive load in multimedia presentation using an eye tracker. Multimedia Tools and Applications 6(7), 1‒15, https://doi.org/10.1007/s11042-022-13294-0
Cross, J. S., Keerativoranan, N., Carlon, M. K. J., Tan, Y. H. Rakhimberdina, Z. & Mori, H., (2019) Improving MOOC quality using learning analytics and tools, 2019 IEEE Learning With MOOCS (LWMOOCS), 2019, (pp. 174-179), http://doi.org/ 10.1109/LWMOOCS47620.2019.8939617.
Deng, R., Benckendorff, P., & Gannaway, D. (2019). Progress and new directions for teaching and learning in MOOCs. Computers & Education 129, 48–60, https://doi.org/10.1016/j.compedu.2018.10.019
Deterding, S., (2011). Situated motivational affordances of game elements: A conceptual model, in: Presented at Gamification: Using Game Design Elements in Non-Gaming Contexts, a workshop at CHI 2011. http://gamification-research.org/wp-content/uploads/2011/04/09-Deterding.pdf (Accessed 28 October 2015).
Gordillo A., López-Pernas S., & Barra E. (2019). Effectiveness of MOOCs for teachers in safe ICT use training. Comunicar. Open Access. 27(61), 98-107, 2019. ISSN 11343478, https://doi.org/10.3916/C61-2019-09
Grodecka, K., Mokwa-Tarnowska, I., & Peszko, P. (2019). Praca zbiorowa NAVOICA Polska platforma edukacyjna oferująca kursy typu MOOC Wytyczne dla twórców kursów. A. Kaczmarek-Kacprzak, K. Kurowska-Wilczyńska, B. Muczyński (red.) [Warszawa 2019 NAVOICA Polish educational platform offering MOOCs Guidelines for course developers.] Wydawnictwo Fundacja Młodej Nauki, Warszawa 2020.
Kameas, A., Quarta, B. & Maratou, V. (2018) Educating Global Citizens with the Help of a MOOC, EDULEARN18 Proceedings, pp. 10933‒10942.
Kim, D. & Choi, M. (2022) Impacts of Device Caching of Content Fractions on Expected Content Quality, in IEEE Wireless Communications Letters, 11(5), (pp. 1022‒1026), May 2022, htttp://doi.org/10.1109/LWC.2022.3153125.
Koukis N., & Jimoyiannis A., (2018) MOOCs and teacher professional development: A case study on teachers’ views and perceptions MCCSIS 2018 ‒ Multi Conference on Computer Science and Information Systems In Proceedings of the International Conferences on e-Learning 2018 Volume 2018-July, (pp. 19–26) International Conference e-Learning 2018, part of the Multi Conference on Computer Science and Information Systems 2018, MCCSIS 2018 Madrid 17 July 2018 through 19 July 2018 Code 912893820 ISBN 978-989853378-4
Landers, R. N. (2014). Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming, 45(6), 752–768.
Majuri, J., Koivisto, J., & Hamari, J. (2018) Gamification of education and learning: A review of empirical literature In Proceedings of the 2nd International GamiFIN Conference (GamiFIN 2018) (pp. 11‒19). Eds. by Koivisto J. & Hamari J. CEUR Workshop Proceedings. 2186. CEURWS. ISSN: 1613‒0073. http://ceur-ws.org/Vol-2186/paper2.pdf. https://trepo.tuni.fi/bitstream/handle/10024/104598/gamification_of_education_2018.pdf
Molanes-Lopez, E.M., Rodriguez-Ascaso, A., Leton E., & Perez-Martin, J. (2021) Assessment of Video Accessibility by Students of a MOOC on Digital Materials for All IEEE Access Open Access. 9, (pp. 72357–72367) Article number 9427559, https://doi.org/10.1109/ACCESS.2021.3079199, ISSN 21693536
Morze, N. & Smyrnova-Trybulska, E. (2021) Web-Based Community-Supported Online Education During the COVID-19 Pandemic. International Journal of Web Based Communities (IJWBC) 17(1), 9–34 http://doi.org/10.1504/IJWBC.2021.10032201
Patru, M., & Balaji, V. (2016) Making sense of MOOCS: a guide for policy makers in developing countries (2016) Commonwealth of Learning, UNESCO. Assistant Director-General for Education, 2010-2018 (Qian Tang). ISBN: 978-92-3-100157-4 2016
Polish MOOCs platform Navoica (www.navoica.pl).
Razzali, K. K. S., Shahbodin, F., & Noor, H. (2016). Measuring validity and reliability of perception of online collaborative learning questionnaire using Rasch model. International Journal on Advanced Science Engineering and Information Technology, December, 966-974. http://dx.doi.org/10.18517/ijaseit.6.6.1343
Seaborn, K., & Fels, D. I. (2014). Gamification in theory and action: A survey. International Journal of Human-Computer Studies, 74, 14–31.
Sebai, D. & Mani, E. (2020) MPEG-DASH users quality of experience enhancement for MOOC videos, 2020 IEEE International Symposium on Multimedia (ISM), 2020, pp. 166-167, https://doi.org/10.1109/ISM.2020.00036.
Segovia García, N. (2021) Quality criteria of a Massive Open Online Course (MOOC) based on students’ assessment [Critères de qualité d’un MOOC basés sur l’évaluation des étudiants][Criterios De Calidad De Un MOOC Basado En La Valoración De Los Estudiantes]. Bordon. Revista de Pedagogia 73(4), 145–160 https://doi.org/10.13042/Bordon.2021.87938, ISSN 02105934
Sekret, I. (2021). Students’ Evaluation of Teaching: International Practices from the Students’ Perspectives. International Journal of Pedagogical Advances in Technology-Mediated Education, 2 (1), 26‒38. ISSN 2651-4427. http://patme-journal.iatels.com/index.php/patme/article/view/students-evaluation-of-teaching
Sekret, I., & Jansen, D. (2019). Multilevel Study of the Higher Education Challenges Caused by the Migration Crisis in Turkey. In: Smyrnova-Trybulska, E., Kommers, P., Morze, N., Malach, J. (eds) Universities in the Networked Society. Critical Studies of Education, vol. 10. Springer, Cham. https://doi.org/10.1007/978-3-030-05026-9_9
Smyrnova-Trybulska E., Morze, N., Sekret, I. McKay, E., & Asquith, K., (2022) Development of the digital competences of pre-service and in-service teachers using the MOOC: Rasch Measurement Model in Assessing. (In Press)
Smyrnova-Trybulska E., Sekret, I., & Morze, N. (2021) Preliminary Analysis of the Development And Implementation of the MOOC Project: A Case Study, In Smyrnova-Trybulska E. (ed.) E-learning in the Time of COVID-19. “E-learning” Series. Vol. 13 (2021) Katowice-Cieszyn: STUDIO NOA for University of Silesia. 2021. (pp. 137‒150). ISSN 2451-3644 (print edition) ISSN 2451-3652 (digital edition) ISBN: 978-83-66055-25-4 https://doi.org/10.34916/el.2021.13.12
Smyrnova-Trybulska, E., McKay, E., Morze, N., Yakovleva, O., Issa, T. & Issa, Th. (2019). Develop and Implement MOOCs Unit: a Pedagogical Instruction for Academics, Case Study In: E. Smyrnova-Trybulska, P. Kommers, N. Morze, J. Malach (Eds.). Universities in the Networked Society. Cultural Diversity and Digital Competences in Learning Communities. Springer. Critical Studies of Education 10 (pp. 103–132), ISBN 978-3-030-05025-2, https://doi.org/10.1007/978-3-030-05026-9_7.
Smyrnova-Trybulska, E., Ogrodzka-Mazur, E., Szafrańska-Gajdzica, A., Morze, N., Makhachashvili, R., Noskova, T., Pavlova, T., Yakovleva, O., Issa, T. & Issa, Th. (2016). MOOCs – Theoretical and Practical Aspects: Comparison Of Selected Research Results: Poland, Russia, Ukraine, and Australia. In: P. Kommers, T., Issa, Th. Issa, E., McKay, & P., Isaías. Proceedings Of The International Conferences On Internet Technologies & Society 2016 (ITS 2016) Educational Technologies 2016 (IcEduTech 2016) And Sustainability, Technology And Education 2016 (STE 2016) Melbourne, Australia 6–8 December, IADIS 2016 (pp. 107–114). ISBN: 978-989-8533-58-6.
Stracke, C.M., & Trisolini, G. A (2021) Systematic Literature Review on the Quality of MOOCs. Sustainability, 13, 5817. https://doi.org/10.3390/su13115817
Su, P.-Y., Guo, J.-H., & Shao, Q.-G. (2021) Construction of the Quality Evaluation Index System of MOOC Platforms Based on the User Perspective. Sustainability, 13, 11163. https://doi.org/10.3390/su132011163
Svoboda, P., & Mynaříková, L. (2021). MOOC Courses as a Tool for the Development of Digital Competencies of Teachers. In: Nazir, S., Ahram, T.Z., Karwowski, W. (eds) Advances in Human Factors in Training, Education, and Learning Sciences. AHFE 2021. Lecture Notes in Networks and Systems, vol 269. Springer, Cham. https://doi.org/10.1007/978-3-030-80000-0_29
Werbach, K., & Hunter, D. (2015) The Gamification Toolkit: Dynamics, Mechanics, and Components for the Win. Wharton School Press https://wsp.wharton.upenn.edu/book/gamificationtoolkit/
Yu, B., & Zadorozhnyy, A. (2022). Developing students’ linguistic and digital literacy skills through the use of multimedia presentations. ReCALL, 34(1), 95-109. doi:10.1017/S0958344021000136
University of Silesia in Katowice Poland
https://orcid.org/0000-0003-1227-014X
STARTINFORUM International Project Management and Business Consultancy Turkey
https://orcid.org/0000-0002-4802-113X
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The Copyright Owners of the submitted texts grant the Reader the right to use the pdf documents under the provisions of the Creative Commons 4.0 International License: Attribution-Share-Alike (CC BY SA). The user can copy and redistribute the material in any medium or format and remix, transform, and build upon the material for any purpose.
1. License
The University of Silesia Press provides immediate open access to journal’s content under the Creative Commons BY-SA 4.0 license (http://creativecommons.org/licenses/by-sa/4.0/). Authors who publish with this journal retain all copyrights and agree to the terms of the above-mentioned CC BY-SA 4.0 license.
2. Author’s Warranties
The author warrants that the article is original, written by stated author/s, has not been published before, contains no unlawful statements, does not infringe the rights of others, is subject to copyright that is vested exclusively in the author and free of any third party rights, and that any necessary written permissions to quote from other sources have been obtained by the author/s.
If the article contains illustrative material (drawings, photos, graphs, maps), the author declares that the said works are of his authorship, they do not infringe the rights of the third party (including personal rights, i.a. the authorization to reproduce physical likeness) and the author holds exclusive proprietary copyrights. The author publishes the above works as part of the article under the licence "Creative Commons Attribution-ShareAlike 4.0 International".
ATTENTION! When the legal situation of the illustrative material has not been determined and the necessary consent has not been granted by the proprietary copyrights holders, the submitted material will not be accepted for editorial process. At the same time the author takes full responsibility for providing false data (this also regards covering the costs incurred by the University of Silesia Press and financial claims of the third party).
3. User Rights
Under the CC BY-SA 4.0 license, the users are free to share (copy, distribute and transmit the contribution) and adapt (remix, transform, and build upon the material) the article for any purpose, provided they attribute the contribution in the manner specified by the author or licensor.
4. Co-Authorship
If the article was prepared jointly with other authors, the signatory of this form warrants that he/she has been authorized by all co-authors to sign this agreement on their behalf, and agrees to inform his/her co-authors of the terms of this agreement.
I hereby declare that in the event of withdrawal of the text from the publishing process or submitting it to another publisher without agreement from the editorial office, I agree to cover all costs incurred by the University of Silesia in connection with my application.
