Start Submission Become a Reviewer

Reading: Comparison of the Satisfaction of Students who Use Different Learning Management Systems in ...

Download

A- A+
Alt. Display

Research articles

Comparison of the Satisfaction of Students who Use Different Learning Management Systems in Distance Education Processes

Author:

Erdi Okan Yilmaz

Uşak University, TR
X close

Abstract

This study aims to compare the distance education process and system-related satisfaction of students using different learning management systems (Moodle, Google Classroom). Research samples of this mixed explanatory design comprise a total of 7318 students enrolled in Uşak University for the 2020–2021 academic year. The sample categories include 3294 students using Moodle and 3332 utilizing Classroom system besides the 692 students who experienced both environments. Research data were collected by a questionnaire, semi-structured interview form and system log records. At the end of the six-week application process, a distance education process satisfaction questionnaire was applied to the students with the experience in both education software and their opinions were obtained. The results regarding the sample including students with experience on both software tools indicate the students using Classroom have higher satisfaction levels in the distance education process in system infrastructure, online lecture system, assessment, and evaluation dimensions than the ones using Moodle. The qualitative data collected from the interview form hints at problems the students experienced in the synchronous- asynchronous, test and assessment systems of the Moodle system which reduced their satisfaction.
How to Cite: Yilmaz, E. O. (2022). Comparison of the Satisfaction of Students who Use Different Learning Management Systems in Distance Education Processes. Open Praxis, 14(2), 96–109. DOI: http://doi.org/10.55982/openpraxis.14.2.152
110
Views
36
Downloads
1
Twitter
  Published on 09 Dec 2022
 Accepted on 25 May 2022            Submitted on 18 Sep 2021

Introduction

Recent technological developments play a certain role in the education sphere as well as in others. Technological innovations in informatics, in particular, directly affect the distance education sector and contribute to the technological development in distance education. The distance education process, starting from printed materials such as newspapers and letters in the 1800s, has now been replaced by digital and particularly web-based technologies.

During the COVID-19 pandemic emerged in the 2020s, education and training standards transformed and the transition from face-to-face education to distance education accelerated. Distance education activities have gained momentum globally, as public and private educational institutions have initiated distance education processes. The distance education practices conducted in this period were branded “emergency distance education” as all the synchronous/asynchronous technologies were utilized to perform distance education activities. Numerous distance education practices had been implemented at each education level from primary to higher.

T.R. Ministry of National Education (MONE) has intensified their mission on primary and secondary education at Education Informatics Network (EBA). Furthermore, the satellite and terrestrial television broadcasts and web platforms were utilized to maintain education without interruptions. The Higher Education Council of the Turkish Republic (YOK) tasked higher education institutions transition to conduct distance learning as all educational activities in formal universities have been transferred to distance education. Universities with a functional system infrastructure have quickly transited their formal education at the associate, undergraduate and graduate levels to distance education, while institutions with inadequate infrastructure started to install distance education system through vigorous efforts.

The 72.6% of foundations universities and 60.6% of state institutions of higher education in Turkey initiated distance education activities in March 2020 using synchronous and asynchronous learning technologies. Foundation and state universities have started to provide approximately 99% of their theoretical courses and approximately 74% of the applied courses suitable for remote teaching were performed through distance education. Moreover, most of these institutions (foundation 77.4% and state 70.8%) have used their genuine learning management systems (YOK, 2020).

A learning management system (LMS) is a software tool through which management and organization of educational materials besides monitoring and supervision activities regarding the users registered on the system (student, educator, coordinator, etc.) can be performed (Ozan, 2008). These tools, also known as “LMS” in short, are web-based systems that enable online collaborations, and several educational processes from online course distribution to management and organization of educational resources. On these systems, the progress of the students can be monitored by the instructors or system administrators as well as the parents (Mahnegar, 2012).

Today, there are hundreds of LMS tools (Sasikumar, 2012). There are certain LMS with paid versions as there also free tools. Moreover, some of the LMS should be hosted on the institutions’ own servers as some function with a cloud-based server. In YÖK (2020) report, LMS used by universities are analysed. The utilized LMS both by state and foundation universities in Turkey are summarized in Table 1.

Table 1

LMS Usage Rates in Turkey (YÖK, 2020).


MOODLE ALMS BLACKBOARD CANVAS EDMODO SAKAI OTHER

State Univ. %37.8 %33.0 %5.5 %8.7 %4.7 %2.4 %7.9

Foundation Univ. %42.0 %21.0 %19.3 %1.6 %1.6 %6.5 %8.0

Table 1 presents the LMS used by universities in Turkey with usage rates. Based on this table, it is found that the most preferred LMS by both foundation and state universities is Moodle which is an open source and free web-based option. Furthermore, it was observed that many learning management systems are used in some universities. YOK (2020) report also indicates that 25.2% of the learning management systems used in Turkish universities are reported to be cloud-based. It is reported that one of the most preferred cloud based LMS is the Classroom and Edmodo. But, in the analyses made by the Council of Higher Education in Turkey, it was seen that Google Classroom was not included detailed in the LMS usage rates report.

Moodle is a learning management system that was designed by a developer named Martin Dougiamas in 1999 and first used at Curtin University in 2001. In 2002, version 1.0 was released as free and open-source code. Moodle, reaching one million users as of 2010, is used by approximately 190 million users on 145000 websites by 2020 (Moodle, 2020).

The Classroom is an LMS, which was first announced by Google in 2014, does not require any setup or infrastructure because it works on the cloud. It is found that the approximate number of users as of 2020 is 100 million (Vynck & Bergen, 2020). Classroom system, which is paid for commercial use, is offered free of charge to public education institutions. Numerous universities that did not have adequate infrastructure utilized the Classroom to initiate the urgent process of distance education. One of these universities was Uşak University.

Uşak University, founded in 2006, is a state university operating in Turkey. As of 2022, it provides education to approximately 27.000 students in 25 academic units. Distance education services have been provided in some departments of the university since 2010. In addition, some courses that are compulsory to be taught in Turkey are given to formal education students as hybrids through distance education (approximately 9.000 students) at Uşak University.

Classroom, which is the cloud-based LMS, was preferred first in Uşak University during the COVID-19 emergency distance education process. Subsequently, Moodle LMS was also utilized. Uşak University uses learning management systems for both synchronous education and homework uploads and exam applications. As the necessity of taking the student views about the distance education processes from the ones with experience on both software tools to examine which LMS is more effective, and their satisfaction with the distance education processes regarding these two systems.

This study aims to compare the students’ satisfaction levels with their distance education processes. The satisfaction level regarding the learning management systems used by the students in the distance education process is measured for the technical system infrastructure, live lesson (synchronous-synchronous system) infrastructure and testing – assessment (examination) system satisfaction dimensions with a quantitative approach. Furthermore, the views of students who experienced both LMS systems were examined with a qualitative approach. The main contributions of this paper rest on three pillars. First, the determination of students’ satisfaction rates for the two LMS with the crystallization of their strengths and weaknesses through the user experiences. Second, producing a guiding report on LMS selection for institutions and practitioners. Third, comparing process satisfaction in regards to Moodle and Classroom systems.

Learning Management Systems (LMS)

LMS are web-based platforms through which teaching planning, distribution, assessment and management processes can be performed (Teasley, 2009). With these technologies, also known as course management systems or virtual learning environments, operations such as creating and conveying learning contents, monitoring students and evaluating their performance are conducted (Alias & Zainuddin, 2005). According to Teasley (2009), LMS are platforms that facilitate teaching processes, improve students’ learning, save time, facilitate course management, and communication between instructors and students. Moreover, it also has features such as content management, testing and assessment, teaching planning, reporting, providing collaboration and conveying announcements to students (Kulshrestha & Kant, 2009).

The success of an LMS is a reflection of students’ satisfaction for using that particular system (Freeze et al., 2010; Limayem & Cheung, 2011). Students’ adoption of using an LMS and their subsequent usage of it is related to their satisfaction with that system (Yuen et al., 2019). Naveh, et al. (2012) argued that some aspects that affect student satisfaction in LMS are components such as the access to course contents, the timeliness of the contents, ease of navigation and access within LMS, and fast access to the instructor. As Xu and Mahenthiran (2016) stated that distance education student satisfaction on LMS depends on the course organization and content, the ease of assessment and evaluation activities and the user-friendliness of the LMS system. According to Ohliati and Abbas (2019), factors such as the quality of information and infrastructure and the friendliness perceived by students affect their satisfaction with LMS.

Distance Education Process Satisfaction in Learning Management Systems

Pino et al. (2017) defined satisfaction as the state of meeting students’ expectations from the educational institution. Elliott and Healy (2001) and Weerasinghe and Fernando (2017) defined satisfaction as the process of developing an attitude by evaluating students’ learning experiences. Moreover, Elliott and Shin (2002) defined satisfaction as the subjective evaluation of learning outcomes and learning experience.

Student satisfaction after a distance learning experience on LMS is one of the most important factors affecting their learning performance and achievement. Korkmaz et al. (2015) asserted that there is a positive correlation between students’ satisfaction levels in distance education environments and their achievements. Therefore, it can be argued that monitoring and tracking the student satisfaction in a distance learning with LMS is substantial for ensuring an efficient process.

The study conducted by Kuzu and Balaman (2014) obtained the students’, with a learning experience on Moodle, views. It was found that the students were satisfied with Moodle, felt comfortable in the environment and were motivated. Moreover, it was revealed that they experienced communication difficulties with their friends and instructors and had problems in accessing the system due to computer and internet problems hence a reduced student satisfaction. Suner (2018) analysed the satisfaction levels of students, with a learning experience on Moodle LMS, regarding the distance education process and found that they had high satisfaction rates. There are numerous attempts, on the impact of using Classroom and Moodle in distance education, in the literature that indicate and increase in students’ satisfaction levels after using Classroom (Filho et al., 2019; Shaharanee et al., 2016).

This study analyses the satisfaction levels of students with a learning experience on different LMS regarding distance education processes. In this regard, the aim is to reveal students’ satisfaction levels regarding the distance education processes on different LMS, and to assess their current circumstances in detail through evaluating their opinions about these processes.

Methodology

Research Model

Explanatory mixed design (Creswell & Clark, 2007) was used in this study. The mixed model synthesizes quantitative and qualitative approaches. It is the approach in which qualitative data are collected to complement and support quantitative data (Büyüköztürk et al., 2013). The mixed descriptive design was selected to support the obtained quantitative data with qualitative data, ensure the data integrity, and reach more concrete inferences. The distance education process and system satisfaction student groups using different learning management systems were evaluated and compared.

Research Samples

Research samples of this study comprise a total of 7318 from 29813 students enrolled in Uşak University for the 2020–2021 academic year and approved to participate in this study with the voluntary online questionnaire and interviews. The sample categories include 3294 students using Moodle, 3332 operated the Classroom system and including the 692 students who experienced both environments.

Data Collection Tools

The distance education process satisfaction questionnaire developed by the author as the quantitative and a semi-structured interview form as the qualitative data collection tool besides LMS system records (log) was used for the sample comprises students who experienced both learning environments.

The questionnaire is a data collection tool including a number of questions to describe the participants’ attitude, belief, behaviour or life conditions (Thomas, 1998). As the distance education process evaluation questionnaire is a five-point Likert-type (1: strongly disagree, 5: totally agree) questionnaire with six graded questions. First, aims and questions were determined in line with the research problem in the survey development process. As items were prepared to assess the satisfaction levels of the students with a learning experience in different LMS environments through their distance education processes. Three field experts were consulted about the accuracy of the items for the questionnaire’s content validity test. The questionnaire items were finalized in line with expert opinions.

The other qualitative data collection method used in the study is interviews. The interview technique was selected to elaborate on the feelings, thoughts and opinions towards the distance education process and to make sophisticated inferences about the quantitative data. Students’ views were collected online via a written semi-structured interview form. Three main themes were determined by the author as “opinion”, “complaint” and “suggestion” regarding the distance education process and question texts were prepared. Open-ended questions directed to students are as follows:

  • What do you think about the distance education process?
  • Did you have any difficulties and complaints during the distance education process, if yes please specify…
  • Do you have any suggestions about the distance education process?

The qualitative data obtained from the interview forms guided the category formation, evaluation and coding by two field experts. Moreover, the inter-coder reliability was measured with Cohen’s Kappa statistics and the analysis was also reported.

Learning Management Systems

No customization or changes has been made on default learning management systems to affect their perceptions of students. Any theme installation, plug-in/module installation etc. processes were not carried out. The default versions of learning management systems were used. Moodle version 3.9 is used. The cloud-based default version of Classroom software provided by Google was used.

Application Procedure

During the application process, 3332 students experienced only Classroom LMS and 692 students utilized both LMS to finalize their education on Classroom for six weeks. Students had a distance learning experience on Classroom for six weeks, attended their daily synchronous (live) lessons, used asynchronous course materials provided by the instructors, and took online exams via the LMS. A distance education process satisfaction questionnaire was applied at the end of the application to 3332 students who experienced only Classroom LMS.

After the first application 3294 students experienced only Moodle LMS as 692 students in the group with experience in both LMS, continued their education on Moodle LMS for six weeks. A distance education process satisfaction questionnaire was applied at the end of the application to 3294 students who experienced only Moodle LMS and to 692 students with experience in both systems. Moreover, the views of students with experience on both environments in distance education processes were collected online via a semi-structured interview form.

Data Analysis

Descriptive analyses were made primarily of the data obtained from the measurement tools. The percentage, average and frequency tables were formed in this purpose. A t-test analysis was performed to test whether the distance education process assessment questionnaire scores differ for the students using different learning management systems. Survey score ranges are as follows (Tekin, 1991) 4.20–5.00 very satisfied, 3.40–4.19 satisfied, 2.60–3.39 partially satisfied, 1.80–2.59 not satisfied, 1.00–1.79 not satisfied at all. The item score averages of two student groups who experienced only one LMS were analysed by independent samples t-test. For the sample including students with experience in both learning environments dependent samples were analysed with the t-test. SPSS program was used for statistical analysis. It was also examined whether the data were suitable for parametric analysis before the parametric tests. The –2/+2 interval specified by George and Mallery (2010) was considered as an acceptable range for kurtosis and skewness values in the data analysis. Furthermore, effect size values obtained through the tests (eta squared- η2) are also reported. The significance value is designated as p < .05.

Opinion forms obtained from students who experienced both learning environments were analysed with content analysis through qualitative methods. Content analysis is a technique performed to obtain objective results by making valid and systematic inferences from qualitative data (Krippendorff, 2004). Student messages were examined by two field experts and coded into categories under three themes for content analysis. The data obtained are presented in percentage and frequency tables.

Findings

Descriptive Findings Regarding All Units

The distribution of 7318 students participating in the study by the environment they experience is detailed in Table 2.

Table 2

Participant of the Study.


GENDER CLASSROOM GROUP – 1 MOODLE GROUP – 2 CLASSROOM + MOODLE GROUP – 3 F %



F % F % F %

Female 1779 53.40 1922 58.35 474 68.50 4175 57.05

Male 1553 46.60 1372 41.65 218 31.50 3143 42.95

Total 3332 3294 692 7318 %100

4175 (57.05%) of the participants were female and 3143 (42.95%) were male students. 1779 (53.40%) were female and 1553 (46.60%) were male of the 3332 students who experienced only the Classroom environment. As 1922 (58.35%) were female and 1372 (41.65%) were male students of the 3294 students who experienced only the Moodle environment. Moreover, 474 (68.50%) were female and 218 (31.50%) were male of 692 students experiencing both environments. It can be argued that the number of female students is slightly higher in all study groups.

Descriptive Findings Regarding Distance Education Process Satisfaction of Students Using Either Moodle or Classroom

A distance education process questionnaire was applied to a total of 6626 students who experienced either Classroom or Moodle environments. Descriptive findings of the questionnaire are shown in Table 3.

Table 3

Descriptive Findings Regarding Distance Education Process Satisfaction of Students Using Either Moodle or Classroom.


DISTANCE EDUCATION PROCESS CLASSROOM MOODLE


SS N SS N

1. How satisfied are you with the distance education process? 2.89 1.414 3332 2.29 1.289 3294

2. How satisfied are you with the knowledge and competencies of your instructors in distance education? 3.39 1.271 3332 3.30 1.138 3294

3. How satisfied are you with the learning management system software you use? 3.60 1.271 3332 2.43 1.241 3294

4. How satisfied were you with the synchronous (live) lessons? 3.16 1.374 3332 2.79 1.342 3294

5. How satisfied were you with the online exams? 2.85 1.495 3332 2.53 1.365 3294

6. Do you want to continue distance education in the future? 2.52 1.583 3332 2.41 1.538 3294

General Average 3.06 - 3332 2.63 - 3294

It can be asserted that only students who have had a learning experience in Classroom LMS are partially satisfied with their distance education processes (X̄: 3.06). The general averages indicate that the distance education process satisfaction of students using only Moodle (X̄: 2.63) is slightly lower than the students using only Classroom (X̄: 3.06). Whether this difference between the averages is significant or not is elaborated in the next sections.

Comparison of Students’ LMS Satisfaction by Gender

Whether the satisfaction levels of the students who had a learning experience either on Classroom or Moodle LMS differ by gender was analyzed by t-test. The statistical analysis results of the t-tests performed are presented in Table 4.

Table 4

Distance Education Process Satisfaction Levels of Students Using Classroom and Moodle Separately by Gender.


LMS GENDER N SS SD T P

Classroom Female 1779 2.9859 1.0798 3206.42 4.666 .000

Male 1553 3.1668 1.1475

Moodle Female 1922 2.5731 0.9320 2716.24 3.447 .001

Male 1372 2.6958 1.0578

Table 4 shows that the distance education process satisfaction of the students who only experienced Classroom shows a significant difference by gender [t (3206.42) = 4.666, p < .05]. The distance education process satisfaction average (X̄: 3.16) of male students who only experienced the Classroom environment was higher than female students (X̄: 2.98). It is revealed that there is a significant difference with a moderate impact value across gender-based groups of students using only the Classroom environment (η2 = 0.065). The analysis presented in Table 4 hints at a finding that the distance education process satisfaction of the students who only used Moodle shows a significant difference by gender [t (2716.24) = 3.447, p < .05]. The distance education process satisfaction average of male students who only experienced the Moodle environment (X: 2.69) was higher than females (X̄: 2.57). The independent sample t-test indicates that the small effect value of the students using experienced only Moodle environment differs significantly by gender (η2 = 0.004).

The general averages indicate that the distance education process satisfaction of students using only Moodle (X̄: 2.63) is slightly lower than the students using only Classroom (X̄: 3.06). Independent samples t-test was used to examine whether these average differences were significant. The statistical analysis results of the t-test are provided in Table 5.

Table 5

T-Test Analysis Results on the Comparison of Distance Education Process Satisfaction Levels of Students Using Either Moodle or Classroom.


RESEARCH SAMPLES N SS SD T P

Classroom Users 3332 3.06 1.1154 6545.85 17.232 .000

Moodle Users 3294 2.63 0.9880

Findings in Table 5 points to a situation that students using Classroom have significantly higher distance education process satisfaction than the ones using Moodle. [t (6545.85) = 17.232, p < .05]. The independent sample t-test revealed a significant difference with a moderate effect value among the student groups using Classroom and Moodle environments separately (η2 = 0.042).

Student Group Comparisons Regarding Samples Using Either Classroom or Moodle Based on Survey Items on Distance Education Process

The distance education process evaluation questionnaire items were also statistically analyzed and compared separately for each group (using either Classroom or Moodle environments). Item-based independent samples t-test analysis results are summarized collectively in Table 6.

Table 6

T-Test Analysis Results on the Comparison of the Distance Education Process Questionnaire of Students Using Either Classroom or Moodle.


ITEMS RESEARCH SAMPLES N SS SD T P η2

1. How satisfied are you with the distance education process? Classroom 3332 2.89 1.414 6581.302 18.271 .000 0.048

Moodle 3294 2.29 1.289

2. How satisfied are you with the knowledge and competencies of your instructors in distance education? Classroom 3332 3.39 1.271 6561.003 3.218 .001 0.001

Moodle 3294 3.30 1.138

3. How satisfied are you with the learning management system software you use? Classroom 3332 3.60 1.271 6624 37.872 .000 0.177

Moodle 3294 2.43 1.241

4. How satisfied were you with the synchronous (live) lessons? Classroom 3332 3.16 1.374 6624 11.068 .000 0.018

Moodle 3294 2.79 1.342

5. How satisfied were you with the online exams? Classroom 3332 2.85 1.495 6582.524 9.117 .000 0.012

Moodle 3294 2.53 1.365

6. Do you want to continue distance education in the future? Classroom 3332 2.52 1.583 6621.923 2.986 .003 0.001

Moodle 3294 2.41 1.538

The analysis results in Table 6 reveals that there are significant differences in all questionnaire items in favor of Classroom experience.

Comparison of Distance Education Process Satisfaction of Students Using Both Moodle and Classroom (Paired Samples T-Test)

This section summarizes the findings regarding 692 students who experienced both LMS. The change in distance education process satisfaction and gender-oriented satisfaction of the student group who experienced both environments was examined by a paired t-test (Table 7).

Table 7

Paired T-Test Comparison for Students Experienced Both Classroom and Moodle Environments.


ITEMS ENVIRONMENT N SS SD T P η2

1. How satisfied are you with the distance education process? Classroom 692 2.85 1.36 691 9.435 0.000 0.11

Moodle 692 2.32 1.31

2. How satisfied are you with the knowledge and competencies of your instructors in distance education? Classroom 692 3.34 1.24 691 2.518 0.012 0.01

Moodle 692 3.20 1.19

3. How satisfied are you with the learning management system software you use? Classroom 692 3.61 1.26 691 22.895 0.000 0.43

Moodle 692 2.25 1.23

4. How satisfied were you with the synchronous (live) lessons? Classroom 692 3.12 1.37 691 5.363 0.000 0.04

Moodle 692 2.81 1.33

5. How satisfied were you with the online exams? Classroom 692 2.75 1.49 691 4.511 0.000 0.02

Moodle 692 2.47 1.36

6. Do you want to continue distance education in the future? Classroom 692 2.50 1.59 691 0.194 0.846

Moodle 692 2.48 1.51

General Averages Classroom 692 3.02 1.09 691 10.512 0.000 0.13

Moodle 692 2.58 1.03

Table 7 reveals the findings of comparison regarding the distance education process satisfaction of the students who experienced the same environments on item-based and holistic contexts. Regarding students’ desire to continue their distance education there is no significant difference between Classroom and Moodle. In other words, students’ willingness to continue their distance education process does not change according to the LMS they use within the scope of this study. Moreover, it can be argued that there is a significant difference in favor of students using Classroom LMS for all remaining questionnaire items.

Comparison of Distance Education Process Satisfaction of Students Experienced the Same Environment by Gender (Independent Samples T-Test)

The statistical analysis results of the t-test are provided in Table 8.

Table 8

Comparison of the Distance Education Process Satisfaction of Students Experienced the Same Environment by Gender (Independent Samples T-Test).


LMS GENDER N SS SD T P η2

Classroom Female 474 2.92 1.04 382.466 3.496 .001 0.03

Male 218 3.24 1.17

Moodle Female 474 2.47 0.94 353.842 4.084 .000 0.05

Male 218 2.84 1.16

Table 8 illustrates the analysis report on the comparison of the general distance education process satisfaction levels of male and female students who experienced both Classroom and Moodle environments. Accordingly, the satisfaction average scores of male students who experienced Classroom were significantly higher [t (382.466) = 3.496, p < .05]. Similarly, satisfaction average scores of male students who experienced Moodle LMS were significantly higher than females [t (353.842) = 4.084, p < .05]. As it can be epitomized the satisfaction level of the distance education process of male students who experienced both environments are significantly higher.

Student Views-Oriented Findings

692 students who experienced both LMS were asked to submit their views on the distance education process, the difficulties and problems, and their suggestions for the online process using a one-time semi-structured interview form after their learning experiences. A total of 420 messages under three themes in the opinion forms transmitted by 279 students were examined with content analysis. As 119 of the messages conveyed are opinion messages, 162 of them are complaints and 139 are suggestions. The messages transmitted under 3 different themes were transferred to the Excel program and relevant categories were created. There were 7 categories under the opinion, 6 under the complaint and 7 under the suggestion themes. The theme categories (Holsti, 1969) were specified as distinctive and differentiating, holistic, objective, comprehensive, suitable for the research purpose and meaningful. A total of 420 messages under three main themes were coded with a category table guide by two field experts. All messages and categories were transferred to an Excel table, as the experts were informed about the research in the digital environment. The consistency between experts’ evaluations was examined using Cohen’s Kappa statistics. The Kappa scores obtained were graded with score ranges specified by Landis and Koch (1977) (<0.00 weak, 0.00–0.20 very low, 0.21–0.40 low, 0.41–0.60 good, 0.61–0.80 very good and 0.81–1.00 excellent harmony).

Inter-coder reliability was examined separately for messages sent under three main themes and for all messages. The examination of 119 messages sent under the ←opinion⟶ theme (7 categories) indicate conformity between the two coders κ = .824 (p < .05) excellent harmony level as the ratio for ←complaint⟶ theme (6 categories) regarding 169 messages was κ = .852 (p < .05) excellent harmony level and for ←suggestion⟶theme (7 categories) of 139 messages κ = .850 (p < .05) an excellent conformity. Also, intercoder reliability for all messages transmitted indicates excellent harmony with a ratio κ = .845 (p < .05).

The distribution of student views by themes and categories is demonstrated in Table 9.

Table 9

Student Views and Categorical Distributions.


MAIN THEME NUMBER OF VIEWS CATEGORIES F %

View 119 Distance education is inefficient, I am not satisfied with distance education 63 44.68

I had difficulty in the distance education process because I did not have the necessary devices and internet access 23 16.31

I am very satisfied with the distance education 23 16.31

I cannot learn and understand the subjects via distance education 16 11.34

I don’t have any opinions 11 7.80

I think live lessons take too long 4 2.83

I think Moodle is better than Classroom 1 0.73

Complaints 162 Troubles in Moodle measurement and evaluation system 89 40.45

I didn’t like Moodle I had troubles operating 57 25.90

Trouble in the synchronous system 32 14.54

Lack of access to previous video recordings 18 8.18

Asynchronous system log in problem 17 7.72

No picture appeared in visual questions 7 3.21

Suggestion 139 Face-to-face education should replace the distance education 36 21.56

Homework should be assigned instead of online exam 34 20.35

Moodle LMS infrastructure should be improved 34 20.35

Classroom LMS should be used instead of Moodle 33 19.76

Distance education should continue 16 9.58

Interaction should be increased in live lessons 7 4.20

Live lessons should be shorter 7 4.20

Table 9 reveals that students were not satisfied with their distance education experience (f:63, 44.68%), and that they had difficulties in the process due to device and internet access problems (f:23, 16.31%). However, 16.31% (f:23) of the students stated that they were satisfied with the distance education process. The students also stated that they could not learn and understand the subjects in distance education (f: 16, 11.34%) and live lessons took too long (f:4, 2.83%).

The categorical distribution of the messages sent for complaint purposes conveyed indicates that the most frequently mentioned subject was Moodle LMS’s online exam system (f:89, 44.45%). The second most mentioned complaint subject was regarding the problems with the LMS operations on the Moodle system (f:57, 25.90%). Students also expressed their complaints regarding the difficulties in live synchronous lessons, previous course video recordings, access problems in asynchronous course areas on the Moodle system. The students did not express a complaint message regarding the Classroom system.

Student suggestions mostly centralized the return to face-to-face education instead of distance education (f:36, 21.56%). Students also suggested homework assignments and software updates in the Moodle LMS instead of online tests (f:34, 20.35%). They also stated that Classroom LMS should be preferred over Moodle (f:33, 19.76%), there should be more interaction and lesson duration should be shortened in live synchronous lessons. The portion of students who said that distance education should continue was 9.58% (f:16).

Conclusion and Discussions

Distance education process and system satisfaction of student groups using different learning management systems were compared with a mixed model. This section presents a thorough discussion of the research findings within the framework of the relevant literature. Furthermore, recommendations for researchers, distance education practitioners and institutions are included.

The quantitative findings obtained from the distance education process questionnaire indicates that students using Classroom only had significantly higher overall distance education process satisfaction than the ones used Moodle only. However, it can be argued that students who utilized both LMS were significantly more satisfied with the distance education process they experienced in Classroom. It was revealed that the students were significantly more satisfied with the distance education processes, knowledge and competencies of instructors, the technical infrastructure of the system, the live classroom system (synchronous), the online testing system on Classroom compared to ones using Moodle. It can be asserted that results of this research are in parallel with several studies that revealed students’ satisfaction levels are positively affected by the use of Classroom (Filho, et al., 2019; Shaharanee et al., 2016).

Çalışkan et al. (2017) stated that student satisfaction in the distance education process is generally high. Accordingly, Goyal and Purohit (2011) reached results indicating that the use of Moodle LMS positively affected students’ expectations and satisfaction in their study. Umek et al. (2015) conducted a study on student satisfaction levels for Moodle-based e-learning platforms and revealed that students’ satisfaction levels increased. Similarly, Suner (2018) found that students using the Moodle system have high satisfaction levels regarding distance education process. However, this study produced results contrary to some studies in the literature implying the distance education process satisfaction of students using Moodle was significantly low.

Damnjanovic et al. (2015) discovered that the software system does not affect student satisfaction. Barman and Karthikeyan (2019) stated that the Classroom environment does not have a complex structure making it user friendly compared to other LMS as Moodle has a more complex structure. It can be argued that the students are more satisfied with the distance education processes on the Classroom system may be related to the user friendliness. However, student views hint that there are substantial amounts of complaints about the Moodle software system as numerous suggestions for improvement.

The student views points to an argument that the majority of the students (44% of total) think that distance education is inefficient and they are generally not satisfied with the processes. The student complaints include difficulties in testing systems (40% of total), as some express their dislike towards Moodle LMS as they experienced difficulties (25% of the students who submitted suggestions). The qualitative data obtained from student opinions indicate problems such as sound interruption, image freeze, and disconnection in the live lesson (synchronous virtual classroom) on Moodle system (14.54% of the students who submitted suggestions). However, students highlighted difficulties in accessing previous lecture video recordings on Moodle and having problems logging into the system. These difficulties experienced by the students are considered as factors that negatively affected their satisfaction with the distance education processes on the Moodle system. The study conducted by Kuzu and Balaman (2014) obtained the students’, with a learning experience on Moodle, views. As it was found that the students were satisfied with Moodle, they felt comfortable and motivated in the environment, but this study produced contradictory results implying that students were generally not satisfied with Moodle LMS.

The suggestions submitted by the students regarding the distance education processes include face-to-face education should replace distance education, homework should be assigned instead of online tests, and Moodle software system should be improved. Another noteworthy suggestion made by the students was to use Classroom instead of Moodle in distance education processes (19.76% of the students who submitted suggestions). It can be argued that the students are not satisfied with the Moodle system and they want Classroom to be preferred instead.

Other suggestions include increased teacher-student interaction in synchronous live lessons and shortened time of live lessons. The log records of distance education systems show that the lessons (approximately 30 thousand live lecture videos each) were 46 minutes 31 seconds on average. Several studies in the literature indicate that the average length of live lessons should be 20 minutes (Osipov et al., 2015; Ferriman, 2015). It can be asserted that the students were negatively affected by long live virtual lessons.

The satisfaction levels by genders reveal that male students who were in the group who used either Classroom or Moodle and who experienced both environments had significantly higher distance education process satisfaction. Yalman (2013) and Horvat et al. (2015) stated that student satisfaction on Moodle learning management system there is no difference between male and female students’ satisfaction levels but male students’ satisfaction was found to be higher in this study.

Recommendations for Decision-Makers

The comparison between Classroom and Moodle, reveals the important differences such as the being cloud-based or not, physical hardware (server) requirement, installation and software preparation for the users. The Moodle system is institution-based and all installation, adjustments, scaling, hardware and network software activities must be performed by the authorized administrators. Users and decision-makers in education are recommended to consider the above-mentioned positive and negative conditions, especially when choosing LMS for large student groups.

Competing Interests

The author has no competing interests to declare.

References

  1. Alias, N. A., & Zainuddin, A. M. (2005). Innovation for better teaching and learning: adopting the learning management system. Malaysian Online Journal of Instructional Technology, 2(2), 27–40. 

  2. Barman, B., & Karthikeyan, J. (2019). Facilitating elt through Moodle and Google Classroom. Restaurant Business, 118(10), 506–518. DOI: https://doi.org/10.26643/rb.v118i10.9570 

  3. Büyüköztürk, Ş., Çakmak, E. K., Akgün, Ö. E., Karadeniz, Ş., & Demirel, F. (2013). Bilimsel araştırma yöntemleri (14th ed.). Pegem Akademi. 

  4. Çalışkan, S., Suzek, S., & Özcan, D. (2017). Determining student satisfaction in distance education courses. Procedia Computer Science, 120(1), 529–538. DOI: https://doi.org/10.1016/j.procs.2017.11.275 

  5. Creswell, J., & Plano Clark, V. L. (2007). Understanding mixed methods research. J. Creswell, & V. L. Plano Clark in, Designing and conducting mixed methods research (pp. 1–19). Sage Publishing. 

  6. Damnjanovic, V., Jednak, S., & Mijatovic, I. (2015). Factors affecting the effectiveness and use of Moodle: students’ perception. Interactive Learning Environments, 23(4), 496–514. DOI: https://doi.org/10.1080/10494820.2013.789062 

  7. Elliott, K. M., & Healy, M. A. (2001). Key factors influencing student satisfaction related to recruitment and retention. Journal of Marketing for Higher Education, 10(4), 1–11. DOI: https://doi.org/10.1300/J050v10n04_01 

  8. Elliott, K. M., & Shin, D. (2002). Student Satisfaction: An alternative approach to assessing this important concept. Journal of Higher Education Policy and Management, 24(2), 197–209. DOI: https://doi.org/10.1080/1360080022000013518 

  9. Ferriman, J. (2015, February 15). The ideal elearning course duration. Learn Dash. https://www.learndash.com/the-ideal-elearning-course-duration/ 

  10. Filho, A. S., Fantini, W. D., Ciriaco, M. A., Santos, J. D., Moreira, F., & Gomes, A. S. (2019). Health Student Using Google Classroom: Satisfaction Analysis. Learning Technology for Education Challenges. LTEC 2019. Cham: Springer. DOI: https://doi.org/10.1007/978-3-030-20798-4_6 

  11. Freeze, R. D., Alshare, K. A., Lane, P. L., & Wen, H. J. (2010). IS success model in e-learning context based on students’ perceptions. Journal of Information Systems Education, 2, 173–184. 

  12. George, D., & Mallery, M. (2010). SPSS for Windows step by step: A simple guide and reference (17 b.). Pearson. 

  13. Goyal, E., & Purohit, S. (2011). Using Moodle to enhance student satisfaction from ICT. International Conference on Technology for Education, 191–198. DOI: https://doi.org/10.1109/T4E.2011.37 

  14. Holsti, O. R. (1969). Content analysis for the social sciences and humanities. Addison-Wesley Publishing Company. 

  15. Horvat, A., Dobrota, M., Krsmanovic, M., & Cudanov, M. (2015). Student perception of Moodle learning management system: a satisfaction and significance analysis. Interactive Learning Environments, 23(4), 515–527. DOI: https://doi.org/10.1080/10494820.2013.788033 

  16. Korkmaz, Ö., Çakır, R., & Tan, S. S. (2015). Öğrencilerin e-öğrenmeye hazır bulunuşluk ve memnuniyet düzeylerinin akademik başarıya etkisi. Ahi Evran Üniversitesi Kırşehir Eğitim Fakültesi Dergisi, 16(3), 219–241. 

  17. Krippendorff, K. (2004). Content analysis: An introduction to its methodology. Sage Publications. 

  18. Kulshrestha, T., & Kant, A. R. (2009). Benefits of learning management system (lms) in Indian education. International Journal of Computer Science & Engineering Technology (IJCSET), 4(8), 1153–1164. 

  19. Kuzu, S., & Balaman, F. (2014). Moodle kullanılarak gerçekleştirilen web destekli eğitim hakkındaki öğrenci görüşleri. Eğitim ve Öğretim Araştırmaları Dergisi, 3(2), 234–243. 

  20. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. International Biometric Society, 33(1), 159–174. DOI: https://doi.org/10.2307/2529310 

  21. Limayem, M., & Cheung, C. M. (2011). Predicting the continued use of Internet-based learning technologies: the role of habit. Behaviour & Information Technology, 30(1), 91–99. DOI: https://doi.org/10.1080/0144929X.2010.490956 

  22. Mahnegar, F. (2012). Learning management system. International Journal of Business and Social Science, 3(12), 144–150. 

  23. Moodle. (2020). About Moodle. Moodle https://docs.moodle.org/310/en/History 

  24. Naveh, G., Tubin, D., & Pliskin, N. (2012). Student lms use and satisfaction in academic institutions: the organizational perspective. Internet and Higher Education, 13(3), 127–133. DOI: https://doi.org/10.1016/j.iheduc.2010.02.004 

  25. Ohliati, J., & Abbas, B. S. (2019). Measuring students satisfaction in using learning management system. International Journal of Emerging Technologies in Learning, 4(4), 180–189. DOI: https://doi.org/10.3991/ijet.v14i04.9427 

  26. Osipov, I. V., Nikulchev, E., Volinsky, A. A., & Prasikova, A. Y. (2015). Study of gamification effectiveness in online elearning systems. International Journal of Advanced Computer Science and Applications, 6(2), 71–77. DOI: https://doi.org/10.14569/IJACSA.2015.060211 

  27. Ozan, Ö. (2008). Öğrenme yönetim sistemlerinin (learning management systems-lms) değerlendirilmesi [Conference presentation]. XIII. Türkiye’de İnternet Konferansı inet-tr’08, (pp. 61–65). Ankara, Türkiye. 

  28. Pino, G. F., Herrera, M. M., Roca, S. C., & Antúnez, A. M. (2017). Grado de satisfacción de los estudiantes de 6° y 7° semestre de comercio y negocios internacionales en la Universidad Simón Bolívar de Barranquilla respecto a su plan de estudio. Liderazgo Estratégico, 7(1), 44–58. https://revistas.unisimon.edu.co/index.php/liderazgo/article/view/3259 

  29. Sasikumar, M. (2012). Role of artificial intelligence in enhancing the effectiveness of education. CSI Communications, 12–17. 

  30. Shaharanee, I. N., Jamil, J. M., & Rodzi, J. M. (2016). The application of Google Classroom as a tool for teaching and learning. Journal of Telecommunication, Electronic and Computer Engineering, 8(10), 5–8. DOI: https://doi.org/10.1063/1.4960909 

  31. Suner, A. (2018). Moodle ile internet destekli biyoistatistik dersinin değerlendirilmesi. Ege Tıp Dergisi, 57(4), 201–211. DOI: https://doi.org/10.19161/etd.416659 

  32. Teasley, S. L. (2009). Saving time or innovating practice: Investigating perceptions and uses of learning management systems. Computers & Education, 53(3), 686–694. DOI: https://doi.org/10.1016/j.compedu.2009.04.008 

  33. Tekin, H. (1991). Eğitimde ölçme ve değerlendirme (6th ed.). Ankara: Yargı Yayınları. 

  34. Thomas, R. M. (1998). Conducting educational research: A comparative view. Bergin & Garvey Publishing. 

  35. Umek, L., Aristovnik, A., Tomaževič, N., & Keržič, D. (2015). Analysis of selected aspects of students’ performance and satisfaction in a moodle-based e-learning system environment. Eurasia Journal of Mathematics, Science & Technology Education, 11(5), 1495–1505. DOI: https://doi.org/10.12973/eurasia.2015.1408a 

  36. Vynck, G. D., & Bergen, M. (2020). Google Classroom users doubled as quarantines spread. Bloomberg Quint. https://www.bloombergquint.com/business/google-widens-lead-in-education-market-as-students-rush-online 

  37. Weerasinghe, I. S., Lalitha, R., & Fernando, S. (2017). Students’ satisfaction in higher education. American Journal of Educational Research, 5(5), 533–539. DOI: https://doi.org/10.12691/education-5-5-9 

  38. Xu, H., & Mahenthiran, S. (2016). Factors that influence online learning assessment and satisfaction: using moodle as a learning management system. International Business Research, 9(2), 1–18. DOI: https://doi.org/10.5539/ibr.v9n2p1 

  39. Yalman, M. (2013). Eğitim fakültesi öğrencilerinin bilgisayar destekli uzaktan eğitim sistemi (Moodle) memnuniyet düzeyleri. Turkish Studies, 8(8), 1395–1406. DOI: https://doi.org/10.7827/TurkishStudies.5357 

  40. YÖK, Y. K. (2020). Üniversitelerimizde küresel salgın sürecinde uzaktan öğretim uygulamaları anket sonuçları. YÖK. https://covid19.yok.gov.tr/Documents/AnaSayfa/uzaktan-ogretim-anketi.pdf 

  41. Yuen, A. H., Cheng, M., & Chan, F. H. (2019). Student satisfaction with learning management systems: A growth model of belief and use. British Journal of Educational Technology, 50(5), 2520–2535. DOI: https://doi.org/10.1111/bjet.12830 

comments powered by Disqus