Start Submission Become a Reviewer

Reading: Surfing on Three Waves of MOOCs: An Examination and Snapshot of Research in Massive Open Onl...

Download

A- A+
Alt. Display

Research articles

Surfing on Three Waves of MOOCs: An Examination and Snapshot of Research in Massive Open Online Courses

Author:

Aras Bozkurt

Anadolu University, TR; UNISA, ZA
X close

Abstract

The purpose of this research is to examine the research that has been done on MOOCs by applying data mining and analytic approaches and to depict the current state of MOOC research. The text mining revealed four broad themes: (I) MOOCs as a mainstreaming learning model in HE, (II) motivation and engagement issues in MOOCs, (III) assessment issues in MOOCs, and (IV) MOOCs for social learning. The social network analysis indicated that MOOCs are a significant aspect of online education and that learning analytics are considered a solution to current MOOC handicaps. Both pivotal contribution analysis and timeline analysis demonstrated that MOOC research has a heavy focus on motivation and engagement, high drop out and low retention rates, and instructional quality. Overall, the research concludes that the first wave of MOOC research largely concentrated on the qualitative characteristics of the phenomenon, while the second and third waves of MOOC research concentrated on the quantitative characteristics.

How to Cite: Bozkurt, A. (2021). Surfing on Three Waves of MOOCs: An Examination and Snapshot of Research in Massive Open Online Courses. Open Praxis, 13(3), 296–311. DOI: http://doi.org/10.5944/openpraxis.13.3.132
382
Views
109
Downloads
7
Twitter
  Published on 31 Dec 2021
 Accepted on 17 Aug 2021            Submitted on 19 Oct 2020

Introduction

The phenomenon of massive open online courses (MOOCs) has attracted much attention in the fields of higher education (HE), lifelong learning, distance education (DE), and open and distance learning (ODL). At its core, MOOCs promise an open and flexible way of learning, regardless of where learners live or when learners participate. Attracting millions of participants, MOOCs differ from formal online courses in that they address a larger audience and provide students the flexibility to drop in and drop out. As a result of continuing efforts and earlier practices in the open education field, MOOCs first appeared in 2008 and since then, millions of participants have benefited from them. After the inception of MOOCs, three waves of MOOCs followed. The first wave was the connectivist MOOCs, followed by the second wave of extension MOOCs and the third wave, which was a hybrid of the first and second MOOCs, insofar as it benefited from the design principles of the first and the second waves.

Despite the increasing demand and interest in MOOCs, many questions remain unanswered regarding what MOOCs really are and where they are headed in terms of their impact on educational institutions and educational opportunities. To address this question, this study aims to examine the research on MOOCs research by conducting a systematic review of the empirical MOOC publications from 2016 to 2018.

Literature Review

There have been many efforts put forth by the academic world to understand the impacts of the MOOCs. Some of the studies have examined MOOCs from the perspective of social media and networked discourses (Bulfin et al., 2014; Chen, 2014; Deimann, 2015; Kovanović et al., 2015; Shen & Kuo, 2015). These studies revealed that while there was a generally positive response towards MOOCs (Bulfin et al., 2014; Shen & Kuo, 2015), the use of social media presented challenges as well as opportunities (Chen, 2014), triggering discussions that went beyond simply the pedagogical and economic aspects of MOOCs (Deimann, 2015).

Other studies have looked closely at the MOOC research in scholarly publications (Bozkurt et al., 2016; Costello et al., 2018; Ebben & Murphy, 2014; Gašević et al., 2014; Joksimović, et al., 2018; Kennedy, 2014; Lee et al., 2019; Liyanagunawardena et al., 2013; Raffaghelli et al., 2015; Sa’don et al., 2014; Sangrà et al., 2015; Veletsianos & Shepherdson, 2015, 2016; Zawacki-Richter et al., 2018; Zhu et al., 2018; Wong et al., 2019). These studies concluded that the interest MOOCs has attracted in education and computer science disciplines (Veletsianos & Shepherdson, 2015) has helped to build knowledge on a wide array of topics (Liyanagunawardena et al., 2013) by providing social learning opportunities (Gašević et al., 2014) at the HE level (Sangrà et al., 2015).

The interest in MOOCs is still alive, as attested to by the increase in MOOC research. In this context, this study aims to provide an overview of MOOC research from 2016 to 2018 by identifying trends and patterns through a systematic review of the related literature through data mining, analytics and visualization techniques. The following research questions were developed to guide this aim.

What are the scholarly trends and patterns between 2016 and 2018 in:

  • MOOC research?
  • Published articles’ keywords?
  • Intellectual bibliometric network?

Research Method and Design

This study applied a systematic review method (Gough et al., 2012), using data mining and analysis approaches, such as text mining (Hearst, 2003) and social network analysis (SNA) (Hansen et al., 2010). The main purpose of using different analytic approaches (e.g., text mining for lexical relationships, social network analysis of the keywords and references) is to triangulate the research data (Thurmond, 2001) and to examine MOOCs from multiple perspectives in order to gain a broader understanding. By triangulating the research findings, the reliability and validity issues improved, and researchers were enabled to report a more complete meta picture of the state of the art in MOOC research. The findings in each research strand were visualized through data mining and analytic approaches (Fayyad, et al., 2002) to discover research patterns and better interpret the sheer volume of data.

Sampling

The selected articles were identified by using the following search strings: “MOOC* OR Massive Open Online Course*”. As a follow-up study to the study by Bozkurt et al. (2017), who examined the MOOC research from 2018 to 2015, the present study examined publications from 2016 to 2018. The inclusion criteria for sampling were that the study be indexed in Scopus, published in peer-reviewed journals between 2016 and 2018, written in English, be accessible online, and have the searched keywords in the title. The rationale of including articles that have search strings only in their title is to build a more focused, concentrated research corpus and, therefore, reach more accurate and representative findings that reflect the trends and patterns in MOOC research. Accordingly, after following the PRISMA Framework (Page et al., 2021), a total of 633 articles that met the inclusion criteria were included in the research corpus (See Figure 1).

Identification of final research corpus through Prisma Diagram
Figure 1 

Identification of final research corpus through Prisma Diagram.

Data analysis

For the text mining portion of the research, sentiment analysis algorithms were used to make meaningful connections. Bayesian statistics, which record the “occurrence of a word and connects it to the occurrence of a series of other words”, were included as part of this analysis (Ward et al., 2014, p. 119). For text mining, titles and abstracts of the articles in the final research corpus merged, and after omitting nonrelevant words (e.g., pronouns, prepositions), the data was analyzed and then visualized. The textual data was then quantified by considering the frequency of the words, co-occurrences, and their meaning in the context before generating concepts and themes. For the text mining, to identify different thematic patterns, the paths emerged through lexical analysis reported in fractions.

In the social network analysis of the keywords, each keyword is considered as a node, and their co-occurrences are considered as ties. The analysis included betweenness centrality (BC), which refers to a node’s bridging score and its centrality in the network. Degree centrality (DC), which refers to the total number of unique edges, and PageRank (PR), which refers to the influence score, were also calculated to better interpret the significance of the nodes. The node sizes were adjusted according to their betweenness centrality. Moreover, the edge weights were calculated to indicate the strength of the connections, and the thickness of ties were adjusted according to their edge weight values. In the social network analysis of the bibliometric data, the density value, which ranges from 0 to 1 and explains the degree to which the nodes are interconnected in the network, the modularity value, which is used to measure the strength of division of a network into modules, and silhouette, which ranges from -1 to 1 and measures the quality of a clustering configuration, were calculated.

Strengths and limitations

The strength of this study lies in the methodological approaches used. First, the findings were visualized in such a way to easily make sense of the large volume of data compiled. Second, since machine-based approaches were used, the analysis of the data was free from researcher bias. Third, the use of different analytical approaches facilitated data triangulation, which allowed different layers of the research data to be explored. However, in addition to the strengths of the study, there were some limitations that should be taken into account. First, though the research provides a panoramic outlook of the MOOC research, the data was limited to articles indexed in the Scopus database. In other words, despite the comprehensiveness of the data, the research presents only a partial view. Second, the research included only peer-reviewed articles in the research corpus. However, the researcher acknowledges that other scholarly sources, such as books, book chapters, reports, conference proceedings, wikis and blogs may provide additional valuable insights into MOOC research.

Findings and Discussions

Text Mining and qualitative content analysis

This section presents the findings derived from the lexical analysis involving a text-mining approach of the titles and abstracts from the sampled 633 publications (Figure 2).

Themes and concepts derived from the MOOC research
Figure 2 

Themes and concepts derived from the MOOC research.

Lexical analysis meaningfully dissects words to identify them as concepts, a process referred to as parsing or tokenizing, to present them as meaningful syntaxes for the generation of a thematic concept map. The research presented meaningful paths that aligned with the purposes of the research. Accordingly, the following four themes were identified: (I) MOOCs as a mainstreaming learning model in HE, (II) motivation and engagement issues in MOOCs, (III) assessment issues in MOOCs, and (IV) MOOCs for social learning.

MOOCs as a mainstreaming learning model in HE (see path: Universities, MOOCs, learning, higher, and education): There is an increasing trend to utilize MOOCs by integrating them with emerging educational delivery models, such as blended learning (Bralić & Divjak, 2018) or flipped classroom (Cao, 2018). MOOCs have benefitted from the partnerships they have established with MOOC providers, HE institutions and corporations, where the purpose in the case of the latter has been to equip employees with specific skills suitable to the competitive business sector (Calonge & Shah, 2016). The shrinking half-life of knowledge and the constantly changing dynamics of the industry require that employees as well as new graduates have the opportunity to engage in continuous learning so that they have the necessary qualifications (Egloffstein & Ifenthaler, 2017), and MOOCs, by offering micro credentials or nano degrees, help the business sector to meet these needs (Lemoine & Richardson, 2015). The fact that MOOCs are capable of meeting this need strengthens their acceptance as a mainstreaming learning model (Costa et al., 2018). However, despite the transformational potential of MOOCs’ sustainable business model (Belleflamme & Jacqmin, 2016), there are still concerns and skepticism about how to sustain this business model (Egloffstein, & Ifenthaler, 2017). It is also important to note that while there is a positive outlook on the mainstreaming of MOOCs in the U.S (Li et al., 2018), the mainstreaming of MOOCs in other regions of the world (e.g., Africa) does not look as promising (Rambe & Moeti, 2017).

The findings indicate that MOOCs will continue their growth trend in HE on account of not only their potential in this area but also their integration with new educational models (e.g., blended learning and flipped classroom), both of which shall help them to forge their existence and sustainability in HE. However, it is also noteworthy that the perception of MOOCs is not the same all around the world, and that before fully exploring MOOCs’ potential, they should be looked at in terms of social, cultural, and economic aspects, and the needs of the learners should be examined. The first wave of MOOCs, termed connectivist MOOCs (cMOOCs), which ran from 2008–2011, seems to have played the role of disruptive innovation, while the second wave, termed extension MOOCs (xMOOCs), which ran from 2012–2015, transformed the first wave into sustainable innovation. The third and current wave of MOOCs has been a hybrid of the first two waves, having connectivist and extension properties, and has served to further evolve MOOC pedagogies into a supplementary mode to already existing delivery modes, such as blended learning or flipped classroom.

Motivation and engagement issues in MOOCs (see path: motivation, MOOCs, and learning; see also: MOOCs, learning, engagement, learners, exploring, and intention): Low competition and high drop-out rates are major concerns in MOOC research (Sinclair & Kalvala, 2016) and these concerns make two variables a significant issue in MOOC research: motivation and engagement. Research suggests that innovative tools and approaches (e.g., gamification) that take into account learners’ digital skills and competencies can be used to sustain motivation and engagement. However, related studies also show, in reference to the massive number of learners taking MOOCs, that learners come from diverse backgrounds and have different motivations, goals and experiences, which underlines the importance of developing self-regulated learning strategies (Alario-Hoyos et al., 2017; Littlejohn et al., 2016). While studies have recommended that MOOCs be designed with an emphasis on motivational and engagement, other improvements, like using learning analytics to tailor learning design and learning experience, should also be considered (De Barba et al., 2016; Pursel et al., 2016).

The issues of motivation and engagement have a long history in all the educational modes. The massiveness characteristic of MOOCs has proven to be a challenge for both MOOC platforms and MOOC instructors. While MOOCs have been strongly criticized for their high drop-out rates and low completion rates, as indicated by Lambert (2020), they have been praised for contributing to student equity and social inclusion, two factors responsible for its widening participation. In this regard, beyond the quantified outcomes, it can be argued that MOOCs promote universal values in education and, therefore, their contribution at any educational level is significant. It is also promising that there is an increasing trend in using learning analytics to deal with the massive number of learners. One area of concern, however, is that the perceived learning in MOOCs has been neglected as a focus of research. This is important to note because learning goes beyond quantified learning objectives.

Assessment issues in MOOCs (see path: educational, MOOCs, assessment, design, and students): Among the many challenges that MOOCs have in dealing with a massive number of students is the issue of assessments, particularly in terms of emerging assessment concepts: Formative assessment (e.g., automated assessment, peer assessment, etc.) and summative assessment (e.g., badges, certificates, micro credentials, etc.) (Xiong, & Suen, 2018). Automated assessments and peer assessments have been the topics most examined in the research corpus.

Formative assessment approaches are a challenge for leisure learners who are lured to MOOCs by their openness dimension. Likewise, summative assessment techniques are a challenge insofar as they present concerns about assessment security issues (Xiong, & Suen, 2018). The massive number of students involved in MOOCs require that MOOC providers develop alternative assessments, like peer assessments, to traditional assessment methods. However, “the validity of [peer assessment] process is still under discussion, suffers from a lack of credibility and has many weaknesses” (Haddadi et al., 2018, p. 1873). It is reported that peer assessment is not particularly reliable nor valid (Formanek et al., 2017). From the perspectives of learners, while some benefit from peer assessment, others do not find it useful (Meek et al., 2017). Another assessment method mentioned in the research corpus was automated assessments (Beg & Beg, 2018; Rossano et al., 2017; Santamaría et al., 2018); however, it was observed that most of the focus has been directed on handling learning at scale, as opposed to providing effective assessment approaches. Moreover, summative assessment approaches appeared to be less of a concern in the research corpus.

This theme revealed that in terms of assessment and evaluation perspectives, automated assessment approaches were highly favored in the research corpus on account of their use of artificial intelligences and the attractiveness of algorithms. While peer assessment sounds promising, considering the massive number of learners, it might not be a working solution for all learners.

MOOCs for social learning (see path: MOOCs, learning, and social): The studies in the research corpus indicated that most of the social interactions in MOOCs take place on social media platforms (Alario-Hoyos et al., 2016) or in discussions forums, as it is a real challenge to build a one-to-one communication environment where a massive number of students exist (Zhang, Peck et al., 2016). Moreover, the findings support the idea that those who so socially interact (i.e., commenting) tend to complete MOOCs (Sunar et al., 2016; Swinnerton et al., 2017), and that those who demonstrate an active presence are likely to be more engaged and hold strategic positions in the learning network (Wise & Cui, 2018; Zhang, Skryabin et al., 2016). However, there are different types of participants that demonstrate different levels of social engagement (Kahan et al., 2017) or, in some cases, these participants are not visible in socially constructed learning processes (Bozkurt et al., 2020). Studies have therefore suggested that the success of MOOCs cannot be measured based on drop-out or completion rates, but rather, on the learning behaviors of the participants (Kahan et al., 2017). The advantage of MOOCs, in terms of social learning, is their ability to form social learning communities (Gallagher & Savage, 2016) that “would arise around the course, would remain over time, and involve participants contributing to with new proposals” (de Lima & Zorrilla, 2017).

For any educational process, the main ingredient of learning has always been the socially constructed interactions. Third generation MOOCs, in this regard, have benefited from both connectivist MOOC approaches (e.g., social media and networking) and extension MOOC approaches (e.g., discussion forums) to better provide social learning opportunities. However, it is noted that not all learners learn by visible interaction (e.g., lurkers or legitimate peripheral participants) or wish be a part of the entire MOOC (e.g., drop ins). Interestingly, some MOOCs help learners to form a learning community, and these communities provide more learning opportunities, even outside of the defined MOOC concept. The problematic view according to the studies in the research corpus is when social learning is framed around predefined MOOC dates alone, ignoring their contribution to lifelong learning. This perhaps stems from the influence of HE, which tends to resist change, and from interpreting MOOCs from a strictly structured HE view.

Social Network Analysis of the Keywords

Of all the 1312 keywords included in the analysis, 92 had 325 edges with a minimum of three occurrences (Figure 3). In social network analysis, nodes can represent any entity and ties can represent the relationships. In the case of the present study, the keywords of the articles served as the nodes and ties the co-occurrences of the keywords represented the relationships. The sizes of the nodes are defined according to their betweenness centrality (BC) metrics, which can be explained as the bridging score between the other nodes. The thicknesses of ties are defined according to edge weight (EW) values, which emphasize the strength of the relationships. Degree centrality (DC) and PageRank (PR) for each keyword were also calculated.

SNA of the keywords from sampled publications
Figure 3 

SNA of the keywords from sampled publications.

The keywords with the highest betweenness centrality and other social network analysis metrics are given in Appendix 1. The most salient keywords that held strategic positions were as follows: MOOCs (DC: 90; BC: 2832.643), online learning (DC: 29; BC: 142.871), learning analytics (DC: 29; BC: 141.045), Internet (DC: 9; BC: 117.775), e-learning (DC: 24; BC: 86.835), distance learning (DC: 23; BC: 61.252), lifelong learning (DC: 8; BC: 59.971), distance education (DC: 20; BC: 57.073), education (DC: 18; BC: 50.851), and higher education (DC: 21; BC: 48.880). The strongest ties with high EW were as follows: MOOCs – online learning (EW: 241056), e-learning – MOOCs (EW: 155520), learning analytics – MOOCs (EW: 90625), higher education – MOOCs (EW: 65625), and MOOCs – instructional design (EW: 46971). The keywords were grouped into the following seven clusters (Figure 3):

  • Massive Open Online Courses (MOOCs) (Cluster 1),
  • MOOC design, types, and environments (Cluster 2),
  • Educational delivery modes (Cluster 3),
  • Research variables (Cluster 4),
  • MOOC research methods (Cluster 5),
  • Theoretical and conceptual backgrounds (Cluster 6),
  • Disruptive potential of MOOCs (Cluster 7).

Massive Open Online Courses (MOOCs) (Cluster 1): The first cluster includes only the keyword, ‘MOOCs’, in order to see its relationships with other keywords and clusters. The central keyword dominates the whole network and as expected, emerges as a central node.

MOOC design, types, and environments (Cluster 2): The trending topics of this cluster included the Internet as an online learning infrastructure, the need for a specific instructional design, communication and interaction (e.g., social networking, discussion forums, etc.). In addition to the basic MOOC types (e.g., xMOOCs and cMOOCs), small private online courses (SPOCs) emerged as significant keywords. The publications in the research corpus highlight that instructional design in MOOCs is highly complex and challenging (Watson et al. 2016). Moreover, perceived learning matters in MOOCs, likely due to their lifelong dimension (Kim et al., 2016).

Educational delivery modes (Cluster 3): Distance education delivery modes (e.g., e-learning, m-learning, online education), and with a specific focus on HE, open learning, and lifelong learning, were the central nodes in this cluster. The publications also argue that MOOCs are inherent to distance education (Zou, 2016) and are “characterized by openness, online and large-scale, all of which are conducive to creating a profound reform in education and teaching, thus significantly affecting higher education” (Liu, 2017, p. 785). Their application for blended learning (Bralić, & Divjak, 2018) or flipped classrooms (Cao, 2018) further makes them popular from a HE perspective.

Research variables (Cluster 4): Confirming the text mining findings, this cluster indicated that variables like motivation, engagement, completion, quality, assessment and evaluation in computer mediated learning processes are trending hot topics. This finding is in line with earlier research on MOOCs which reported similar variables as the central research focus in MOOC publications (see Albelbisi et al., 2018; Kennedy, 2014).

MOOC research methods (Cluster 5): This cluster indicated that learning analytics was the central keyword, followed by machine learning, content analysis, social network analysis and educational data mining. The publications on implementation of learning analytics in MOOCs (see Bystrova et al., 2018; Yulianto et al., 2018) showed that emerging research methods are used to predict learners’ probability of drop out and success, and their performance.

Theoretical and conceptual backgrounds (Cluster 6): Collaborative learning, peer learning, personalized learning and theoretical approaches, such as self-regulated learning, were salient in this cluster. The research suggests that the primary characteristic of MOOCs, namely their massiveness, hinders the community formation process, and therefore the ability to engage in collaborative learning and peer interaction (Sanz-Martínez et al., 2018). However, it is reported that learners with self-regulated skills tend to reach their goals (Kizilcec et al., 2017; Littlejohn et al., 2016).

Disruptive potential of MOOCs (Cluster 7): Disruption and innovation were the leading keywords in this cluster, which was the smallest to emerge. The impact of the MOOCs in HE is much discussed, with some arguing that MOOCs are more of a sustaining innovation than a disruptive innovation (Al-Imarah, & Shields, 2019; Bozkurt et al., 2017). It is further argued that the disruptive potential of MOOCs does not the fall along the same lines as that seen in the first wave of MOOCs (Connectivist MOOCs), and furthermore, that “behind the MOOC rhetoric of disrupting and democratizing higher education lies the projection of top academic brands on the marketing pedestal, financial piggybacking on the hype, and the politics of academic exclusion” (Rambe & Moeti, 2017, p. 631).

Overall, the social network analysis showed that MOOCs are equated with online education at the HE level. The use of learning analytics has been proposed as a way to address the criticisms (e.g., drop out and retention) and as a tool to increase quality indicators (e.g., motivation and engagement). Self-regulated learning skills are vital to the success of MOOCs, and MOOCs should be viewed not as a disruptive innovation, but rather, as a sustaining innovation.

Analysis of intellectual bibliometric network

Scholarly publications are important sources for explaining, sharing, storing, and disseminating knowledge. Among the different types of scholarly publications, journals and articles have a pivotal role insofar as they are considered reliable because they are supervised and reviewed through an editorial process. Articles, in this regard, serve to build an intellectual network by linking different resources via citations, which are included in the References section. Citing and being cited, in this sense, is an indicator of the value of the scholarly publication and further tell us what the scholarly community focuses on, the direction the disciplines are moving, and how the research interests shift over time.

From this perspective, the present research examined the intellectual bibliometric network of MOOC research, first, by identifying the most cited publications through time zone analysis to identify the pivotal scholarly publications (Figure 4). Second, the present research aimed to reveal temporal patterns, shifting attention as well as collective interest by examining the cited and referenced publications in the sampled articles through timeline analysis (Figure 5).

Articles that were most cited between 2016 and 2018
Figure 4 

Articles that were most cited between 2016 and 2018.

References cited by the articles published between 2016 and 2018
Figure 5 

References cited by the articles published between 2016 and 2018.

In the time zone analysis, the 150 top-cited articles and their relationships among each other were visualized. Of these articles, 83 were labeled as having a minimum citation of ten. The density of the network was found to be 0.1443, which indicates a tightly connected network structure.

The findings revealed that pivotal contributions were made by the articles that covered the issues of motivation (Barak et al., 2016; De Barba et al., 2016; Littlejohn et al., 2016), engagement (Hew, 2016), self-regulated learning skills (Kizilcec et al., 2017), dropout and retention (Hone & El Said, 2016; Xing et al., 2016), continuance intention behaviors to use MOOCs (Wu & Chen, 2017), and innovative potential of MOOCs for online learning (Kaplan & Haenlein, 2016).

The examination of pivotal contributions confirms the findings revealed in text-mining and social network analysis. Accordingly, it can be claimed that the MOOC research created its own research echo-chamber, whereby MOOCs were continually examined from certain perspectives. The questions that are raised in MOOC research stand in contrast to the first wave of cMOOCs’ core values and promises, addressing rather, the second wave xMOOCs, which in turn, has prevented exploration of the real potential of third wave hybrid MOOCs.

In the timeline analysis, 16.389 references cited in the sampled publications were analyzed. To get a clear view, the 134 references that were found to pass the threshold and 988 links were visualized. The density value was 0.1109; the modularity value was 0.4057 and the silhouette value was 0.6008 (Figure 5).

The analysis revealed that though some of the references dated back earlier than the emergence of MOOCs, the first footprints of MOOC research became visible in the seminal work of Siemens (2005), Connectivism: A learning theory for the digital age, which introduced principles of networked learning and connectivism and further inspired the first wave cMOOCs. While the focal point of MOOC research in 2011 was cMOOCs and the concepts of diversity, participation, collaboration and networked learning (for example, see Kop et al., 2011; Mackness et al., 2010; McAuley et al., 2010), intellectual attention turns towards extension MOOCs (xMOOCs) in 2012 by comparing two different pedagogical MOOC types (see Rodriguez, 2012; Yuan & Powell, 2013) and problems of xMOOCs. By the time of the advent and dominance of xMOOCs in related literature (Bozkurt et al., 2017), different issues, like motivational challenges (Hew & Cheung, 2014), instructional quality (Margaryan et al., 2015), addressing diversity of learners and learning at scale (Breslow et al., 2013), and retention and dropout rates (Jordan, 2014) are the main concerns.

The timeline analysis demonstrated that first wave cMOOCs ignited a scholarly intellectual landscape, where publications (from 2008 to 2011) that interpreted MOOCs as an open, networked, and participatory practice were at the forefront. Publications from 2012 to 2015 shifted their attention to xMOOCs. By 2016, hybrid MOOC designs emerged. From connectivist MOOCs to extension MOOCs, from extension MOOCs to hybrid MOOCs, three waves of MOOCs are shown in Figure 6.

The three waves of MOOCs
Figure 6 

The three waves of MOOCs.

The findings of the timeline analysis were in line with the results from the pivotal contribution analysis and further confirmed the results derived from the text mining and social network analysis. To this end, it was seen that constantly articulated issues have become the major concerns, with the focal point of MOOC research being motivational challenges, instructional quality, and low retention and high drop-out rates.

Conclusions and suggestions for future research directions

This study examined a total of 633 scholarly MOOC publications from 2016–2018. The text mining revealed four major themes: (I) MOOCs as a mainstreaming learning model in HE, (II) motivation and engagement issues in MOOCs, (III) assessment issues in MOOCs, and (IV) MOOCs for social learning. In the social network analysis, the keywords used in the research corpus were examined in 7 clusters, and it was found that MOOCs are utilized by HE through online learning delivery modes. It was noteworthy that learning analytics is considered to be a viable solution to many ongoing criticisms. In line with findings of the social network analysis of the keywords, the bibliometric social network analysis conducted by examining pivotal contributions and through timeline analysis showed that MOOCs still suffer from low retention and high drop-out rates, two issues at the center of many of the negative discourses on this subject. Moreover, the instructional quality of MOOCs remains a topic under question, and it was found that researchers frequently examine motivational and engagement factors, as they relate to MOOCs.

In the three phases of the examination of the research data, that is, lexical analysis through text mining, and keyword and bibliometric analysis through social network analysis, some interesting patterns emerged in MOOC research. In reference to the letters constituting the MOOC abbreviation, it was found that most of the current discourses have shifted to the quantified massiveness features of the MOOCs. In this context, massiveness is largely interpreted as the number of students, and arguments are built upon this notion. Second, while the qualitative feature, that is, openness, was the central issue of focus in the first wave connectivist MOOCs, it was less of a concern in the second wave extension and third wave hybrid MOOCs. In this sense, it can be argued that from the inception of the first wave connectivist MOOCs, the concept has evolved enormously, with quantitative values having replaced qualitative core values as focuses of attention in research. This can be further interpreted as the assimilation of the original MOOC concept. For instance, openness refers to the flexibility of access to enter and withdraw easily, which was one of the most valuable promises of first wave MOOCs. In contrast, second wave MOOCs see this flexibility as a limitation, linking it to drop out and retention rates. While the first wave MOOCs was a fertile territory for leisure learners, and learning was associated with perceived learning, the second and third wave MOOCs strived to keep the learners in the MOOC, and thus, motivation and engagement have become a trending hot topic. Additionally, it can be concluded that while the first wave MOOCs were considered to be a disruptive innovation because it challenged conventional education with its features of openness, the second and third wave MOOCs served well as a sustaining innovation for traditional education, that is, brick and mortar HE institutions. Finally, it can be argued that ignoring the openness feature of MOOCs hinders its real potential, in terms of social justice and widening participation.

Referring to third letter in the MOOC acronym, online has become significant, not only for MOOCs but also for HE institutions due to the capacity increase to reach many learners, especially for-profit purposes. Rather than exploiting opportunities that are inherent to online learning, learning analytics, to better monitor learners, and automated assessment and evaluation approaches, to deal with the massive number of learners, come to the forefront. Ignoring for the time being the instructional quality and learners’ needs, course, which is represented with the final letter in the MOOC acronym, is now associated with HE practices instead of with the lifelong learning journey. Interestingly, MOOCs have also been integrated with conventional education as a part of the blended learning and flipped classroom models, and the current efforts in micro credentials and nano degrees might yield interesting results.

Based on the research findings and the impressions gained from the examined publications, this study argues that the real potential of MOOCs cannot be quantitatively measured, but rather, this potential should be considered in terms of the qualitative contributions provided by MOOCs. To this end, it is suggested that MOOC providers focus more on the social justice and widening participation aspects of MOOCs.

Appendix

Appendix 1

Top 30 keywords with highest betweenness centrality.


VERTEX DEGREE BETWEENNESS CENTRALITY CLOSENESS CENTRALITY EIGENVECTOR CENTRALITY PAGERANK CLUSTERING COEFFICIENT

MOOCs 90 2832.643 0.011 0.064 9.541 0.079

online learning 29 142.871 0.006 0.027 3.003 0.180

learning analytics 29 141.045 0.006 0.027 3.064 0.197

internet 9 117.775 0.006 0.010 1.164 0.361

e-learning 24 86.835 0.006 0.025 2.494 0.264

distance learning 23 61.252 0.006 0.026 2.326 0.312

lifelong learning 8 59.971 0.006 0.012 0.962 0.464

distance education 20 57.073 0.006 0.022 2.086 0.326

education 18 50.851 0.006 0.020 1.897 0.320

higher education 21 48.880 0.006 0.024 2.135 0.305

computer-mediated communication 4 31.283 0.006 0.005 0.611 0.500

instructional design 16 30.583 0.006 0.016 1.721 0.333

flipped classroom 7 29.687 0.006 0.008 0.904 0.333

learning environments 18 29.426 0.006 0.020 1.818 0.405

motivation 13 26.261 0.006 0.012 1.561 0.269

Coursera 15 25.581 0.006 0.018 1.543 0.381

virtual reality 4 21.192 0.006 0.007 0.553 0.500

teaching reform 3 15.696 0.006 0.005 0.499 0.333

open learning 12 15.576 0.006 0.014 1.353 0.394

social networks 12 14.392 0.006 0.014 1.295 0.439

online education 11 11.676 0.006 0.014 1.217 0.455

SPOC 10 11.033 0.006 0.012 1.114 0.400

collaborative learning 11 9.833 0.006 0.013 1.196 0.473

discussion forums 10 9.268 0.006 0.013 1.124 0.556

connectivism 10 8.150 0.006 0.011 1.098 0.489

assessment 8 7.750 0.006 0.010 0.999 0.393

blended learning 10 7.210 0.006 0.015 1.082 0.533

teaching 10 6.900 0.006 0.014 1.085 0.556

machine learning 8 6.867 0.006 0.010 0.959 0.393

educational technology 9 6.411 0.006 0.012 0.997 0.556

engagement 8 6.075 0.006 0.010 0.930 0.464

Data Accessibility Statement

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Funding Information

This study was supported by Anadolu University, Scientific Research Projects commission as part of the project entitled “Examination of digital transformation and blended hybrid learning designs in higher education in the context of distance education” with grant no 2106E084.

Competing Interests

The author has no competing interests to declare.

References

  1. Alario-Hoyos, C., Estévez-Ayres, I., Pérez-Sanagustín, M., Delgado Kloos, C., & Fernández-Panadero, C. (2017). Understanding Learners’ Motivation and Learning Strategies in MOOCs. The International Review of Research in Open and Distributed Learning, 18(3). DOI: https://doi.org/10.19173/irrodl.v18i3.2996 

  2. Alario-Hoyos, C., Muñoz-Merino, P. J., Pérez-Sanagustín, M., Delgado Kloos, C., & Parada, G. H. A. (2016). Who are the top contributors in a MOOC? Relating participants’ performance and contributions. Journal of Computer Assisted Learning, 32(3), 232–243. DOI: https://doi.org/10.1111/jcal.12127 

  3. Albelbisi, N., Yusop, F. D., & Salleh, U. K. M. (2018). Mapping the factors influencing success of massive open online courses (MOOC) in Higher Education. Eurasia Journal of Mathematics, Science and Technology Education, 14(7), 2995–3012. DOI: https://doi.org/10.29333/ejmste/91486 

  4. Al-Imarah, A. A., & Shields, R. (2019). MOOCs, disruptive innovation and the future of higher education: A conceptual analysis. Innovations in Education and Teaching International, 56(3), 258–269. DOI: https://doi.org/10.1080/14703297.2018.1443828 

  5. Barak, M., Watted, A., & Haick, H. (2016). Motivation to learn in massive open online courses: Examining aspects of language and social engagement. Computers & Education, 94, 49–60. DOI: https://doi.org/10.1016/j.compedu.2015.11.010 

  6. Beg, A., & Beg, A. (2018). Using open technologies for automatically creating question-and-answer sets for engineering MOOCs. Computer Applications in Engineering Education, 26(3), 617–625. DOI: https://doi.org/10.1002/cae.21913 

  7. Belleflamme, P., & Jacqmin, J. (2016). An economic appraisal of MOOC platforms: business models and impacts on higher education. CESifo Economic Studies, 62(1), 148–169. DOI: https://doi.org/10.1093/cesifo/ifv016 

  8. Bozkurt, A., Akgün-Özbek, E., & Zawacki-Richter, O. (2017). Trends and Patterns in Massive Open Online Courses: Review and Content Analysis of Research on MOOCs (2008–2015). International Review of Research in Open and Distributed Learning, 18(5), 118–147. DOI: https://doi.org/10.19173/irrodl.v18i5.3080 

  9. Bozkurt, A., Koutropoulos, A., Singh, L., & Honeychurch, S. (2020). On Lurking: Multiple perspectives on lurking within an educational community. The Internet and Higher Education, 44(2020), 100709. DOI: https://doi.org/10.1016/j.iheduc.2019.100709 

  10. Bozkurt, A., Ozdamar Keskin, N., & de Waard, I. (2016). Research trends in massive open online course (MOOC) theses and dissertations: Surfing the tsunami wave. Open Praxis, 8(3), 203–221. DOI: https://doi.org/10.5944/openpraxis.8.3.287 

  11. Bralić, A., & Divjak, B. (2018). Integrating MOOCs in traditionally taught courses: achieving learning outcomes with blended learning. International Journal of Educational Technology in Higher Education, 15(1), 2. DOI: https://doi.org/10.1186/s41239-017-0085-7 

  12. Breslow, L., Pritchard, D. E., DeBoer, J., Stump, G. S., Ho, A. D., & Seaton, D. T. (2013). Studying learning in the worldwide classroom research into edX’s first MOOC. Research & Practice in Assessment, 8, 13–25. https://www.rpajournal.com/dev/wp-content/uploads/2013/05/SF2.pdf 

  13. Bulfin, S., Pangrazio, L., & Selwyn, N. (2014). Making MOOCs: The construction of a new digital higher education within news media discourse. The International Review of Research in Open and Distributed Learning, 15(5), 291–305. DOI: https://doi.org/10.19173/irrodl.v15i5.1856 

  14. Bystrova, T., Larionova, V. A., Sinitsyn, E., & Tolmachev, A. (2018). Learning Analytics in Massive Open Online Courses as a Tool for Predicting Learner Performance. Educational Studies Moscow, 4, 139–166. DOI: https://doi.org/10.17323/1814-9545-2018-4-139-166 

  15. Calonge, D. S., & Shah, M. A. (2016). MOOCs, Graduate Skills Gaps, and Employability: A Qualitative Systematic Review of the Literature. The International Review of Research in Open and Distributed Learning, 17(5), 67–90. DOI: https://doi.org/10.19173/irrodl.v17i5.2675 

  16. Cao, J. (2018). The Future Development Practice and Research on Flipped Classroom Model Based on MOOC in Higher Education. Journal of Advanced Oxidation Technologies, 21(2). 

  17. Chen, Y. (2014). Investigating MOOCs through blog mining. The International Review of Research in Open and Distributed Learning, 15(2), 85–106. DOI: https://doi.org/10.19173/irrodl.v15i2.1695 

  18. Costa, C., Teixeira, L., & Alvelos, H. (2018). Exploring the Usage of MOOCs in Higher Education Institutions: Characterization of the Most Used Platforms. International Journal of Information and Communication Technology Education (IJICTE), 14(4), 1–17. DOI: https://doi.org/10.4018/IJICTE.2018100101 

  19. Costello, E., Brown, M., Mhichíl, M. N. G., & Zhang, J. (2018). Big course small talk: twitter and MOOCs—a systematic review of research designs 2011–2017. International Journal of Educational Technology in Higher Education, 15(44), 1–16. DOI: https://doi.org/10.1186/s41239-018-0127-9 

  20. De Barba, P. G., Kennedy, G. E., & Ainley, M. D. (2016). The role of students’ motivation and participation in predicting performance in a MOOC. Journal of Computer Assisted Learning, 32(3), 218–231. DOI: https://doi.org/10.1111/jcal.12130 

  21. de Lima, M., & Zorrilla, M. E. (2017). Social networks and the building of learning communities: An experimental study of a social MOOC. International Review of Research in Open and Distributed Learning, 18(1), 40–64. DOI: https://doi.org/10.19173/irrodl.v18i1.2630 

  22. Deimann, M. (2015). The dark side of the MOOC: A critical inquiry on their claims and realities. Current Issues in Emerging eLearning, 2(1). https://scholarworks.umb.edu/ciee/vol2/iss1/3/ 

  23. Ebben, M., & Murphy, J. S. (2014). Unpacking MOOC scholarly discourse: A review of nascent MOOC scholarship. Learning, Media and Technology, 39(3), 328–345. DOI: https://doi.org/10.1080/17439884.2013.878352 

  24. Egloffstein, M., & Ifenthaler, D. (2017). Employee perspectives on MOOCs for workplace learning. TechTrends, 61(1), 65–70. DOI: https://doi.org/10.1007/s11528-016-0127-3 

  25. Fayyad, U., Grinstein, G. G., & Wierse, A. (Eds.). (2002). Information visualization in data mining and knowledge discovery. Morgan Kaufmann. 

  26. Formanek, M., Wenger, M. C., Buxner, S. R., Impey, C. D., & Sonam, T. (2017). Insights about large-scale online peer assessment from an analysis of an astronomy MOOC. Computers & Education, 113, 243–262. DOI: https://doi.org/10.1016/j.compedu.2017.05.019 

  27. Gallagher, S. E., & Savage, T. (2016). Comparing learner community behavior in multiple presentations of a massive open online course. Journal of Computing in Higher Education, 28(3), 358–369. DOI: https://doi.org/10.1007/s12528-016-9124-y 

  28. Gašević, D., Kovanovi, V., Joksimovi, S., & Siemens, G. (2014). Where is research on massive open online courses headed? A data analysis of the MOOC Research Initiative. The International Review of Research in Open and Distributed Learning, 15(5), 134–176. DOI: https://doi.org/10.19173/irrodl.v15i5.1954 

  29. Gough, D., Oliver, S., & Thomas, J. (2012). An introduction to systematic reviews. Sage. 

  30. Haddadi, L., Bouarab-Dahmani, F., Guin, N., Berkane, T., & Lazib, S. (2018). Peer assessment and groups formation in massive open online courses. Computer Applications in Engineering Education, 26(5), 1873–1887. DOI: https://doi.org/10.1002/cae.22005 

  31. Hansen, D., Shneiderman, B., & Smith, M. A. (2010). Analyzing social media networks with NodeXL: Insights from a connected world. Morgan Kaufmann. DOI: https://doi.org/10.1016/B978-0-12-382229-1.00002-3 

  32. Hearst, M. (2003). What is text mining? http://people.ischool.berkeley.edu/~hearst/text-mining.html 

  33. Hew, K. F. (2016). Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS. British Journal of Educational Technology, 47(2), 320–341. DOI: https://doi.org/10.1111/bjet.12235 

  34. Hew, K. F., & Cheung, W. S. (2014). Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review, 12, 45–58. DOI: https://doi.org/10.1016/j.edurev.2014.05.001 

  35. Hone, K. S., & El Said, G. R. (2016). Exploring the factors affecting MOOC retention: A survey study. Computers & Education, 98, 157–168. DOI: https://doi.org/10.1016/j.compedu.2016.03.016 

  36. Joksimović, S., Poquet, O., Kovanović, V., Dowell, N., Mills, C., Gašević, D., … & Brooks, C. (2018). How do we model learning at scale? A systematic review of research on MOOCs. Review of Educational Research, 88(1), 43–86. DOI: https://doi.org/10.3102/0034654317740335 

  37. Jordan, K. (2014). Initial trends in enrolment and completion of massive open online courses. The International Review of Research in Open and Distributed Learning, 15(1). DOI: https://doi.org/10.19173/irrodl.v15i1.1651 

  38. Kahan, T., Soffer, T., & Nachmias, R. (2017). Types of Participant Behavior in a Massive Open Online Course. The International Review of Research in Open and Distributed Learning, 18(6). DOI: https://doi.org/10.19173/irrodl.v18i6.3087 

  39. Kaplan, A. M., & Haenlein, M. (2016). Higher education and the digital revolution: About MOOCs, SPOCs, social media, and the Cookie Monster. Business Horizons, 59(4), 441–450. DOI: https://doi.org/10.1016/j.bushor.2016.03.008 

  40. Kennedy, J. (2014). Characteristics of massive open online courses (MOOCs): A research review, 2009–2012. Journal of Interactive Online Learning, 13(1), 1–16. DOI: https://doi.org/10.1007/978-3-658-26296-9_3 

  41. Kim, W., Watson, S. L., & Watson, W. R. (2016). Perceived learning in three MOOCs targeting attitudinal change. Educational Media International, 53(3), 168–183. DOI: https://doi.org/10.1080/09523987.2016.1236890 

  42. Kizilcec, R. F., Pérez-Sanagustín, M., & Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers & education, 104, 18–33. DOI: https://doi.org/10.1016/j.compedu.2016.10.001 

  43. Kop, R., Fournier, H., & Mak, J. S. F. (2011). A pedagogy of abundance or a pedagogy to support human beings? Participant support on massive open online courses. The International Review of Research in Open and Distributed Learning, 12(7), 74–93. DOI: https://doi.org/10.19173/irrodl.v12i7.1041 

  44. Kovanović, V., Joksimović, S., Gašević, D., Siemens, G., & Hatala, M. (2015). What public media reveals about MOOCs: A systematic analysis of news reports. British Journal of Educational Technology, 46(3), 510–527. DOI: https://doi.org/10.1111/bjet.12277 

  45. Lambert, S. R. (2020). Do MOOCs contribute to student equity and social inclusion? A systematic review 2014–18. Computers & Education, 145, 103693. DOI: https://doi.org/10.1016/j.compedu.2019.103693 

  46. Lee, D., Watson, S. L., & Watson, W. R. (2019). Systematic literature review on self-regulated learning in massive open online courses. Australasian Journal of Educational Technology, 35(1), 28–41. DOI: https://doi.org/10.14742/ajet.3749 

  47. Lemoine, P. A., & Richardson, M. D. (2015). Micro-Credentials, Nano Degrees, and Digital Badges: New Credentials for Global Higher Education. International Journal of Technology and Educational Marketing (IJTEM), 5(1), 36–49. DOI: https://doi.org/10.4018/ijtem.2015010104 

  48. Li, Y., Sun, J., & Sun, M. (2018). Analysis of the development status and impact of MOOCs in American higher education. Educational Sciences: Theory & Practice, 18(6), 3442–3448. DOI: https://doi.org/10.12738/estp.2018.6.251 

  49. Littlejohn, A., Hood, N., Milligan, C., & Mustain, P. (2016). Learning in MOOCs: Motivations and self-regulated learning in MOOCs. The Internet and Higher Education, 29, 40–48. DOI: https://doi.org/10.1016/j.iheduc.2015.12.003 

  50. Liu, L. (2017). The influence and countermeasure of Mooc for Chinese tertiary education based on open and distance education. Agro Food Industry Hi-Tech, 28(3), 785–788. DOI: https://doi.org/10.1108/nfs.2008.01738aab.008 

  51. Liyanagunawardena, T., Adams, A., & Williams, S. (2013). MOOCs: A systematic study of the published literature 2008–2012. The International Review of Research in Open and Distributed Learning, 14(3), 202–227. DOI: https://doi.org/10.19173/irrodl.v14i3.1455 

  52. Mackness, J., Mak, S., & Williams, R. (2010). The ideals and reality of participating in a MOOC. In Proceedings of the 7th international conference on networked learning 2010 (pp. 266–275). United Kingdom: University of Lancaster. 

  53. Margaryan, A., Bianco, M., & Littlejohn, A. (2015). Instructional quality of massive open online courses (MOOCs). Computers & Education, 80, 77–83. DOI: https://doi.org/10.1016/j.compedu.2014.08.005 

  54. McAuley, A., Stewart, B., Siemens, G., & Cormier, D. (2010). The MOOC model for digital practice. Charlottetown, Canada: University of Prince Edward Island. http://www.elearnspace.org/Articles/MOOC_Final.pdf 

  55. Meek, S. E., Blakemore, L., & Marks, L. (2017). Is peer review an appropriate form of assessment in a MOOC? Student participation and performance in formative peer review. Assessment & Evaluation in Higher Education, 42(6), 1000–1013. DOI: https://doi.org/10.1080/02602938.2016.1221052 

  56. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., … & Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372. DOI: https://doi.org/10.1136/bmj.n71 

  57. Pursel, B. K., Zhang, L., Jablokow, K. W., Choi, G. W., & Velegol, D. (2016). Understanding MOOC students: motivations and behaviours indicative of MOOC completion. Journal of Computer Assisted Learning, 32(3), 202–217. DOI: https://doi.org/10.1111/jcal.12131 

  58. Raffaghelli, J., Cucchiara, S., & Persico, D. (2015), Methodological approaches in MOOC research: Retracing the myth of Proteus. British Journal of Educational Technologies, 46(3), 488–509. DOI: https://doi.org/10.1111/bjet.12279 

  59. Rambe, P., & Moeti, M. (2017). Disrupting and democratising higher education provision or entrenching academic elitism: towards a model of MOOCs adoption at African universities. Educational Technology Research and Development, 65(3), 631–651. DOI: https://doi.org/10.1007/s11423-016-9500-3 

  60. Rodriguez, C. O. (2012). MOOCs and the AI-Stanford Like Courses: Two Successful and Distinct Course Formats for Massive Open Online Courses. European Journal of Open, Distance and E-Learning. http://www.eurodl.org/?p=current&article&article=516#MobiMOOC 

  61. Rossano, V., Pesare, E., & Roselli, T. (2017). Are computer adaptive tests suitable for assessment in MOOCs. Journal of e-Learning and Knowledge Society, 13(3). DOI: https://doi.org/10.20368/1971-8829%2F1393 

  62. Sa’don, N. F., Alias, R. A., & Ohshima, N. (2014). Nascent research trends in MOOCs in higher educational institutions: A systematic literature review. In Web and Open Access to Learning (ICWOAL), 2014 International Conference (pp. 1–4). IEEE. DOI: https://doi.org/10.1109/ICWOAL.2014.7009215 

  63. Sangrà, A., González-Sanmamed, M., & Anderson, T. (2015). Metaanálisis de la investigación sobre MOOC en el período 2013–2014. Educación XXI: revista de la Facultad de Educación, 18(2), 21–49. DOI: https://doi.org/10.5944/educxx1.13463 

  64. Santamaría Lancho, M., Hernández, M., Sánchez-Elvira Paniagua, Á., Luzón Encabo, J. M., & de Jorge-Botana, G. (2018). Using Semantic Technologies for Formative Assessment and Scoring in Large Courses and MOOCs. Journal of Interactive Media in Education, 2018(1), 1–10. DOI: https://doi.org/10.5334/jime.468 

  65. Sanz-Martínez, L., Er, E., Dimitriadis, Y. A., Martínez-Monés, A., & Bote-Lorenzo, M. L. (2018). Supporting Teachers in the Design and Implementation of Group Formation Policies in MOOCs: A Case Study. Journal of Universal Computer Science, 24(8), 1110–1130. http://www.jucs.org/jucs_24_8/supporting_teachers_in_the/jucs_24_08_1110_1130_martinez.pdf 

  66. Shen, C. W., & Kuo, C. J. (2015). Learning in massive open online courses: Evidence from social media mining. Computers in Human Behavior, 51, 568–577. DOI: https://doi.org/10.1016/j.chb.2015.02.066 

  67. Siemens, G. (2005). Connectivism: A learning theory for the digital age. International Journal of Instructional Technology and Distance Learning, 2(1). http://www.itdl.org/Journal/Jan_05/article01.htm 

  68. Sinclair, J., & Kalvala, S. (2016). Student engagement in massive open online courses. International Journal of Learning Technology (IJLT), 11(3), 218–237. DOI: https://doi.org/10.1504/IJLT.2016.079035 

  69. Sunar, A. S., White, S., Abdullah, N. A., & Davis, H. C. (2016). How learners’ interactions sustain engagement: a MOOC case study. IEEE Transactions on Learning Technologies, 10(4), 475–487. DOI: https://doi.org/10.1109/TLT.2016.2633268 

  70. Swinnerton, B., Hotchkiss, S., & Morris, N. P. (2017). Comments in MOOCs: who is doing the talking and does it help? Journal of Computer Assisted Learning, 33(1), 51–64. DOI: https://doi.org/10.1111/jcal.12165 

  71. Thurmond, V. A. (2001). The point of triangulation. Journal of Nursing Scholarship, 33(3), 253–258. DOI: https://doi.org/10.1111/j.1547-5069.2001.00253.x 

  72. Veletsianos, G., & Shepherdson, P. (2015). Who studies MOOCs? Interdisciplinarity in MOOC research and its changes over time. The International Review of Research in Open and Distributed Learning, 16(3), 1–17. DOI: https://doi.org/10.19173/irrodl.v16i3.2202 

  73. Veletsianos, G., & Shepherdson, P. (2016). A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. The International Review of Research in Open and Distributed Learning, 17(2), 198–221. DOI: https://doi.org/10.19173/irrodl.v17i2.2448 

  74. Ward, V., West, R., Smith, S., McDermott, S., Keen, J., Pawson, R., & House, A. (2014). The role of informal networks in creating knowledge among health-care managers: a prospective case study. Health Services and Delivery Research, 2(12). http://eprints.whiterose.ac.uk/92126/1/FullReport-hsdr02120.pdf. DOI: https://doi.org/10.3310/hsdr02120 

  75. Watson, S. L., Loizzo, J., Watson, W. R., Mueller, C., Lim, J., & Ertmer, P. A. (2016). Instructional design, facilitation, and perceived learning outcomes: An exploratory case study of a human trafficking MOOC for attitudinal change. Educational Technology Research and Development, 64(6), 1273–1300. DOI: https://doi.org/10.1007/s11423-016-9457-2 

  76. Wise, A. F., & Cui, Y. (2018). Learning communities in the crowd: Characteristics of content related interactions and social relationships in MOOC discussion forums. Computers & Education, 122, 221–242. DOI: https://doi.org/10.1016/j.compedu.2018.03.021 

  77. Wong, J., Baars, M., Davis, D., Van Der Zee, T., Houben, G. J., & Paas, F. (2019). Supporting Self-Regulated Learning in Online Learning Environments and MOOCs: A Systematic Review. International Journal of Human–Computer Interaction, 35(4–5), 356–373. DOI: https://doi.org/10.1080/10447318.2018.1543084 

  78. Wu, B., & Chen, X. (2017). Continuance intention to use MOOCs: Integrating the technology acceptance model (TAM) and task technology fit (TTF) model. Computers in Human Behavior, 67, 221–232. DOI: https://doi.org/10.1016/j.chb.2016.10.028 

  79. Xing, W., Chen, X., Stein, J., & Marcinkowski, M. (2016). Temporal predication of dropouts in MOOCs: Reaching the low hanging fruit through stacking generalization. Computers in Human Behavior, 58, 119–129. DOI: https://doi.org/10.1016/j.chb.2015.12.007 

  80. Xiong, Y., & Suen, H. K. (2018). Assessment approaches in massive open online courses: Possibilities, challenges and future directions. International Review of Education, 64(2), 241–263. DOI: https://doi.org/10.1007/s11159-018-9710-5 

  81. Yuan, L., & Powell, S. (2013). MOOCs and open education: Implications for higher education. Bolton, UK: JISC, Centre for Educational Technology & Interoperability Standards. http://publications.cetis.ac.uk/2013/2667 

  82. Yulianto, B., Prabowo, H., Kosala, R., & Hapsara, M. (2018). Implementation of Learning Analytics in MOOC by Using Artificial Unintelligence. Journal of Computer Science, 14(3), 317–323. DOI: https://doi.org/10.17323/1814-9545-2018-4-139-166 

  83. Zawacki-Richter, O., Bozkurt, A., Alturki, U., & Aldraiweesh, A. (2018). What research says about MOOCs – An explorative content analysis. International Review of Research in Open and Distributed Learning, 19(1), 242–259. DOI: https://doi.org/10.19173/irrodl.v19i1.3356 

  84. Zhang, J., Skryabin, M., & Song, X. (2016). Understanding the dynamics of MOOC discussion forums with simulation investigation for empirical network analysis (SIENA). Distance education, 37(3), 270–286. DOI: https://doi.org/10.1080/01587919.2016.1226230 

  85. Zhang, Q., Peck, K. L., Hristova, A., Jablokow, K. W., Hoffman, V., Park, E., & Bayeck, R. Y. (2016). Exploring the communication preferences of MOOC learners and the value of preference-based groups: Is grouping enough? Educational Technology Research and Development, 64(4), 809–837. DOI: https://doi.org/10.1007/s11423-016-9439-4 

  86. Zhu, M., Sari, A., & Lee, M. M. (2018). A systematic review of research methods and topics of the empirical MOOC literature (2014–2016). The Internet and Higher Education, 37, 31–39. DOI: https://doi.org/10.1016/j.iheduc.2018.01.002 

  87. Zou, M. (2016). Exploration of Application Mode of MOOC-based Distance Education Mode in Digital Illustration. International Journal of Emerging Technologies in Learning (iJET), 11(09), 61–65. DOI: https://doi.org/10.3991/ijet.v11i09.6127 

comments powered by Disqus