Massive Open Online Courses (MOOCs) are tuition-free, open-enrollment learning environments that have transformed online education (North, Ronny & Max, 2014). Through MOOCs, participants have the opportunity to interact with each other and with course materials regardless of geographic location. By design, MOOC participants engage with a combination of online reading materials, videos, quizzes, discussion forums, and assessments throughout their learning. Due to their openness, MOOC environments afford participants the autonomy to choose what, when, and with whom they will interact, allowing participants to proceed at their own pace. A drawback of these two intrinsic characteristics is that MOOCs are also known for a high occurrence of participant drop out (Rosé et al., 2014; Ho et al., 2014; Coetzee, Fox, Hearst & Hartmann, 2014; Yang, Sinha, Adamson & Rosé, 2013). This high level of dropout has drawn the attention of researchers for better understanding of to what extent participants’ engagement with the MOOC contributes to their achievement or conclusion (e.g., Onah, Sinclair & Boyatt, 2014; Sinha, Jermann, Li & Dillenbourg, 2014).
Although MOOCs have gained popularity as free courses in which anyone can participate, MOOCs for teachers have emerged as a new wave in MOOCs, providing free professional development for teachers around the globe and the opportunity for these teachers (participants) to establish a global community (network). Similar to regular MOOCs such as the ones provided by Coursera or edX platforms, MOOCs for teachers also make use of videos and discussion forums as a material base. Thus, this paper explores the effect participants’ engagement has on their course completion in a MOOC designed for statistics teachers that is offered by a large American university. Using logistic regression models the study presents the degree to which participants’ professional background, and their engagement with videos and with forums can be the basis for predicting course completion and receiving a certificate.
Videos and discussion forums are primarily and extensively used in MOOC design. Videos serve as a main entrance, attracting students to manage learning at their own pace. Discussion forums are opportunities for participants to establish connections with other participants, to present their knowledge, and to gain from mutual sharing in a community that is being created. In this section the literature on videos, discussion forums, and MOOC completion are briefly reviewed to inform the basis of the present study.
Videos are the primary venue of content delivery in MOOCs. They are used for a variety of purposes including content development, tutorials and demonstrations, lectures, guides for assignment completion, etc. (Sinha et al., 2014). Videos also contribute to engagement in discussion forums by providing input for topics to be discussed later by participants. Videos vary in length, in position within the course, and in purpose. Morris and Lambe (2014) describe a classification for MOOC videos characterizing them as: (a) introductory videos in which course instructors explain the course and its purpose, (b) video recordings of lectures given to real students, (c) animations with audio narration in which the course content is explained, (d) documentary style video, (e) interviews or conversations among instructors and guests, and (f) video with built-in questions. When MOOCs are designed for teachers, designers may make use of a particular video style or a combination of different styles according to the purpose of learning.
Research highlights the benefits of videos in online learning, such as the ability to pause, repeat or skip, allowing flexibility in the learning process and reinforcing student autonomy in MOOCs (Triay, Sancho-Vinuesa, Minguillón & Daza, 2016; Morris & Lambe, 2014). For Glance, Forsey and Riley (2013), the use of short videos intertwined with quizzes emulates one–on–one tutoring and tends to fit into a manageable amount of time that students can dedicate to MOOCs. Knowing that video watching is an activity under participants’ control (who decide if and when they will engage with videos), research on the context of video lectures has investigated video features and video styles that relate to students’ engagement and lead to viewership activity (e.g., Guo, Kim & Rubin, 2014). Although participants’ engagement with videos is an essential element in MOOC design, Sinha et al. (2014) state that from “100% students who register [in a MOOC], 75% show up: 50% of them primarily watch video lectures and the rest 25% additionally work out homework and assignments” (p. 02). This indicates that MOOC participants heavily rely on videos as their main MOOC activity. According to these authors, participants’ engagement with videos in MOOCs is based on their perception of video lectures as being difficult or simple to understand, their expertise level on the subject matter, and their motivation to learn or pursue specific outcomes.
Another important component of MOOCs are discussion forums. Forums can be a space for participants to interact with others, get to know their peers, and learn through their experiences (Young, 2012). They can also be a space for participants to check their understanding of the subject matter and to ask questions regarding a task or an assignment (Young, 2012; Darabi, Arrastia, Nelson, Cornille & Liang, 2011). Forums replace the face–to–face tutorial mode of teaching, allowing a voice for any of the MOOC’s participants (Walker, 2007). Forums are an environment for cooperation among participants (Coetzee et al., 2014) in which they form relationships (Graham & Misanchuk, 2005) and learn from their interactions with others and from their interactions with course materials (Thomas, 2002).
Research presents inconclusive results regarding the effectiveness/benefits of discussion forums in online environments. Some researchers found that discussion forums prompt collaborative thinking (Ruberg, Moore & Taylor, 1996) by encouraging participants to reflect on peers’ contributions (posts), and engage in a higher order of thinking as they articulate their own understanding of the theme being discussed and/or associating it with previous contributions from peers (previous posts) (Walker, 2007; Bates, 1995). Due to their written characteristic, forums are spaces in which the lack of non-verbal clues allows for a democratic communication among participants (Ruberg et al., 1996). This implies opportunities for introverted participants to actively engage in discussions alongside their extroverted counterparts. Forums provide spaces in which everyone’s perspective can be stated with equal value (Shank & Cunningham, 1996). In addition to giving a voice to participants, the implementation of discussion forums provides opportunities for participants to initiate discussions and to drive their own learning (Darabi et al., 2011).
In contrast, critics of the use of discussion forums state that forums produce isolated participation in discussion threads and do not encourage participants to interact with each other. This perspective sees forum participants as acting individually based upon the theme that is being discussed and upon their understanding of another participant’s writing (Thomas, 2002). Since in MOOCs a significant portion of forum posts do not receive any type of reply or view, Thomas (2002) argues that it would be more appropriate to conceptualize forums as data storage areas that can be accessed by participants instead of as spaces for collaborative engagement. Although some may conceive forums as spaces for peer collaboration and democratic participants’ contribution, forums may perform as spaces for individual voices that in some cases aren’t heard by anyone reducing them to spaces for participants’ opinions instead of interactive dialogue among them (Thomas, 2002). It is also possible that participants engage in forums as a means of information acquisition rather than critical thinking (Kanuka & Anderson, 2007). For these authors, forums may support students’ increase of knowledge, but still fall short in presenting evidence of being a venue for development of students’ new knowledge.
As free and open courses, MOOCs often attract a high volume of participants, although only a small percentage of enrolled participants effectively complete the course (Gutiérrez-Rojas, Alario-Hoyos, Pérez-Sanagustín, Leony, & Delgado-Kloos, 2014). Dropouts have been described in the MOOC literature as a drawback of this initiative that aims to make knowledge available to many people (e.g., Liyanagunawardena, Adams & Williams, 2013; Anderson, 2013; Carr, 2012). According to Ramesh, Goldwasser, Huang, Daumé III and Getoor (2013) even for those participants who stated their intention of completing the MOOC during the registration process, 75% of these individuals do not conclude the course.
The reasons for this high level of drop-out may be associated with a combination of factors such as students’ autonomy, their perceptions of openness, lack of interactivity, diversity among participants, lack of financial commitment, age, time management, and self-motivation (Rosé et al., 2014; Wen, Yang & Rosé, 2014; Ramesh et al., 2013). What it is known is that participants engage in MOOCs for several reasons such as to learn the MOOC content (Carr, 2013), to take advantage of its social aspect represented by the discussion forums (Ramesh et al., 2013), curiosity about the MOOC concept (Zheng, Rosson, Shih & Carroll, 2015), and professional development (Zheng et al., 2015).
Clow (2013) characterized the pattern of attrition in MOOCs as a ‘funnel of participation’, which includes students who are enrolling just to get familiar with a MOOC and see how it operates, students who are already familiar with MOOCs but don’t want the commitment of all work embedded in it, and a group of participants who enroll to fully take advantage of this free professional development. Instead of studying participants’ drop-out, the main purpose of the current study is to determine the effect of forum interaction and watching videos on participants’ completion in a MOOC designed for professional development of statistics teachers. Thus, this study aims to answer the following research questions: (I) What are the characteristics of participants taking a MOOC for professional development purposes? (II) What factors predict completion in MOOCs for professional development purposes? (III) Is MOOC completion associated with participants’ number of posts and number of videos watched?
Connectivism highlights the potential of connections among people and content within the network (Siemens, 2004). According to Kop and Hill (2008) connectivism “frames learning in terms of learners connecting to nodes on a network, suggesting that knowledge does not reside in one location, but rather that it is a confluence of information arising out of multiple individuals seeking inquiry related to a common interest and providing feedback to one another” (p. 04). Connectivism is used in this study as theoretical lens to understand participants’ engagement (represented by their connections with peers and with materials) in the MOOC. Downes (2010) suggests four key characteristics to achieve network learning in connectivism: autonomy, diversity, openness, and connectedness/interactivity.
Autonomy means that participants have “choice of where, when, how, with whom, and even, what, to learn” (Mackness, Mak & Williams, 2010, p. 266). In this sense, autonomy brings the notion of participants self-organization having flexibility and control over their learning process and the ability to choose how much and in what way(s) they will engage in the course. It is important to note that there are boundaries to autonomy, such as levels of expertise, personal styles of learning (individualist or groups), levels of fluency in the course language, etc. (Mackness et al., 2010).
Diversity means participants from different generations, different cultures, and different backgrounds. This allows them to gain knowledge from each other as an outcome of these varied perspectives and at the same time, stretching them beyond the typical boundaries of their comfort zones.
Openness can be seen as open access to the course and course materials (Mackness et al., 2010), meaning that participants are able to “freely enter and leave the system, and there ought to be a free flow of ideas and artifacts within the system” (Downes, 2010, para. 8). Openness also means one’s freedom in choosing to work in private or in groups, contributing or not contributing to the course.
Connectedness/Interactivity refers to connective opportunities for participants and are the elements that sustain learning in a connectivism environment. Mackness et al. (2010) contend that connectivity and interactivity can be afforded by choosing certain kinds of technology.
The context of this study is a MOOC for Educators (MOOC-Ed) offered by a large American university that has been specifically designed for teachers to learn about statistics teaching and the use of statistical investigations in teaching. According to Kleinman, Wolf and Frye (2013) the “MOOC-Ed explores a specific model designed to provide K–12 educators with self-directed, supported, flexible, yet structured learning opportunities” (p. 01).
This MOOC provided 12 discussion forums for participants, distributed as two discussion forums per unit (5 units) plus an introductory forum and a project forum. The MOOC made use of videos to introduce each course unit, and to show statistical simulations, students engaging with statistics, and use of statistical tools. To obtain a certificate of 20 hours of professional development, participants had to post at least once in each forum, and access and engage with materials by completing the tasks from two specific sections presented in each unit of the MOOC. These tasks comprise assessment of participants’ statistical knowledge, analysis and adaptation of statistical tasks, quizzes related to students performing statistical investigations, and self-assessment about their confidence in teaching statistics.
This study makes use of multiple sources of data collection with the intention of improving the quality of the research findings (Patton, 1990). Raw data was extracted from the Moodle Platform where the MOOC is hosted, de-identified (exclusion of participants’ names and use of participants ID numbers), and consolidated using the Perl programming language. The data components in this study were the total number of videos watched, the total number of discussion forum posts, and participants’ demographics all taken from the same MOOC offered in Fall of 2015.
The number of videos watched represents instances in which a participant clicked the play button of a video in this MOOC, including instances where participants clicked the play button for a particular video more than once. Although the variable of analysis is labeled as number of videos watched, there is an inherent limitation in that, since it is not possible to know with certainty if a participant is, in fact, cognitively engaged with the video being played or if this participant is sharing his/her attention with other activities besides this MOOC. By design, this study did not focus on participants engagement within videos such as pausing, fast-forward, and skipping video parts. The number of forum posts is comprised of participants posting new discussion threads and replying to each other’s posts. Demographics data is comprised of participant country, gender, education level background, job role, and professional experience. The demographics also captured participants’ weekly number of hours available to spend on this MOOC, if they received any incentives from their school or district to participate in this MOOC, and if they have engaged in other professional development related to preparing to teach statistics before. Each of these data sources provided the study with a specific type of information about the nature of participants’ interactions in this professional development. The purpose is to use multiple data sources joined together to generate evidence that will help in answering the following research questions:
When analyzing participant completion in online learning, one must consider that completion is related to a unique combination of course characteristics and participant profiles. This means that other MOOCs with similar course characteristics (videos and forums) may not produce the same degree of participants’ completion as seen in the MOOC analyzed in this study.
This section presents results of the analyses conducted to examine the characteristics of participants taking a MOOC for professional development purposes, the factors that predict completion, and an investigation of whether MOOC completion is associated with participants’ number of posts and number of videos watched. Afterwards, discussion and conclusion are presented. Statistical analyses were performed using the R statistical programming language and the R-Studio IDE.
Exploratory data analysis was used to build answers to this first research question, with participant demographics from a MOOC designed for teachers to learn about statistics teaching used as the data source. From 817 participants registered in this MOOC (n=817), 597 participants were from the United States, 68 were from New Zealand, 26 were from the U.K., 20 were from Australia, and 106 were from other countries (Table 1). Participants were distributed as 541 females and 276 males within a variety of education level backgrounds: 161 participants have a Doctoral Degree, 428 participants have a Masters Degree, 170 participants have a 4-Year College Degree, 13 participants have a 2-Year College Degree, 16 participants have a Professional Degree (e.g. JD, MD), and 29 participants have a High School Degree. Participants’ gender in this MOOC seems to follow a skewed distribution in teacher gender as described by the U.S. Department of Education National Center for Education Statistics (2016).
|Country||Number of Participants||Country||Number of Participants||Country||Number of Participants||Country||Number of Participants|
|United Arab Emirates||2||Greece||8||Malaysia||2||United Republic of Tanzania||2|
|Argentina||1||Hong Kong, SAR China||1||Nepal||2||United States of America||597|
|Australia||20||Honduras||1||Niue||1||British Virgin Islands||3|
|Germany||5||Islamic Republic of Iran||2||Puerto Rico||2|
|Dominican Republic||1||Italy||4||Palestinian Territory||1|
In terms of the roles these participants have in their jobs, Table 2 shows that most participants were Classroom Teachers (K-12 and Special Education teachers) followed by College Instructors (College Professor and Math/Stat College Professor). With respect to their professional experience, most participants had between 5 and 10 years of professional experience. When asked how many hours they expected to have available to spend on this MOOC per week, 421 participants stated intent to engage 1-2 hours, 305 participants stated intent to engage 3-4 hours, 71 participants stated intent to engage 5-6 hours, and 20 participants stated intent to engage more than 6 hours. 774 participants reported not being required to take this MOOC, while 43 stated their school or organization required their engagement in this professional development. The number of hours participants expected to have available for this MOOC was a categorical variable labeled as Time. 761 participants reported they haven’t received any incentives from their school or district to participate in this MOOC, while 56 participants have received incentives from their employers to take part in this MOOC. 436 participants stated they have never engaged in other professional development related to preparing to teach statistics, while 38 participants have participated in other forms of professional development to teach statistics.
|Participants’ job role||Job_role Code||Number of Participants|
|Classroom Teachers (K-12 and Special Education teachers)||Job_role0||410|
|Teacher Developers (Curriculum Specialist, Professional Development Consultant, Teacher Education, and Instructional Coaches)||Job_role1||109|
|College Instructors (Math/Stat College Professor, College Professor – Other)||Job_role2||115|
|College Student Graduates||Job_role3||25|
|College Student Undergraduates (Pre-Service teacher and College Student undergraduate)||Job_role4||50|
|Other (Statistician, Educational Product/Service Provider, and Instructional Technology Facilitator)||Job_role5||108|
Univariate logistic regression was used to gain knowledge about this second research question, which intends to identify important covariates (predictors) that are at least moderately associated with response. Participant demographics, total number of videos watched, and total number of forum posts were the data sources in respective analysis. Gender, Education Level, Experience, Job Role, Time, Number of Posts, Number of Videos, and Required were used as initial independent variables to predict participants’ probability of course completion as presented in Table 3. Results show that only Time (p = 0.02843), N_Posts_Made (p = 0.00000), and N_Videos_Watched (p = 0.00000) were statistically significant when considering participants’ probability of MOOC completion (Table 3).
|Variable Names||Pr(>|z|)||Variable Names||Pr(>|z|)|
Stepwise multiple binomial logistic regression was used to build answers to the third research question. This procedure “retests, at each stage, terms added at previous stages to see if they are still significant” (Agresti, 2002, p. 214). For a purposeful selection model, the process starts with univariate analysis of each variable as presented in previous section. Any variable presenting a significant univariate test (p-value smaller than 0.05) is selected as a candidate for the multiple binomial logistic analysis. In multiple binomial logistic analysis, an interactive process is used for variable selection. Parameter estimates are removed if they are not significant (p-value greater than 0.05). The process of deleting parameter estimates, refitting, and verifying is repeated until it appears that all important variables are included in the model. Akaike Information Criterion (AIC) (Akaike, 1973) is used to compare these non-nested models and decide upon the model that contains the best predictor subset. Since AIC represents the expected information loss, we are looking for a model with the lowest AIC, which means that the fitted values by that model are considered to be closer to the true values (Agresti, 2002). Further, interaction effects are explored by adding these respective interactions in the model and assessing the joint significance of these variables. Changes are made in the model by using a p-value cut-off point of 0.05. Again, AIC is used to compare the models and decide upon the model that contains the best predictor subset.
By fitting a multiple binomial logistic regression, previously significant estimation of independent terms in univariate binomial regressions may be reduced, and some independent variables may become insignificant. This happens since multiple binomial logistic regression asks about the relationship between the dependent variables and the independent variables, controlling for the other independent variables. At the end of the results section, Table 7 shows the models used in this procedure, as well their respective AICs.
Participant demographics, total number of videos watched, and total number of forum posts were the data sources for this analysis. Results from binomial logistic regression (Table 4), model 1, presented both N_Posts_Made (p = 2.78e-13) and Job_role2 (College Instructors, p = 0.00614) as statistically significant when considering participants’ MOOC completion (AIC = 208.34 and G2 = 319.35). Surprisingly, N_Videos_Watched (p > 0.05) was not statistically significant regarding to participants’ MOOC completion.
|(Intercept)||-3.927728||-12.276||< 2e-16 ***|
|Note: *p<0.1; **p<0.05; ***p<0.01|
|Null deviance||319.35 on 816 degrees of freedom|
|Residual deviance||192.34 on 809 degrees of freedom|
The 1.21 odds ratio for N_Posts_Made indicates that a one point increase in N_Posts_Made is associated with MOOC completion increasing by a multiplicative factor of 1.21 (Table 5). The odds of completing a MOOC for participants who work as college instructors (Job_role2) over the odds of completing a MOOC for participants who work as classroom K-12 teachers (Job_role0) is exp (-2.821467) = 0.060 (Table 5). This means that each one point increase in job role rank is associated with MOOC completion decreasing by a multiplicative factor of 0.60.
|Coefficients||Odds Ratio||Confidence Interval (2.5%, 97.5%)|
Logistic regression including interaction terms related to MOOC completion and their moderation effect associated to the number of posts made and participants’ job roles was explored in model 2 (Table 6). For the moderation effect of MOOC completion model 2 showed the best outcome when comparing results from other models (AIC = 201.59, see Table 7). By including the interaction in the model, previously significant estimation of independent terms were reduced, and some independent variables became insignificant. Only the interaction N_Posts_Made and participants’ Job_role2 (College Instructors) presented a statistically significant result of -0.158 (odds ratio = 0.85) which indicates that MOOC completion presents a moderating effect between number of posts made (participation) and participants’ professional activities. Table 7 presents all the previous models analyzed during the stepwise binomial logistic regression procedure. Model 2 presented the lowest AIC being the best fit among the analyzed models.
|(Intercept)||-4.2370||-10.496||< 2e-16 ***|
|N_Posts_Made: Job_role2||-0.1580||-3.195||0.0014 **|
|Note: *p<0.1; **p<0.05; ***p<0.01|
|Null deviance||319.35 on 816 degrees of freedom|
|Residual deviance||175.59 on 804 degrees of freedom|
|Model_A1<- MOOC_Completion~ N_Videos_Watched+ N_Posts_Made||212.38|
|Model_A2<- MOOC_Completion~ N_Videos_Watched+ N_Posts_Made+ Job_role+ Time+Required,||212.27|
|Model_A3<- MOOC_Completion~ N_Videos_Watched+ N_Posts_Made+ Job_role+Required+Time+Education_Level,||219.22|
|Model_A4<- MOOC_Completion~ N_Videos_Watched+ N_Posts_Made+ Job_role+Time+Education_Level,||217.86|
|Model_A5<- MOOC_Completion~ N_Videos_Watched+ N_Posts_Made+ Job_role+Required+Time,||212.27|
|Model_A6<- MOOC_Completion~ N_Videos_Watched+ N_Posts_Made+ Job_role+Time,||211.83|
|Model1<- MOOC_Completion~ N_Videos_Watched+ N_Posts_Made+ Job_role,||208.34|
|Model2<- MOOC_Completion~ N_Videos_Watched+N_Posts_Made+Job_role+ N_Posts_Made*Job_role,||201.59|
Results showed that the number of videos watched by participants is not significant in predicting the probability of MOOC completion. To better understand this fact, we first make use of the intrinsic characteristics of this MOOC as well as the knowledge established by the literature of videos in MOOCs. Different from other MOOCs offered by high volume platforms such as Coursera or edX, this MOOC was designed for statistics teachers. Therefore, it primarily attracts teachers as well other professionals of education such as teacher educators, pre-service teachers, and college students. By design, this MOOC did not made use of videos to stream lectures (videos focused on content subject), nor did it use interactive videos such as quizzes during a video stream, a common practice in MOOCs. Thus, videos in this MOOC comprise: (a) introductory videos made to introduce the topic covered by each course unit, (b) cartoon videos of students doing statistical simulations, (c) videos of students learning statistics with technology, and (d) videos of the main instructor interviewing senior statistics educators. The duration of the videos ranges from 1.5 to 21 minutes, with 7.5 minutes being the average.
The particular use of video in this MOOC may help in understanding the reasons video wasn’t a significant factor when investigating participants’ MOOC completion. The literature informs that only a small amount of participants who engage with videos complete the MOOC (Sinha et al., 2014). Additionally, passive engagement such as watching video or lectures doesn’t guarantee the best scenario for learning. Diversity of learning activities such as projects, tasks, quizzes, presentations, and discussion forum participation could be used in tandem to videos to maximize opportunities for student’s learning in MOOCs (Koedinger, Kim, Jia, McLaughlin & Bier, 2015). Connectivism allows us to acknowledge participants’ autonomy and their perceptions of educational value in videos, which can help us to understand that participants’ interactivity in MOOCs is, in fact, a product of their choices in shaping their own behavior which may or may not affect their course completion.
In regard to participants’ engagement in discussion forums, results are aligned with the branch of the literature in MOOCs in which forum participation supports participants’ course completion (Breslow et al., 2013; Kizilcec, Piech & Schneider, 2013). With regard to connectivism, participants’ interactivity in forums reinforces the idea that learning is a network formation process of connecting to specialized nodes (Siemens, 2004). Forums have the potential to amalgamate participants’ interactions with others and with course content within the network created by this MOOC. Although many may intuitively believe that participants’ interactivity with forums and with videos seems to be related to achievement, this study advances the field by showing how much these engagement types have the potential to affect (or not) student achievement.
Diversity of participants’ professional background is highlighted by connectivism as an important characteristic of massive open courses such as this MOOC. This can be seen in the fact that K-12 teachers, middle school teachers, college instructors, teacher educators, and pre-service teachers are all taking part in the same professional development. Regarding the impact that job role has on MOOC completion, results exposed that participants who work as college instructors have a lower probability of completing this professional development than do K-12 teachers. This outcome seems appropriate considering that K-12 teachers may be seeking this free professional development opportunity to gain knowledge about how to teach statistics to their students and perhaps to keep their teaching certification. In contrast, a college instructor may participate in this MOOC with the simple intention of updating his or her knowledge regarding content and/or statistical tools, and therefore would not have external pressure to complete the course. Thus, the lack of completion for participants who work as college instructors may not necessarily mean failure in the MOOC.
This study modeled participants’ MOOC completion based upon behavioral features such as number of posts and number of video watched (i.e., videos played). The resulting evidence extends the literature on videos in MOOCs by demonstrating that interaction with videos is not a significant contributor to the likelihood of a participant completing a MOOC for teachers’ professional development. Results add to the debate in MOOC forums literature indicating that engagement with discussion forums is still a very important component of participant learning as they create networks and interact with others. Additionally, the results extend the literature of MOOC completion showing that participants’ professional background acts as a moderator on completion, in which participants with more prosperous jobs tend to complete less of a MOOC for professional development purposes.
Given the important role of participation in forums and level of expertise, implications for practice suggest that MOOC designers can use the results of this study as a rationale to improve success and nurture of a virtual community by using a combination of activities that foster participants’ interactivity and active engagement such as group work, live interactive discussions via webcasts, projects for participants to implement with their students, and forums designed to foster expert-expert and expert-novice interactions. With the limited usefulness of video in MOOC completion, this study invites us to think about alternatives to make videos more attractive to MOOC participants. Results of this study can be used as a rationale by MOOC designers to implement the use of a video information guide where participants could browse video content before fully engaging with it, and to split a long video into smaller portions so participants could decide upon engaging with each portion. By making videos more interactive and becoming familiar with the results of this study designers can develop learning materials that will increase the likelihood of participants’ completion of MOOCs for teachers’ professional development.
When considering implications for teachers’ professional development, using MOOCs as a venue emerges as a viable method that can support teachers’ self-development in an affordable and widely available way. MOOCs for professional development provide opportunity for teachers to improve their skills in the subject area, re-examine their teaching practices, and make instructional use of new approaches or tools. The scalability and reach of a MOOC focused on preparing teachers to teach statistics is a very appealing option in closing the gap in teacher preparation regarding of knowledge in statistics concepts and teaching.
Regarding implications to research, this study sheds light on the notion that teachers’ interactions in this form of professional development are a product of their autonomy, meaning that they decide which videos to watch and which discussions to engage with. Thus, MOOC researchers need to be prepared to understand and explore the idea that MOOC comprises diverse ecologies of participation (Fischer, 2011) and one size fits all may not be the best approach to keep participants engaged with the course (Murugesan, Nobes & Wild, 2017). Results of this study indicate that MOOCs should create space and provide support for distinct participants’ roles based on their interests and levels of expertise. Additional studies are necessary to further understand and to model the factors that drive engagement with materials and other participants, and the factors that make participants more likely to quit or refuse engagement with others.
This study models participants’ MOOC completion by using two observable behavioral features: engagement with forums and the number of videos watched, and MOOC demographics. Because only the quantity of posts was used to measure engagement with discussion forums, understanding the full extent of cognitive engagement with the MOOC requires further study that takes the content of participants’ posts into consideration. Similarly, the impact of videos was measured using the total number of videos played by each participant throughout the full MOOC period. This variable does not incorporate nested characteristics of the videos such as video length, type of video (lecture, problem solving, simulation, etc.) leaving room for future work that investigates the implications of precisely when, which kind, and in what quantity videos are watched. As mentioned in the methods section, the number of videos played might not be a true representation of participants’ video watching since a participant can play the video and not cognitively engage with its content. This study also did not focus on participants engagement within videos such as pausing, fast-forwarding, and skipping video parts. Data retrieval did not track video views that were downloaded by participants to be watched offline. The study showed that participants’ job role matters when considering MOOC completion. However, results are contingent on participants’ honesty in providing accurate background information when registering for this MOOC (source of demographics data). Results from this study might not generalize to all MOOCs, since the study context was a MOOC for teachers in which participants might be more self-motivated learners. To improve external validity, it is recommended that these analyses should be replicated on different MOOCs for teachers’ professional development.
Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In B. N. Petrov & F. Caski (Eds.). Proceedings of the Second International Symposium on Information Theory (pp. 267–281). Budapest: Akademiai Kiado.
Anderson, T. (2013). Promise and/or peril: MOOCs and open and distance education. Commonwealth of Learning. Retrieved from http://www.col.org/SiteCollectionDocuments/MOOCsPromisePeril_Anderson.pdf
Breslow, L., Pritchard, D. E., DeBoer, J., Stump, G.S., Ho, A.D., & Seaton, D. T. (2013). Studying learning in the worldwide classroom: Research into edX’s first MOOC. Research & Practice in Assessment, 8, 13–25. Retrieved from http://www.rpajournal.com/dev/wp-content/uploads/2013/05/SF2.pdf
Carr, D. F. (2013). MOOC students attracted most by course topics. Retrieved from http://www.informationweek.com/software/mooc-students-attracted-most-by-coursetopics/d/d-id/1110976?
Clow, D. (2013). MOOCs and the funnel of participation. Paper presented at the LAK’13: 3rd International Conference on Learning Analytics & Knowledge, Leuven, Belgium. Retrieved from http://oro.open.ac.uk/36657/1/DougClow-LAK13-revised-submitted.pdf
Coetzee, D., Fox, A., Hearst, M. A., & Hartmann, B. (2014). Should your MOOC forum use a reputation system? In Proceeding of the 17th ACM conference on Computer supported cooperative work & social computing, 1176–1187.
Darabi, A., Arrastia, M. C., Nelson, D.W., Cornille, T., & Liang, X. (2011). Cognitive presence in asynchronous online learning: A comparison of four discussion strategies. Journal of Computer Assisted Learning, 27(3), 216–227. https://doi.org/10.1111/j.1365-2729.2010.00392.x
Downes, S. (2010). What is democracy in education? Retrieved from http://halfanhour.blogspot.com/2010/10/what-is-democracy-in-education.html
Glance, D. G., Forsey, M., & Riley, M. (2013). The pedagogical foundations of massive open online courses. First Monday, 18(5). http://dx.doi.org/10.5210/fm.v18i5.4350
Guo, P. J., Kim, J., & Rubin, R. (2014) How video production affects student engagement: An empirical study of MOOC videos. In Proceedings of the first ACM Learning@scale conference, ACM (2014), 41–50.
Gutiérrez-Rojas, I., Alario-Hoyos, C., Pérez-Sanagustín, M., Leony, D., & Delgado-Kloos, C. (2014). Scaffolding self-learning in MOOCs. In Cress, U. & Kloos, C. (Eds.). Proceedings of the European MOOC Stakeholder Summit 2014 (pp. 43–49). Lausanne, Switzerland: PAU Education. Retrieved from http://educate.gast.it.uc3m.es/wp-content/uploads/2014/02/Scaffolding_self-learning_in_MOOCs.pdf
Kanuka, H., & Anderson, T. (2007). Online social interchange, discord, and knowledge construction. International Journal of E-Learning & Distance Education, 13(1), 57–74. Retrieved from https://eric.ed.gov/?id=EJ584310
Kizilcec, R. F., Piech, C., & Schneider, E. (2013). Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. In Proceeding of the third international conference on Learning Analytics and Knowledge (pp. 170–179). ACM. Retrieved from http://web.stanford.edu/~cpiech/bio/papers/deconstructingDisengagement.pdf
Koedinger, K. R., Kim, J., Jia Z., McLaughlin E., & Bier, N. (2015). Learning is Not a Spectator Sport: Doing is Better than Watching for Learning from a MOOC. Learning at Scale ’15. Retrieved from http://pact.cs.cmu.edu/pubs/koedinger,%20Kim,%20Jia,%20McLaughlin,%20Bier%202015.pdf
Kleinman, G., Wolf, M. & Frye, D. (2013). The digital learning transition MOOC for educators: exploring a scalable approach to professional development [online]. Retrieved from http://all4ed.org/reports-factsheets/the-digital-learning-transition-mooc-for-educators-exploring-a-scalable-approach-to-professional-development/
Kop, R., & Hill, A. (2008). Connectivism: Learning theory of the future or vestige of the past? The International Review of Research in Open and Distance Learning, 9(3). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/523/1103
Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: A systematic study of the published literature 2008-2012. The International Review of Research in Open and Distance Learning, 14(3), 202–227. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1455/2531
Mackness, J., Mak, S. F. J., & Williams, R. (2010). The ideals and reality of participating in a MOOC. In Dirckinck-Holmfeld L, Hodgson V, Jones C, de Laat M, McConnell D & Ryberg T. (Eds.). Proceedings of the 7th International Conference on Networked Learning (pp. 266-274). Aarlborg. Retrieved from http://www.lancs.ac.uk/fss/organisations/netlc/past/nlc2010/abstracts/Mackness.html
Morris, N.P., & Lambe, J. (2014). Studying a MOOC: a guide. Palgrave MacMillan. Retrieved from https://he.palgrave.com/resources/Product-Page-Downloads/M/Morris-Studying-a-MOOC/Studying-a-MOOC-Neil-Morris-James-Lambe.pdf
Murugesan, R., Nobes, A., & Wild, J. (2017). A MOOC approach for training researchers in developing countries. Open Praxis, 9(1), 45–57. http://dx.doi.org/10.5944/openpraxis.9.1.476
North, S. M., Ronny R., & Max, M. N. (2014). To Adapt MOOCS, or Not? That is No Longer the Question. Universal Journal of Educational Research, 2(1), 69–72. Retrieved from https://eric.ed.gov/?id=EJ1054159
Ramesh, A., Goldwasser, D., Huang, B., Daume H. III, & Getoor, L. (2013). Modeling Learner Engagement in MOOCs using Probabilistic Soft Logic. In Workshop on Data Driven Education. In Neural Information Processing Systems (NIPS), 2013.
Rosé, C. P., Carlson, R., Yang, D., Wen, M., Resnick, L., Goldman, P., & Sherer, J. (2014, March). Social factors that contribute to attrition in MOOCS. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 197–198). ACM. Retrieved from http://www.cs.cmu.edu/~mwen/papers/p197-rose.pdf
Ruberg, L.F., Moore, D.M. & Taylor, C.D. (1996). Student Participation. Interaction, and Regulation in a Computer-Mediated Communication Environment: a Qualitative Study. Journal of Educational Computer Research, 14(3), 243–268. Retrieved from https://eric.ed.gov/?id=EJ537913
Shank, G. & Cunningham, D. (1996) Mediated Phosphor Dots. Toward a Post-Cartesian Model of Computer-Mediated Communication Via the Semiotic Superhighway. In C. Ess. (ed.). Philosophical Perspectives on Computer-Mediated Communication (pp. 27–41). State University of New York Press, Albany, NY.
Siemens, G. (2004). Connectivism: A Learning Theory for the Digital Age. Retrieved from http://www.elearnspace.org/Articles/connectivism.htm
Sinha, T., Jermann, P., Li, N., & Dillenbourg, P. (2014). Your click decides your fate: Inferring Information Processing and Attrition Behavior from MOOC Video Clickstream Interactions. Proceedings of the 2014 Empirical Methods in Natural Language Processing Workshop on Modeling Large Scale Social Interaction in Massively Open Online Courses, Qatar, October, 2014.
Thomas, M. J. (2002). Learning within incoherent structures: The space of online discussion forums, Journal of Computer Assisted Learning, 18(3), 351–366. https://doi.org/10.1046/j.0266-4909.2002.03800.x
Triay, J., Sancho-Vinuesa, T., Minguillón, J., & Daza, V. (2016). Analyzing non-linear video usage in an introductory x-MOOC about basic linear algebra. In LASI Spain 2016, 27–28 June 2016, Bilbao, Spain. Retrieved from http://repositorio.grial.eu/handle/grial/624
U.S. Department of Education, National Center for Education Statistics. (2016). Digest of Education Statistics, 2015 (NCES 2016-014), Chapter 2. Retrieved from https://nces.ed.gov/fastfacts/display.asp?id=55
Walker, B. K. (2007). Bridging the distance: How social interaction, presence, social presence, and sense of community influence student learning experiences in an online virtual environment ( unpublished doctoral dissertation). University of North Carolina.
Yang, D., Sinha T., Adamson D., & Rosé, C. P. (2013). Turn on, Tune in, Drop out: Anticipating student dropouts in Massive Open Online Courses. In NIPS Workshop on Data Driven Education. Retrieved from https://pdfs.semanticscholar.org/c134/1b4b24363dc09ba1659c448dcce8442697e6.pdf
Young, J. R. (2012, December 4). Providers of Free MOOC’s Now Charge Employers for Access to Student Data. Chronicle of Higher Education. Retrieved from http://chronicle.com/article/ProvidersofFreeMOOCsNow/136117
Zheng, S., Rosson, M. B., Shih, P. C., & Carroll, J. M. (2015). Understanding student motivation, behaviors, and perceptions in MOOCs. Proceedings of the ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW ‘15), 1882–1895. https://doi.org/10.1145/2675133.2675217