Framework for the Development of OER-based Learning Materials in ODL Environment

This paper describes the framework for the development of OER-based learning materials TCC121/05 Programming Fundamentals with Java for ODL learners in Wawasan Open University (WOU) using three main development phases mainly: creation, evaluation and production phases. The proposed framework has further been tested on ODL learners to promote greater use of OER repositories and further the acceptance of a wider range of learning resources. By using collective feedback sessions, the walkthrough of available OER resources to be integrated or assembled into the learning materials is illustrated in the course development cycle and the interaction among students and instructors and learning experiences of courses units’ content are evaluated at the end of each of the courses with the production of external course assessor’s reports (Quality Assurance). The learners’ interactions with OER materials in LMS have demonstrated the development of a model for effective knowledge transfer using OER.


Introduction
Open Educational Resources (OER) include full courses, course materials, modules, textbooks, streaming videos, tests or other tools that resides in the public domain with intellectual property license that permits free-use or repurpose by using attribution, no derivatives, share-alike, noncommercial or any combinations of these CC licenses to support access to knowledge (Atkins, Brown & Hammond, 2007).The development of digital materials movement has encouraged users to re-use, revise, remix and redistribute resources (Hilton III, Wiley, Stein & Johnson, 2010) through appropriate tools and made available through creative common licensing.
In preparation of the OER-based E-Learning course, workshops and seminars are held to develop capacity of participants for integration of OER in their own teaching and learning at Wawasan Open University (WOU).The OER-Asia initiative in 2010 under the auspices of the University for promoting and transforming the existing course development process is aim to increase the quality and efficiency of developing OER-based course materials (Menon & Ali, 2012).The OER Policy for WOU with implementation strategies are given as follows: "WOU will promote and implement the creation, reuse, remix, repurpose and redistribution of Open Educational Resources (OER) within an Open Licensing framework" The OER repositories are introduced to the course team members in development of the OER-based course for the initial development by Chung and Khor (2012), TCC121/05 Programming Fundamentals with Java who comprised of Course Team Coordinator, Academic Members, Course Writers, Instructional Designers, Editor and External Course Assessor (ECA) for searching, creating and customizing learning contents including content modules, articles, books and journals.TCC121/05 was scheduled for course development in July 2012 as an OER integrated course Open Praxis, vol. 5 issue 4, October-December 2013, pp. 315-324 adopting the WOU-Open License Policy (OLP) (Menon & Ali, 2012) and first presented in January 2013 semester.The course development for TCC121/05 has been conducted smoothly within six months duration and the course materials were made available to the students at its six regional learning centers via WawasanLearn-a LMS based on the open-source system called Moodle.With the experience and feedbacks acquired from four semesters' presentations, the development of TCC121/05 is conducted based on the students' and tutors' experience that have gone through the course.The course module together and OER contents provided in WawasanLearn formed an adequate support for students' learning.
According to McGreal et al. (2013), sharing OER can be a valuable way to disseminate knowledge, diversify teaching activities and gain new insights into other teaching methods for a particular subject.A number of technical issues relating to improving accessibility and usefulness of OER such as the use of open source software, increasing interoperability by using open standards and emerging technologies that affect the open educational resources movement are being investigated (OECD, 2007).Motivations for learners to participate in OER development are varied, and many struggle to engage with courses and keep motivated in the context of an online learning environment (Yuan & Powell, 2010).In order to overcome the aforementioned issues, the Collective Feedback Sessions are initiated through live virtual domain using WizIQ to play the important role in helping to increase and widen participation in accessing and reviewing OER repositories.Virtual sessions were initiated to improve and overcome issue of how to find the resources that are most relevant, adding enriched metadata to resources for classification of educational resources (MELT, 2010).
This paper presents the framework for the development of OER-based learning materials and the evaluation of the student achievement and the accessibility of the OER content in WawasanLearn.The first section overviews the course development phases including creation, evaluation and production phases.The next section describes the Quality Assurance (QA) implementation for the OER-based course development via Collective Feedback Sessions.The findings, surveys and evaluations are discussed in the third section.The final section describes the recommendation and conclusion.The details of the framework documents are illustrated in Table 1.

Quality Assurance via Collective Feedback Sessions
The assurance is made aware throughout course development phases associated with the OERbased course materials with the inclusion of course writer's guidelines, academic members' inputs, ECA's role and guidelines, course team reports, tutors' and students' feedbacks.TCC121/05 course team members are invited to attend Collective Feedback Sessions to ensure the review of each interim reports are taken place.Learners' feedbacks and peer review processes are incorporated in the course development cycle as one of the most used quality assurance processes in academia (OECD, 2007).
Learning environments that encourage active participation and dialogue session provide learners with opportunities to engage in a process of knowledge construction as they try to create meaning from new experiences (Jonassen, 1995).The discussions in the collective feedback sessions held during the course development focused on the "how to develop" and usage of OER particularly in Java programming related areas.Virtual Workshop Sessions engaged using the virtual domain via WizIQ are held with guidance and step-by-step OER creation sessions for the OER content,   • Minutes Meeting (Course Team Coordinator, Course Writers, Academic Members, Instructional Designers) • Minutes Meeting (Course Team Coordinator, Course Writers, Academic Members, Instructional Designers) Course Units write up • Minutes Meeting (Course Team Coordinator, Course Writers, Academic Members, Instructional Designers, Graphics Designers, Learning and Library Services Members) • Minutes Meeting (Course Team Coordinator, Course Writers, Academic Members, Instructional Designers) Completed Course Units (Recommendations, Actions Taken) intellectual property issues, resources that can be reuse, remix, repurpose and integrate can be accessed in the following sessions links as shown in Table 2.
The study of adapting and remixing individual modules into courses (example illustrated using CNX) is presented in Figure 2. The creation of two modules illustrated in Figure 2 covers Module 1: Object and Classes (with image containing Symbols of Flowcharts), Module 2: Intro to Object and Classes (Pseudocode for Java) and Module 3: Java Programming (Control Structures, Arrays).The examples for the modules created consist of self-contained metadata which allows the users to choose varieties of language used and the subject categories.The creation module enable users to import CNXML documents (Words files, OpenOffice files, LaTex, multimedia assets) and select specific elements of the module to edit.The three sample modules created through the Collective Feedback Sessions are:

Key Findings: Summary of Key Information of TCC121/05 Programming Fundamentals with Java
The summary of the key information of the students enrolled in undergraduate course TCC121/05 Programming Fundamentals with Java is presented in Table 3 below: In quantifying the participation rate and total time of access, redundant data from the log files of the activities in the LMS have been eliminated to ensure a true reflection of the learners' learning pattern with the online content.Table 3 shows the learners in OER-based TCC121/05 were higher in average age, with slightly more female learners and it was observed that there is an increase in the learners' participation rate and higher average frequency of activity over a semester.The progress in the higher participation rate of learners in the LMS is also an evident to the wider access to OER learning.The exchange of learning questions and experiences in LMS help learners to clarify any doubt the have with regards to OER repositories and helping to find the most relevant and highest quality of resources.

Key Findings: Patterns of Access in Online LMS WawasanLearn (views and posts)
The pattern of access and activity by the course participants are observed in two separate semesters as depicted in Figure 3 (OER) and Figure 4 (Non OER).It was observed that the average activity level for OER-based TCC121/05 was higher than non OER-based TCC121/05 throughout the semester.The learners enrolled in OER-based TCC121/05 are found to be more initiated in accessing the LMS for OER online resources while non OER-based learners' activity level is with a declining level towards the end of the semester.This indicates that the motivation for researching  and utilising open content among OER-based learners are high as shown in the activity level of Figure 3.The students found that the WawasanLearn is useful as it supports the learning and they found that the forum discussion is relevant to the course contents using OER resources.

Key Findings: Learners' Interaction with OER Content and Resources in LMS
The evaluation and feedback capability of the OER content and resources were investigated in terms of whether it permits one-way non-interactive or two-way interaction between learners-OER content to evaluate the outcome of learning arising from the interaction.Figure 5 shows the students were actively using the online discussion forums in WawasanLearn throughout the semester (58%) for knowledge-sharing activities.Learners in TCC121/05 used more than 50% of their time participating in the online discussion forums that allow asynchronous exchanges of ideas among peers and with their instructors.This indicates that they value online resources which allow for two-way communication and also suggests that they are getting acquainted with active involvement in the online environment.It is also noted that the participants seek to acquire knowledge for selfenhancement and to apply the knowledge gained in their daily work.Learners in TCC121/05 spent more time in accessing the OER course content (19%) as the material involves providing constructive feedback and detailed commentary on course content via critical thinking that leads to knowledge development.The interaction with OER course content demonstrates the knowledge and explains the essential concepts about the course through the entire semester.It was found that the learners preferred to view through the hyperlinks OER resources (13%) as compared to the E-Books and slides (10%).The viewing activity was usually influenced by the content of the WawasanLearn and the posting of message in the forum.In simple, the good use of the OER resources as additional supports in the WawasanLearn was able to attract students to view the LMS and was able to support the students' learning.

Key Findings: Continuous Assessments and Examination Score
Students performed satisfactory in the OCAS (Overall Continuous Assessment Score) during January 2013 (OER) with a mean score of 75.14% and a standard deviation (S.D) of 8.82 as shown Open Praxis, vol. 5 issue 4, October-December 2013, pp. 315-324 in Table 4 below.On the other hand, learners in July 2012 scored lower OCAS with a mean score of 68.84% and S.D 7.10.This reflects the learners tend to perform better in the January 2013 semester than July 2012 semester as the additional OER resources are perceived as a valuable learning resource to test learners' understanding and allow learners to be better prepared for the TMA and final examination.
The assessments of TCC121/05 comprises Tutor Marked Assignments (TMA), mini projects and problem-solving case studies which were given by the Course Coordinator and tutor in the LMS for more in depth Java learning.Mini projects in the TMA tested the students' programming skills, problem solving and analyzing skills, flow chart/diagram and apply major components in Java programming in producing solution design.
As observed in Table 5, the Overall Examination Score (OES) achieved is 54.18%, with a standard deviation of 12.56.The OER-based student performance on the OES is considered to be above average as compared with non OER TCC121/05 with a mean score of 50.98% and standard deviation 11.17.units subjected to changes in technology impacting the course.The findings have indicated that the development framework will speed up the creation of course materials (spans approximately 6 months) and eliminate the necessity of accompanying textbook to achieve long-term costeffectiveness in educational practice.The implementation strategy used in this study is targeted to promote and encourage collaboration in the creation and integration of OER courses within an Open Licensing framework.The above findings have also shown that TCC121/05 OER-based course learners are adapting well to the open course in online distance learning environment.In fact, the learners have shown great participation in, and utilization of, the open course content and OER online resources in the LMS.This initiative have also provided opportunities for ODL educators to share experience and encourage further learning of OER integration via virtual workshops in widening the participation of OER learning environment and spreading knowledge through ODL environment.

Figure 1
Figure 1 depicts the development phases (creation àevaluation àproduction) of OER-based learning materials in this approach by the initiation of the creation of Course Syllabus, Course Development Timeline and Course Blue Print by Course Team Coordinator and respective course team members.In the creation phase, course units write up are produced and reviewed by the instructional designer before being sent to ECA to be assessed through evaluation of ECA (SOP for Course Development, 2010).It is important to ensure that QA standards were maintained in the development of the course materials.Thus, internal and external quality assurances are employed into the evaluation and production phases.As shown below, TCC121/05 comprises of 5 units course write up, self-tests, activities, examples and unit practice exercises which are written based on the integration and assembly of OERs under a CC BY-SA 3.0 license.• Course Overview • Introduction • Unit 1 p Unit content Minutes meeting are created and associated with each interim reports, course blue print, course guide, units write up to document the suggestions made with regards to the course team report.

Figure 1 :
Figure 1: Development Phases of OER-based Learning Materials

Figure 5 :
Figure 5: Analysis from the activity reports of the learners' use of online resources in WawasanLearn