Educational Resources to College Credit : The Approaches of Saylor Academy

Over the past decade great progress has been made in improving the availability of Open Educational Resources (OER). However, one area in which OER has been deficient is in its ability to lead to college or university credit, something that many users of OER may desire. This article describes the work done by the Saylor Academy in amalgamating OER in such a format that college credit is more easily attainable. We describe not only the theory behind what Saylor has done, but also provide details about the initial stages of their program implementation within specific accredited institutions.


Introduction
Open Educational Resources (OER) has been a burgeoning field of study since 2002.Wiley, Bliss and McEwen (2013) provide a comprehensive review of the history, opportunities and challenges associated with OER.One of the overarching goals of OER was succinctly stated by Mike Smith, Director of the Hewlett Foundation Education Program, which provided much of the initial funding for work surrounding OER: At the heart of the open educational resources movement is the simple and powerful idea that the world's knowledge is a public good and that technology in general and the World Wide Web in particular provide an extraordinary opportunity for everyone to share, use, and reuse that knowledge (Smith & Casserly, 2006, p. 10).D'Antoni, (2009) outlines many of the challenges and opportunities of OER, including achieving Smith's stated goal of utilizing technology in order to grant learners greater access to educational materials.Much has been done to help bring this idea to pass.Carson, Kanchanaraksa, Gooding, Mulder and Schuwer (2012) illustrated how some of the benefits and opportunities that have accrued as a result of initiatives such as MIT's OpenCourseWare.Bliss, Robinson, Hilton and Wiley (2013) and Lindshield and Adhikari (2013) have shown that both students and teachers perceive OER to be comparable, and in some instances preferable to traditional texts.Some studies (e.g., Hilton, Gaudet, Clark, Robinson & Wiley, 2013) have shown that, when schools adopt OER, costs borne by the students decrease, and there simultaneously appears to be no negative impact on student learning.
Yet for learners seeking to study on their own, some of the hoped-for possibilities of OER may not have yet been realized.While OER are plentifully available on the Internet, they can often be difficult to locate (Brent, Gibbs & Gruszczynska, 2012).Moreover, even if discrete pieces of OER can be found, students seeking a comprehensive understanding of a specific topic may find it Open Praxis, vol. 6 issue 4, October-December 2014, pp. 365-374 difficult to collate different OER in order to learn the subject matter that they desire to master.In some instances, comprehensive resources that were once openly available (e.g., Flat World Knowledge textbooks) have been taken down from the company website and are more difficult to find online than they once were.
Perhaps a more significant challenge is that even if individuals are able to learn the content of a specific course, they have no way of proving to a potential employer or educational institution that they have obtained the necessary knowledge or skill.Thus one missing piece of the OER puzzle concerns accreditation.While various possibilities have been suggested, practical realities of providing credit for prior learning gained by utilizing OER has proven challenging (Conrad & McGreal, 2012).
Historically credit-by-exam is one manner in which individuals have been able to gain academic credit for prior work or studies.This is a topic that is not new and has been studied and discussed extensively (e.g., Kreplin, 1971;Post & Killian, 1993;Ebersole, 2010).Conrad, Mackintosh, McGreal, Murphy and Witthaus (2013) and Friesen and Wihak (2013) discuss the concept that students could utilize OER, including those available in Massively Open Online Courses (MOOCs), and receive various types of credentials (including, for example, badges).There are many opportunities for learning through MOOCs, such as Coursera, Udacity, and iversity; however, student course completions are relatively low (Heutte, Kaplan, Fenouillet, Caron & Rosselle, 2014).There are also challenges with OER to receive credit for prior learning.For example, McGreal, Conrad, Murphy, Wittthaus and Mackintosh (2014) point out that "current course articulation processes are not well suited to the recognition and credentialing of OER learning" (p.130).The premise of this paper is that OER can be combined with credit-by-exam in such a manner that students can learn material for free online (by utilizing OER) and then obtain credit for their learning by taking an exam.
Specifically, in the United States, Excelsior and Thomas Edison are both accredited colleges that offer a variety of courses in which a student can take a fee-based challenge exam for college credit linked to specific courses (such credits are also typically transferrable to universities.)The concept of specifically tying OER to credit-by-exam, proffered to students by Excelsior, is still in its infancy (Center for Educational Measurement, 2012).However, in some cases it is even possible for students to earn a degree entirely through this type of credit-by-exam process.For instance, Excelsior College offers enough general education and business credit-by-exam options that students can earn Associates (2 year) or Bachelors (4 year) degrees in General Business or Liberal Studies.Because Excelsior's exams can be taken at testing centers around the world, this degree option is also available to international students.Thus a student, anywhere in the world, could earn an Associates or Bachelors degree at a U.S. accredited university at a relatively low cost.Our purpose in this paper is to describe how Saylor Academy has aggregated discrete pieces of OER into courses and aligned those courses with various opportunities for students to receive credit.We will discuss both the conceptual ideas of how obtaining college credit in this manner occurs and also examine the early results stemming from this process.

The Saylor Academy and its Courses
The Saylor Academy, established through a trust funded by Michael Saylor, has been focused on its open education initiative since 2008.The original goal of the Saylor Academy was to harness the capabilities of technology to drive the cost of a college education to zero.The primary mechanism for accomplishing this objective was to develop openly licensed, learning outcomedriven courses comprised of open access and other OER content 1 .In this regard, Saylor acted as an aggregator and distributor of discrete open resources that it curated into full OER courses.From Open Educational Resources to College Credit: The Approaches of Saylor Academy Open Praxis, vol. 6 issue 4, October-December 2014, pp. 365-374 One of the challenges that students face when using OER is that while many OER exist, locating high-quality resources that connect with specific learning outcomes in a particular course is not easy.While efforts have been made to assess the quality of OER (e.g., Bethard, Wetzer, Butcher, Martin & Sumner, 2009;Custard & Sumner, 2005), such efforts often center on individual resources, rather than a holistic review of materials broad enough such that they could be employed to represent an entire course.Learners seeking to use OER for formal learning face the double challenge of needing to both assemble resources and assess their quality, while learning unfamiliar, college level material.Saylor addressed this issue by reviewing and gathering existing OER in order to supply meaningful learning opportunities at a course level.Thus a student seeking to learn the material generally taught in Psychology 101 could take Saylor's Psychology 101 course and find comprehensive OER that teach all of the relevant material rather than having to search for individual pieces of OER throughout the Internet.
Initially Saylor identified and developed 241 courses that would fulfill program requirements for the ten highest enrolling majors in the United States (Saylor's course offerings were eventually expanded to include additional higher education areas of study, as well as K-12 and professional development).The first Saylor course, College Composition I (ENGL001), went live on Saylor.org in September of 2010.In order to build its courses, Saylor developed a rigorous course development and review process, based on the ADDIE model (Molenda, 2003).The primary course developers are credentialed faculty with subject-area teaching experience; they work alongside Saylor editors, instructional designers, and OER experts.As part of the development process, the courses were compared with existing syllabi and resources to create Saylor courses that match established teaching approaches.
Once a general framework had been established, faculty members working on the course developed learning outcomes and identified OER that could be used to effectively help students achieve these outcomes.In general, Saylor was able to utilize existing open content; however they occasionally commissioned creation of additional OER content when suitable materials were not already available.For example, in some instances faculty would create short essays, lectures, or other learning objects.Once a course had been developed in its first iteration, it was peer-reviewed by additional faculty members with content expertise.Feedback received through the peer-review process was then integrated into the course, and the resulting courses are the collaborative effort of several consulting faculty.
In addition to the incidental Saylor-commissioned OER learning objects, faculty created OER assessments, including final exams, for all courses.Upon passing the course final exam, students earn a Saylor Certificate of Completion, which students can use to demonstrate competency to employers or outside institutions.However, as Saylor is not an accredited or degree-granting institution, the certificate is not generally accepted for transfer credit.Saylor students have occasionally self-reported that they were able to bypass course or program prerequisites by presenting their Saylor Certificate of Completion, but absent partnerships between Saylor and the accepting schools, it is difficult to accurately gauge the frequency of such acceptance.Additionally, although the exam is completely free, there is no identity verification component; students login to their Saylor accounts to take the exam directly.Thus, while the certificate does provide students with an artifact of their learning, it has not yet had widespread or sustainable impact on leveraging OER into credit.

Opportunities to Test for Credit
As stated previously, the goal of the Saylor Academy is to increase access to and ultimately lower the cost of education.While providing free online content is indeed a necessary condition, it is Open Praxis, vol.6 issue 4, October-December 2014, pp.365-374 likely not sufficient, as students often seek validated, institutionally backed credentials, which are often necessary to enter or advance in traditional institutions of postsecondary education and find employment.Indeed, an analysis of Saylor Academy's existing user base suggests that students are interested in having their Saylor coursework recognized by other higher education institutions.From the voluntary self-reporting provided by approximately 26,000 Saylor users, 14% reported that interest in earning college credit contributed to their decision to take a course with Saylor Academy.Focusing on self-reporting done by approximately 1,800 students who have earned at least one Saylor Certificate of Completion, interest increases, with 23% of U.S. based students and 16% of international students indicating an interest in earning credit for their Saylor coursework.Of the U.S. based students who have passed a Saylor course, 58% have otherwise earned, at most, a college certificate or Associates degree, and 42% have not yet earned any post-secondary degree.For international students, these numbers are 37% and 26%, respectively.Thus, while most students did not join Saylor specifically to earn credit, many students could benefit if more credit earning options did exist.In order for students to utilize OER and receive credit-by-exam, Saylor pursued three variations on the challenge exam model.

Model I: Credit Recommendation for Saylor Direct Assessment
The first variation Saylor employed to align credit with Saylor's OER courses was an independent course and exam review and subsequent credit recommendation.Saylor identified selected courses and submitted them for review by the National College Credit Recommendation Service (NCCRS).The function of NCCRS is "evaluating training and education programs offered outside of the traditional college classroom setting and translating them into college credit equivalencies" (NCCRS, 2013, n.p.).Over 1,500 colleges and universities recognize evaluations and credit recommendations made by NCCRS.
In order to be approved by NCCRS, a rigorous review of course materials and assessments is required.NCCRS-partner faculty with college-level teaching experience in the specified course complete the NCCRS course reviews.These faculty members reviewed Saylor courses' learning outcomes, resources, assessments, and final exams, considering whether the constituent pieces and the courses holistically were commensurate with comparable courses taught at traditional colleges and universities.In order to maintain course integrity, NCCRS required that Saylor create new final exams for each course, which, in contrast to the finals that are a part of the Saylor course, would not be available for students to independently access online.Rather, students may only take the password protected NCCRS-reviewed exams through a proctor.Typically the cost of taking a proctored NCCRS examination is twenty-five United States dollars.
To date, Saylor has completed two rounds of NCCRS review.In November 2012, Business Law and Ethics (BUS205), Corporate Communication (BUS210), and Introduction to Western Political Thought (POLSC201) were each recommended for 3 credit hours (note that in the United States, an Associates Degree typically requires approximately 60 credit hours, and a Bachelors degree requires 120 credit hours).In November 2013, Principles of Marketing (BUS203), Business Statistics (BUS204), Principles of Management (BUS208), Introduction to Computer Science (CS101), and Beginning Algebra (MA001) were each recommended for 3 credit hours.Additionally in 2013, Saylor's Calculus I (MA005) was recommended for 4 credit hours.Following the first round of NCCRS review in 2012, Saylor began pursuing credit transfer partnerships and has formal agreements with eleven accredited institutions 2 to guarantee credit transfer.

Model II: Align Saylor Courses to Non-Saylor Credit-by-Exam Options
The second version of Saylor's OER-to-credit model is to fully align courses to existing creditby-exam opportunities.Traditionally, students who earn credit-by-exam have enough prior subject From Open Educational Resources to College Credit: The Approaches of Saylor Academy Open Praxis, vol.6 issue 4, October-December 2014, pp.365-374 knowledge through informal or non-transferrable learning that enrolling in and paying for the equivalent full-length college course is unnecessary.In contrast to requiring prior knowledge, Saylor courses allow students to engage in formal learning, but still achieve the cost and time savings of the credit-by-exam opportunities.
Saylor has partnered with Thomas Edison State College and Excelsior College by aligning courses to existing challenge exams offered through Thomas Edison State College Examination Program (TECEP), and UExcel, each college's respective credit-by-exam program 3 .Students who pass challenge exams through these credit-by-exam programs receive credit for the associated course directly from the college, which can then be used towards earning a degree.
Similar to the process undertaken for NCCRS course review, Saylor begins with an existing Saylor course and completes necessary updates to align the course to the corresponding credit-bearing exam.The primary focus of these updates is ensuring that the course adequately covers exam materials by conducting a comparative analysis of the learning outcomes in the Saylor course and the exam.Where gaps are found, OER are employed to align the course more completely to the external exam.For example, in the case of Single-Variable Calculus I (MA101), a comparative learning outcome analysis revealed that the UExcel exam covered a few objectives that were not included in the original Saylor course.In order to fill these gaps, OER were added to the course to ensure adequate coverage of the tested objectives.
The first Saylor Academy course to be aligned to an Excelsior College UExcel exam was launched in January 2013.To date, Saylor has developed courses to match three Excelsior College Exams: Principles of Macroeconomics (ECON102-EXC), Single Variable Calculus I (MA101-EXC), and Introduction to Psychology (PSYCH101-EXC).Additionally, Saylor also has two courses that were realigned to meet the specifications of Thomas Edison State College TECEP Exams.These courses, Computer Communications and Networks (CS402), and Introduction to Statistics (MA121), were launched in June 2013 and February 2014, respectively.The result of these exam alignments is that students who learn all of the material in the Saylor course are prepared to successfully pass the UExcel or TECEP exam.Thus students can learn for free, take a test for approximately $100, and receive three units of course credit that might have cost $1,000 at another institution.

Model III: New Non-Saylor Credit-by-Exam Aligned to Saylor Course
In the case just described, Saylor worked to align its course offerings to existing challenge exams.However, under the third variation of the Saylor credit-by-OER model, an institution can develop an exam that aligns to an existing Saylor course.Following such a model, Thomas Edison State College has now developed four exams based on Saylor courses.Working together, Saylor and Thomas Edison identified Saylor courses in subjects that did not yet have existing opportunities for credit-by-exam 4 .
Once the courses were selected, Thomas Edison created brand new TECEP exams based on the learning outcomes in the existing free Saylor Academy course, rather than matching them to one of the College's tuition based courses.When the exams were complete, Saylor's Introduction to Comparative Politics (POLSC221), World History in Modern and Early Modern Eras (HIST103), Negotiations and Conflict Management (BUS403), and Environmental Ethics, Justice, and World Views (ENVS203) courses were immediately linked to new and affordable college credit opportunities.This variation of OER to credit is a significant milestone because through this model, the existing Saylor OER courses served as the impetus for the development of a new challenge exam.
Open Praxis, vol. 6 issue 4, October-December 2014, pp. 365-374 Thus far we have discussed the content (the Saylor course) and method of assessment (a challenge exam) that are used to award college level credit for OER.An important aspect of assessment for credit pertains to identity verification.Each of the methods described previously have different methods of ensuring identity verification.As mentioned previously, the Saylor Certificate of Achievement is not proctored and therefore is a less-secure method of verifiable assessment.
For Saylor's NCCRS exams, students may have their exam attempts facilitated by ProctorU (an online proctoring service) or by a Saylor Academy vetted in-person proctor.In the case of ProctorU, students may sit for an exam in their own homes, using their own computers and webcams.Webcams are used to monitor student activities during exam attempts and to match a student's physical appearance to an official form of ID.Additionally, students are asked to answer a series of personal questions generated from publicly available data.The requirement for a proctored exam plays a key role in helping to protect the integrity of the exam and the resulting credentials being issued.Remote proctoring options such as ProctorU closely align to the values of OER and asynchronous learning by allowing the flexibility for individual students to be tested at times that work for them-an important consideration for the types of learners currently prone to seeking out non-traditional educational opportunities such as Saylor Academy.
In the case of Excelsior, exams are administered through a partnership with Pearson VUE, a network of testing centers in 175 countries that provides identity verification and exam proctoring services.For Thomas Edison's TECEP exams, students may take the exams on their own computers through ProctorU.In both instances the price of taking the exam (approximately $100) includes the cost of proctoring services.

Results
The results of this OER-to-credit model, although still in their early stages, are promising.As of September 2014, 4,320 students have passed a total of 8,332 Saylor Academy final exams and earned certificates of completion.Within the group of nearly 1,800 of these students who have self-reported demographic information, 25.5% are between the ages of 19-24; 32% are between 25-34; and 18.5% are between 35-44.Students are based in 123 different countries, but nearly 50% of users are from the U.S. India has the next highest percentage of users at 8% (these data are for students who have completed any Saylor exams, not just for specifically credit-aligned exams; Saylor Academy does not yet have detailed demographic information for students earning credit through Saylor courses. To date, the Saylor Academy has created 22 courses aligned to credit-by-exam options. 5Of these 22, nine have associated Saylor-developed NCCRS exams administrated through a proctor.Thus far, students have made 118 attempts at NCCRS credit recommended exams, with 69 passing scores (58% pass rate).The most popular of these courses has been Principles of Marketing (BUS203).Of the students who took and passed an NCCRS recommended exam, 24 have already had official transcripts sent to one of Saylor's partner schools, totaling 115 hours of recognized transfer credit, at a cost to students of approximately $900 (all costs being for the exam proctoring).This amounts to $7.82 per credit hour; in contrast, the cost per credit hour at most public two-year colleges is approximately $100.00 (Baum, Little & Payea, 2011).
For the remaining thirteen courses, getting accurate data is not as simple.Four of these courses are aligned to third-party assessments administered by organizations with which Saylor does not yet have formal working partnerships (e.g., The College Board's AP tests).Because of this, Saylor Academy does not receive data about how many times the aligned assessments are taken and From Open Educational Resources to College Credit: The Approaches of Saylor Academy Open Praxis, vol.6 issue 4, October-December 2014, pp.365-374 passed, and can only rely on student-reported feedback to learn whether they perceive Saylor courses were helpful preparation for earning credit via these exams.When students do self-report, their anecdotal evidence suggests that Saylor OER courses provide appropriate preparation to pass these third-party exams.For example, a student wrote to Saylor explaining, "I earned 9 credits by using Saylor courses (Macro/Microeconomics & Financial Accounting)  It is interesting to note that of the three courses employed to help this student earn credit, only one had been redesigned by Saylor to be specifically aligned to a credit-by-exam option.For the other two, the student simply used the existing version of the Saylor course, which had not yet been mapped to the exam, and was still able to pass.Such an anecdote suggests that the initial course design process appropriately targets standard college level learning in foundational courses.
For the Saylor courses aligned to UExcel or TECEP exams, the partner institutions collect data on whether students arrived at their websites via Saylor.org.These data, however, likely underrepresent the number of students who took a UExcel or TECEP exam after preparing with Saylor courses.This is because students are not required to disclose their manner of preparing for the exam and it remains unclear what percentage of students who take UExcel or TECEP exams utilized Saylor resources.To date however, there have been 31 instances in which students have taken a UExcel exam after specifically using one of the tracking links from a Saylor course.Twenty-six of these attempts resulted in a passed exam, and significant student savings in the costs associated with earning the credits.

Discussion
The results thus far have been limited, partly due to the fact that this proof-of-concept has only been a reality for about 18 months.While that has been enough time to increase student interset in affordable credit opportunities, more widespread acceptance of OER and credit-by-exam needs to take place before it can have a large-scale impact.Nevertheless, the fact that some students have successfully taken Saylor Academy courses and then passed examinations that provide them with college credit indicate that this method of utilizing OER to receive transfer credit is feasible.Each time a student uses a free OER-based course to prepare for a cost effective credit-by-exam, that student reaps meaningful financial benefit.One student wrote to Saylor that "The fact that the courses are free and online was a huge draw for me.One of the biggest reasons that I'm pursuing distance learning is because the costs of attending a brick-and-mortar school are just too much for me.Since these courses are offered at no charge, that was a huge savings for me.The only costs that I incurred with these [NCCRS approved] courses was paying the proctor fee at my testing center which was only $20 each time." Saylor's experience with OER-to-Credit suggests that while college-level OER is plentiful, aggregating OER into meaningful test prep materials is a significant undertaking.However, when sequenced into full-length courses and aligned to credit-bearing exams, OER can be used to successfully earn credit recognition.OER-committed organizations can continue to explore what Anderson and McGreal (2012) have called the "Unbundling of educational services" (p.391) by establishing a system whereby responsibility for the assessment rests with the credit-granting institution, but creation and maintenance of the courses occurs elsewhere.While Saylor has demonstrated that it can provide students with a very low-cost route to college credit, to have the broadest effect, the OER-community will have to make meaningful inroads in the existing system Open Praxis, vol.6 issue 4, October-December 2014, pp.365-374 of higher education.More schools will need to embrace competency-based education, and develop (and then publicize and encourage use of) testing-for credit opportunities.Christensen (1997) outlines the concept of disruptive innovation.A disruptive innovation is a product or technology that enters a given market and serves an audience that is different than the established market.These products generally are not used by the established market for various reasons, but they are appealing to emerging markets because they offer a different set of values, such as price or convenience.These emerging products will eventually increase in their quality and their values will begin to appeal to a larger audience.It is at this point that the original technology is disrupted and often becomes obsolete.

Conclusion
Christensen, Horn and Johnson ( 2008) outline a variety of disruptive possibilities with respect to education.We believe that route we have discussed in this paper with studying materials on Saylor.org leading to students passing assessments and receiving course credit potentially represents a disruptive innovation in education.While some students will continue to prefer more traditional routes to credit, the potential costs and inconveniences may lead an increasing number of students to attempt to self-study OER and receive credit by examination.However, there may be limitations to the extent to which the opportunities described in this paper could be considered disruptive innovations.As stated previously, Saylor Academy is run through a private trust, and as such does not need to maintain profitability in the way that businesses traditionally must.Nevertheless, as the available OER continue to improve, and processes for examinations for credit are streamlined, there is significant potential for an increasing number of students to take advantage of this option.If the numbers become sufficiently large, opportunities may arise for alternative methods of funding, allowing this to become a sustaining innovation.

Acknowledgment
This study was funded in part with a grant from the Bill and Melinda Gates Foundation.

Notes 1
There is a necessary distinction here between Saylor courses, which are OER, and the resources of those courses, which are always open access, but not always openly licensed.Saylor courses are comprised of the course learning outcomes, instructional prose, and overall course structure and approach, and the resulting course itself is OER.However, the distinct resources used to achieve those outcomes and populate the course structure are not always OER.

3
In addition, Saylor offers courses aligned to Advanced Placement (AP), College Board's College-Level Examination Program (CLEP), and StraighterLine credit recommended exams.However, Saylor developed the AP and CLEP aligned courses without a formal working relationship with the College Board, and thus the aligned courses have not been vetted by the respective exam providers.Saylor does have a formal partnership with StraighterLine; however, StraighterLine itself cannot issue degrees or credit directly.Rather, its exams carry credit recommendations similar to Saylor's NCCRS exams.
to prep for the CLEP/DSST test ["CLEP" is an acronym for "College Level Examination Program" and "DSST" refers to tests offered by the United States Department of Defense.]I'm currently in the process right now of using Saylor courses (Principles of Finance & Business Ethics) to prep for my last 2 DSST tests."

2
These institutions are as follows: Bellevue University; Bethel University, College of Adult & Professional Studies; Charter Oak State College; Colorado Technical University; CUNY Baccalaureate for Unique and Interdisciplinary Studies; Excelsior College; Granite State College, Great Bay Community College; Paul Smith's College; Thomas Edison State College; and University of Maryland University College.