Formalising informal learning : Assessment and accreditation challenges within disaggregated systems

This report shares the findings and lessons learned from an investigation into the economics of dis aggregated models for assessing and accrediting informal learners undertaking post secondary education. It presents some key economic and governance challenges for universities to consider in implementing OER assessment and accreditation policies. It also includes discussion of findings from a small-scale survey conducted by two of the authors on perceptions, practices and policies relating to openness in assessment and accreditation in post secondary institutions, with a particular focus on the OER universitas (OERu) concept.


Introduction
The concept of "openness" is arguably the most persistent and controversial educational innovation of recent years, provoking the potential for important change in post secondary education, worldwide.The two key concepts contained within the movement are Massive Open Online Courses (MOOCs) and Open Educational Resources (OER).Following an explanation of these important phenomena, this paper explores the key issues of assessment and accreditation that are raised by the notion of openness in learning.It is based on a report that examines potential models to address the learner assessment, certification and accreditation issues for learners participating in MOOCs as well as in other formal and informal learning contexts using OER at the higher education level (Conrad, Mackintosh, McGreal, Murphy & Witthaus, 2013).We focus here on the organisation and mandate of the Open Education Resource universitas (OERu), a global consortium of post secondary institutions whose collaboration around the assessment and accreditation of learners studying online and using open educational resources (OER) facilitates greater access and flexible learning paths for learners.
MOOCs have experienced a rapid growth in interest, often negative, since their inception in 2008.While permitting access to the same course for large numbers of learners in many countries, the Open Praxis, vol.6 issue 2, April-June 2014, pp. 125-133 introduction of MOOCS has given rise to the need to address issues of learner assessment and accreditation.Learners who access digital learning content via the Internet and acquire knowledge and skills either formally or informally, alone or in groups, cannot readily have their learning assessed, and are consequently unable to receive appropriate academic recognition for their efforts (Taylor, 2011;Mackintosh, McGreal & Taylor, 2011).This critical issue is the sticking point for many educators and as such has attracted attention both from institutions leading the OER movement and from its critics (Phelan, 2012;Yuan, MacNeill & Kraan, 2008;Olcott, 2012).
OER have provoked similar controversy, and institutional participation in the development and use of OER has been patchy across the globe.In many parts of the world, few institutions indicate that they either produce or use OER, and in the regions where production and usage of OER are higher, there is little coordination between institutions and generally not much governmental support to sustain OER activities.(See, for example, the early findings from the POERUP project, Bacsich et al., 2013.)Even fewer institutions have implemented open courses for assessment and accreditation (Conrad et al., 2013).In spite of resistance, the open movement continues to expand as evidenced by the growth of online OER repositories and the ongoing swirl of attention to MOOCs as well as the establishment of collaborative initiatives based on the use and reuse of OER.

The Study
In 2012, two of the authors of this paper developed a quantitative research survey to investigate the perceptions, practices and policies of post secondary institutions worldwide, toward openness in assessment and accreditation.A sample of 110 individuals representing 83 education institutions was obtained over a three-month period from June to August 2012.The survey was programmed and hosted online, with invitations to participate disseminated through social media channels and discussion forums aimed at educators interested in Open Educational Resources (OER).Invitations were also emailed to individual representatives of higher education organisations that were known to be engaged with OER assessment and accreditation initiatives such as the OER universitas initiative (previously known as the OER university).

The Open Education Resource universitas (OERu)
The OERu is a collaborative partnership currently comprising 30 partner institutions.The network includes both nationally accredited universities, colleges and polytechnics and publicly-funded organisations (Mackintosh, Taylor & McGreal, 2011).OERu's current geographic spread of institutions includes Australia, New Zealand, Canada, the USA, England, Ireland, Spain, South Africa, India and the South Pacific.The initiative is coordinated by the OER Foundation, which is an independent, not-for-profit organisation that works internationally to support the mainstream adoption of OER into the formal education sector (Mackintosh, 2012).Formalising informal learning Open Praxis, vol.6 issue 2, April-June 2014, pp.125-133 Mackintosh, Taylor and McGreal (2011) described the aim of the OERu as providing free education to learners worldwide using OER as learning materials, thus providing pathways to enable learners to gain credible qualifications from government-recognised or accredited educational institutions.The OERu concept is rooted in the notion of community service and outreach, and institutions that are members of the OERu network have committed to developing a "parallel learning universe" that will augment and add value to traditional delivery systems in post secondary education.The ultimate vision of the OERu is to provide free learning opportunities on a massive scale for learners who lack the financial means to access traditional higher education (Conrad et al., 2013).
One of the cornerstones of the OERu philosophy is that the components of higher education that are traditionally packaged together in a single institution can be disaggregated and provided by different institutions.Anderson and McGreal (2012) suggest that discount service models may become attractive, suggesting that the Open Educational Resources universitas initiative could support disaggregation as a disruptive model.Murray and Friesen (2011) explain that in traditional education models, learners enrol with a single institution and expect that institution to provide the teaching, the content, the assessment and the eventual accreditation.In the OERu's disaggregated model, "the basic elements of education, traditionally conceived, are redefined as placeholders and are opened up to substitution and disaggregation.Any student can study any content, supported in any number of instructional arrangements" (Murray & Friesen, 2011, p. 4).This "disaggregation" distinguishes the OERu model from other open learning models such as MOOCs and gives rise to the discussion of assessment and accreditation potential that follows.

Accreditation and assessment in post secondary education: Issues and approaches
The assessment of learning and its resultant accreditation toward a credential presents a major hurdle to the integration of open learning with formal learning (Conrad, 2013;Friesen & Wihak, 2013) and provided the raison d'être to the study that backgrounds this paper.
Academic assessment remains, universally, the privilege and purview of individual post secondary systems.While several jurisdictions have developed and implemented national accreditation frameworks, notably the UK and Australia, international accreditation and assessment services are not currently available.It is the premise of this paper that developing a robust system that can service thousands or even hundreds of thousands of learners internationally would change the dynamic of access to post secondary education for learners.The necessary systems technology is already available within post secondary institutions and would remain within their control: payment systems, content management systems, automated examination applications, and online invigilation.
There are two main types of assessment or accreditation relatively common at the post secondary level of education outside of traditional classroom-based assessment: the Recognition of Prior Learning (RPL) and post secondary credit transfer, where formal credentials obtained at another institution are transferred into learners' programs at their current or home institution.RPL is often referred to as Accreditation of Prior and Experiential Learning (APEL), Prior Learning Assessment (PLA, primarily in the USA), and Prior Learning Assessment and Recognition (PLAR, primarily in Canada) as well as several other terms.
RPL processes exist at post secondary institutions in many countries.Research shows that the use of learning portfolios in RPL is very popular, followed by examinations that allow learners to challenge-for-credit through assignments, examinations, interviews, courses, tutorials, demonstrations, self-assessment, external evaluations, essays, face-to-face or online workshops, and a variety of other instruments (Conrad & McGreal, 2012).Proficiency and knowledge acquisition can be demonstrated through the use of a single assessment methodology or a combination of the Open Praxis, vol.6 issue 2, April-June 2014, pp.125-133 above.The least frequently used and available (though it could possibly be the most effective from the perspective of learners) is the option to write a challenge examination or engage in some other form of challenge-for-credit.Other methods of RPL are much more resource intensive, requiring staff dedicated to spending significant time with prospective students.RPL-supportive institutions have developed resources and structures that cover most aspects of accreditation and assessment, including policy, research, repositories and experience with licensing.
Credit transfer refers to the willingness of institutions to grant credit to students who have taken courses at other institutions.While easier and less labour-intensive than RPL, credit transfer-in those institutions that permit it-is sometimes problematic for students.In North America, with its standard three credits for a one-semester course, credit transfer is easier to implement and therefore reasonably common, especially in the first two years of post secondary education.However, this is not the case in jurisdictions outside of the USA and Canada, where many institutions are reluctant to accept transfer credits and the majority of students are restricted to taking all of their courses at one institution.
The MOOC phenomenon has opened up interest in the possibility of alternative assessment and although many institutions around the world are considering these alternatives, breaking down institutional silos continues to present a major hurdle in the "cottage industry" of post secondary education, a hurdle that must be cleared before large-scale OER-based courses can be put in place.Using RPL for assessment could possibly offer a solution (Camilleri & Tannhäuser, 2012;Conrad, 2013).
In a scalable, open environment, the necessary unbundling of services to separate assessment and accreditation from teaching and institutional support can be much easier using OER-based courses rather than commercial content.Scalability is problematic for initiatives that rely on commercial content that is restricted by technological protection measures and restrictive licensing.Because of the ease of copying, adapting and otherwise reusing OER, OER-based initiatives can be scaled up and made freely available in different jurisdictions and institutions.

Learning the hard lessons: Issues around and barriers to implementing OER
Murphy and Witthaus' (2012) findings uncovered issues and barriers to the implementation of OER in post secondary institutions.The study also suggested that the notion of disaggregating traditional university services, while providing a useful conceptual framework for considering assessment and credentialing alternatives in open learning environments, has yet to be fully understood by the wider higher education community and has yet to be operationalised in practice at anything approaching a meaningful scale.
Study findings suggest that there is sufficient evidence to justify the unbundling of traditional services in order to provide more affordable access to post secondary education and formal academic credit.A small number of institutions are already utilising these opportunities as part of their existing delivery models.For example, the University of South Wales is already implementing RPL on a larger scale than most UK universities and is planning to expand the scope of its accreditation activities in partnership with other institutions in the OERu (Witthaus, 2013).Even if we consider the commitment of only the 30 OERu member institutions to establishing a "world OER credit bank" and "specifying what credit they are willing to accord those who successfully complete the learning outcomes associated with [the OER]" (OERu, 2011, p. 20) it is reasonable to assume that the inventory of full programmes of study that will be available under an assessment-only model will grow in the coming years.For example, in 2012, Otago Polytechnic announced that the new Graduate Diploma in Tertiary Education approved by the New Zealand Qualifications Authority will be based entirely on OER and will also cater to assessment-only options for prospective learners (Otago Polytechnic, 2012).Also in 2012, the South African Ministry of Higher Education and Training stated that "collaborative development of high quality learning resources made available as Open Educational Resources (OER) provides the possibility both for increased access and quality and lower unit costs due to reduced duplication and greater usage."(DHET, 2012).Additionally, the DHET (Department of Higher Education and Training) has more recently declared its intention to work "toward creating a post-school distance education landscape based on open learning principles" (DHET, 2013).
Mainstream assessment approaches that are used for summative assessment and credentialing in traditional models, such as examinations, tests, and, increasingly, portfolios, can be reused effectively within a disaggregated system.Challenge-for-credit examinations present an alternative within disaggregated systems that enable learners to prove mastery of a set of learning outcomes by sitting a challenge exam at a reduced fee compared to the full-tuition price of a course (Conrad et al., 2013).And while portfolio assessment processes can accredit knowledge gained from prior experience toward learners' credentials, the labour intensive nature of this process may be less suitable for reuse within disaggregated systems.The study did not identify any material policy barriers for assessment and credentialing that would curtail the implementation of a disaggregated model, although in the UK there is a perception that the Quality Assurance Agency's (QAA) strict rules for "collaborative provision" could make it difficult for UK institutions to operate in this way (Bird & Witthaus, 2012, p. 3).Elsewhere in the world, it appears that the majority of post secondary institutions would be able to implement assessment-only models within existing policy frameworks.However, minor adaptations and refinements to operational management and processes may be required.For example, student administration and processing of payments within a disaggregated model would require the accommodation of individual payment-per-assignment submitted for assessment.Such a system would also create a labour-intensive process that may be problematic within traditional models that require payment by learners prior to the recording of credit on university transcripts.
The practice of providing free online learning opportunities and corresponding solutions for assessment, certification and credentialing services will continue to evolve.Consider, for example, that the American Council of Education (ACE) has begun a project to explore whether credit recommendations from the Council can provide a viable pathway for accreditation of MOOC learning (Fain, 2012).This forms part of a $3 million investment by the Bill and Melinda Gates Foundation in several MOOC-related investigations including the Association of Public and Land Grant Universities (APLU) that are designed to create an interactive learning consortium to study the potential of MOOCs for public community colleges and universities and the Ithaka Strategic Consulting and Research group that will work with the University System of Maryland to test and study the use of MOOCs across their system (Fain, 2012).
Moreover, following the adoption of the 2012 Paris OER Declaration at the UNESCO OER World Congress, governments of member states of the United Nations will be encouraged to adopt policies that require teaching and learning materials produced from public funding to be released under open content licenses (UNESCO, 2012).This action may foster new policy imperatives for publicly funded universities and colleges to encourage them to diversify formal assessment and credentialing protocols for their citizens using OER courses as they can provide cost-effective pathways for widening access to post secondary education.
Although research on OER initiatives is contributing new knowledge for both researchers and practitioners, barriers to forward-movement in "openness" were also identified in the Murphy and Witthaus study (2012).The following section discusses barriers encountered and lessons learned to date.
Open Praxis, vol. 6 issue 2, April-June 2014, pp. 125-133 Current institutional processes for international credit transfer and course articulation are idiosyncratic, usually dependent on institutional policy, with a lack of standardisation when working across regional and national borders (Conrad & McGreal, 2012).As such, current course articulation processes are not well suited to the recognition and credentialing of OER learning.This is potentially the most significant policy barrier for scalable implementations of assessment and accreditation for official recognition of learning on a large and global scale.Current institutional processes often result in unnecessary duplication and inefficiencies that hinder the cost-effective implementation of assessment and credentialing, which could be avoided through carefully structured collaboration around OER (see, for example, the case studies described by Lane, 2012).
Other significant barriers to OER identified by faculty and administration include fear of change, confusion over copyright issues and the use and reuse of OER, concerns regarding the effort required for implementation of OER initiatives, and the possibility of conflict with commercial publishers and other special interest groups (Murphy & Witthaus, 2012).Perceived barriers to the kind of collaboration proposed by the OERu are also numerous, and include concerns about the role of national quality assurance bodies, a belief that students might not get sufficient support, concerns about the "true" cost of collaboration around OER (in financial terms), and some scepticism around the philanthropic motives of the OERu (Bird & Witthaus, 2012, p. 14).However, the greatest bar riers to participation in open assessment and accreditation practices are the lack of availability of committed staff members to support such activities and the potential costs of redeveloping courses as OER (Murphy & Witthaus, 2012).Lack of support for OER-based courses from senior management is also a substantial concern.
These "barriers" can be countered by incentives such as the low cost of entry and use of OER; minimal or non-existent licensing requirements; the ability to localise and update the content and make other changes without restriction; greater acceptance by students; organisational leadership by educators; and the potential to increase the institution's ability to serve a greater number of international students and raise the profile of the institution in the global higher education community.
Within institutions, key factors for the success of open assessment and accreditation implementation appear to be a reliance on a strong base of support within the institution-both in terms of leadership and resources-and an existing culture of openness that includes policies and practices around the creation and use of OER (Murphy & Witthaus, 2012).Policies that enable either open access or recognition of prior learning via credit transfer or RPL are also important.Institutions that already have these features in place are likely to be in the best position to implement assessment and accreditation of OER-based learning services, and, as such, could provide models for other organisations that would like to participate in collaborative open education, assessment and credentialisation initiatives in the future.

Considerations for future research
Several areas for future research become obvious when the barriers and hurdles facing further implementation of OER are considered: Establishing cross-border systems of articulation and transfer.As outlined above, the current state of idiosyncratic course articulation processes does not foster credentialing processes within disaggregated systems for OER learning.Further research is required to explore alternatives for cross-border credit transfer and course articulation in an open and disaggregated manner.As institutional accreditation is closely associated with quality assurance mechanisms, this research will need to consider corresponding implications for quality assurance processes in a disaggregated system.Formalising informal learning Open Praxis, vol.6 issue 2, April-June 2014, pp. 125-133 Applying RPL processes to a disaggregated model of learning.The majority of OERu institutions and practitioners consider RPL methods and approaches to be the main vehicle for the assessment and credentialing OER learning in a disaggregated model (Murphy & Witthaus, 2012).Although a preliminary analysis of the costs to students for RPL are expensive when compared to other credentialing alternatives, for example, automated assessment and course assessment packages, these alternative methods should not be considered as substitutes for RPL because portfolio assessment offers meaningful opportunities for the assessment and accreditation of learning acquired outside of conventional course delivery (Camilleri & Tannhäuser, 2012;Conrad, 2013;Friesen & Wihak, 2013).While for many learners, RPL is the only option available to gain formal recognition of their learning, there may be specific RPL processes which could be reused, repurposed and repackaged to augment and support assessment and credentialing processes for OER learning in a disaggregated system.To investigate these possibilities, further research focusing on the following areas of RPL is recommended: • Activity-based costing analysis of RPL processes to identify cost-behaviours of appropriate RPL activities; • Investigation of possible solutions for cost-effective and scalable RPL processes when working with large numbers of learners; and • Analysis of alternatives for packaging RPL assessment and credentialing processes and corresponding pricing for sustainable operations in a disaggregated system.

Conclusion
Academic boards and senates at many universities are reluctant to reuse open-licensed courses and their corresponding assessments, even though those materials have been formally approved by another accredited university and even though these open courses can be adapted locally at no cost and offered in parallel with existing courses in order to diversify curriculum at the home institution.This reticence may prove to be short-sighted and poor business strategy.As they so often do -think Coursera-the commercial sector will be quick to appreciate the business value of building assessment and credentialing strategies from assets which do not require any upfront investment.And governments hard-pressed to reduce fiscal deficits may consider alternatives for a more cost-effective post secondary sector that recognises the benefits of favouring the disaggregation of traditional university services.The formal post secondary sector has a unique opportunity to take a leadership role in determining its own futures because the token esteem of a university credential continues to be highly valued by both society and economy, as Brown and Duguid (1996) outlined: in our highly commodified society it is naïve to believe that access on its own is enough.Those who have the label but not the experience present one problem.But those who might have the experience but not the label face another.Experience without a formal representation has very limited exchange value-as those whose only degree is from the university of life well know.(p.10) Universities, as institutions, are understood to have the requisite knowledge and experience and have long enjoyed the trust of society in the process of accrediting the formal learning of those seeking formal academic credentials.The disaggregation of assessment and credentialing services for OER learning can provide a viable pathway for more affordable access to post secondary education and formal academic recognition while simultaneously continuing to serve universities' core academic missions of disseminating knowledge and engaging in community service.