Measures of student success with textbook transformations : the Affordable Learning Georgia Initiative

In 2014, the state of Georgia’s budget supported a University System of Georgia (USG) initiative: Affordable Learning Georgia (ALG). The initiative was implemented via Textbook Transformation Grants, which provided grants to USG faculty, libraries and librarians, and institutions to “transform their use of textbooks and other learning materials into using lower cost options”, in other words to use open educational resources (OER) in lieu of a traditional bound textbook. The Round One Textbook Transformation Grants have already shown to be successful in that they saved students approximately $760,000. What is not known, is the collective impact on student learning. This study examines the learning gains or losses preand post-transformation in ALG Round One courses where traditional resources were replaced with OER. It estimates differences between preand posttextbook transformation across the following outcomes: 1) Drop Fail Withdraw (DFW) rates, 2) rates of completion, 3) numbers of students receiving a final grade of A or B, C and D, 4) numerical final grades as a percent, 5) final exam grades as a percent, and, 6) course-specific assessment grades measured in percent. Twenty-four data sets were analyzed for DFW rate, eight data sets for completion rate, fourteen data sets for grade distribution, three data sets for final exam grades, three data sets for course specific assessment and one data set for final grades. The null hypothesis that there would be no differences between preand post-transformation rates in these learning outcomes was supported. Thus, this study demonstrates that the USG’s ALG initiative helped students save money without negatively impacting learning outcomes. In addition, it is the first of its kind to measure some of these learning outcomes (e.g. final exam grade, assessment grade, and distribution of letter grades) at this scale.


Introduction
Successful teaching involves many components such as the knowledge and capabilities of both teachers and students, as well as curriculum materials and other available resources (Charalambous & Hill, 2012).One of the key pieces of curriculum materials in post-secondary education is the textbook (Altbach, Kelly, Petrie & Weis, 1991).Textbooks synthesize information on a particular subject, making them an invaluable reference for any curriculum.Although textbooks are valuable learning resources, their costs have risen dramatically, in some instances making them costprohibitive for many students.Hilton, Robinson, Wiley and Ackerman (2014) found that the average textbook price across seven colleges and multiple general education classes was $90.00.Furthermore, during the 2015-2016 academic year, textbook and supplies costs for a college student ranged from $1,249-$1,364 (College Board, 2016).In addition, research done by the National Association for College Stores (NACS) show that average "new" textbook prices have increased steadily since the 2009-2010 academic year from $62 to $82 (NACS, 2016).
The perceived high cost of textbooks combined with other costs of higher education may negatively impact students from lower socioeconomic backgrounds (Paulsen & St. John, 2002).For example, those with lower incomes are more prone to delay college enrollment than their wealthier peers Open Praxis, vol. 9 issue 1, January-March 2017, pp.93-108 (Provasnik & Planty, 2008).High costs, which include textbook costs, can also result in students taking fewer classes, delaying graduation (Buczynski, 2007).Moreover, many students do not purchase textbooks, which weakens their learning opportunities.One survey suggested that 23% of students regularly forego purchasing required textbooks due to their high cost (Florida Virtual Campus, 2012).One method to circumvent the high cost of textbooks is to replace commercial textbooks with Open Educational Resources (OER).
The following definition of OER was offered by Saul Fisher from the Andrew W. Mellon Foundation in 2002 at the Forum on the Impact of Open Courseware for Higher Education in Developing Countries convened by UNESCO: "The open provision of educational resources, enabled by information and communication technologies, for consultation, use and adaptation by a community of users for non-commercial purposes" (UNESCO, 2002, p. 24).According to the William and Flora Hewlett Foundation, OER can include full courses, course materials, modules, textbooks, streaming videos, tests, software, and any other tools, materials, or techniques used to support access to knowledge (Hewlett, 2013).OER eschews traditional copyright in lieu of licenses that allow others to retain, reuse, revise, remix, and redistribute the materials (Hilton, Wiley, Stein & Johnson, 2010;Wiley, Bliss & McEwen, 2014).
By far the simplest way to implement OER in a college course is to replace the traditional textbook with an "open" textbook.There are scores of high-quality open textbooks available for students and faculty to freely use (Open Textbook Library, 2016;OpenStax, 2016) many of which go through professional peer review and publishing processes.Furthermore, many open textbooks are available in print in addition to being online.Many students prefer to purchase a printed copy regardless of whether the online version is free (Hilton & Wiley 2011).Printed versions of open textbooks cost substantially less than traditional textbooks.
The biggest concerns that faculty have concerning the adoption of OER are: 1.Whether the open resource is of similar quality to the traditional resource and, 2. How students will perform utilizing open resources (Allen & Seaman, 2014).To address the first concern, Bliss, Hilton, Wiley and Thanos (2013) surveyed the experiences of fifty-eight teachers and 490 students across eight colleges in their utilization of open texts.Bliss et al. (2013) found that approximately 50% of students said that the OER textbooks were of the same quality as traditional textbooks and nearly 40% said that they were better.Additionally, 55% of teachers adopting OER reported that the open materials were of the same quality as the materials they had previously used, and 35% felt that they were better.A recent study by Allen & Seaman (2014) found that of 2,144 surveyed college professors, 34% were aware of OER and of that 34%, 61.5% indicated OER had about the same "trusted quality" as traditional resources, 26.3% said that traditional resources were superior, and 12.1% said that OER were superior.Similarly, 68.2% said that the "proven efficacy" was about the same, 16.5% said that OER had superior efficacy and 15.3% said that traditional resources had superior efficacy.Hilton (2016) examined an additional eight studies of perceptions of OER in higher education and found similar results, namely that a strong majority of teachers who had adopted OER felt that they were as good or better than commercial resources.Based on these studies, on average OER appears to be of similar quality to traditional texts.
To address how students will perform utilizing open resources several studies have examined how using OER influences student performance measures.Lovett, Meyer and Thille (2008) measured the efficacy of an OER statistics module in comparison with the traditional educational model at Carnegie Mellon University during fall 2005 and spring 2006 semesters.Their results showed no significant difference between test scores (three midterms and one final exam) of students utilizing OER vs. the students in the traditional class.Bowen, Chingos, Lack and Nygren (2014) also compared the use of a traditional textbook in a face-to-face lecture class with that of a blended Measures of student success with textbook transformations: the Affordable Learning Georgia Initiative Open Praxis, vol. 9 issue 1, January-March 2017, pp. 93-108 approach utilizing OER at Carnegie Mellon's open statistics module.Bowen and colleagues found that, while students who utilized OER scored slightly higher than their peers on standardized exams, the difference was not statistically significant.Allen, Guzman-Alvarez, Molinaro and Larsen (2015) studied 478 students that used ChemWiki, an OER, for its primary textbook, and 448 students who utilized a commercial textbook.Pre-tests, combined with final exams showed no significant differences in individual learning gains between the two groups.These studies show that utilizing OER not only results in cost-savings but do not sacrifice student learning outcomes.
A few studies have shown that student learning increased in OER classes in comparison to a course that used traditional resources.First, Pawlyshyn, Braddlee, Casper and Miller (2013) found that when OER material was integrated into the math courses at Mercy College, student learning significantly increased.The pass rates of math courses increased from 63.6% in fall 2011 (when traditional learning materials were employed) to 68.9% in fall 2012 when all courses were taught with OER.Similarly, students who were enrolled in OER versions of a reading course performed better than their peers who enrolled in the same course using non-OER materials.Second, Hilton & Laman (2012) compared the performance of 690 students using an open textbook in an introductory psychology class to the performance of 370 students who used a traditional textbook in a previous semester.They concluded that students who used the open textbook achieved better grades, had a lower withdrawal rate, and scored better on the final examination.Lastly, Feldstein et al. (2012) found that students in courses using open textbooks typically had higher grades and lower failure and withdrawal rates than those in courses with traditional textbooks.However, it is important to note that the authors pointed out significant limitations in the two latter studies and stressed that these results were not generalizable.Given this, there isn't enough information to universally say that OER will unequivocally increase student learning gains.
In contrast, in one instance, OER were found to be associated with lower outcomes.Robinson (2015) examined OER adoption at seven different institutions of higher education.In the 2012-2013 academic year, 3,254 students across the seven institutions enrolled in experimental versions of eight different courses that utilized OER and 10,819 enrolled in the equivalent versions of the course that utilized traditional textbooks.Robinson (2015) found that there were no statistically significant differences between the two groups in terms of final grades or completion rates in five of the eight classes.However, students in two courses performed significantly worse, receiving one-half to a full grade lower than their peers.Students in one class were significantly more likely to complete the course, although there were no statistically significant differences between groups in the overall course grades.Across all classes there was a small but statistically significant difference between the two groups in terms of the number of credits they took, with students enrolled in OER versions of the course taking on average .25 credits more than their counterparts in the control group.This study demonstrates the confounding factors that need to be taken into account when specific measures of performance are analyzed.
Hilton (2016) synthesized the above studies, as well as some additional ones and found that when students use OER in their classes, student outcomes are the same or better than when a traditional textbook is used.While these results are collectively interesting, they are far from comprehensive.Given the paucity of studies that have measured student performance using OER, much more research needs to be done to determine what relationship (if any) exists between the use of OER and student performance in higher education.In addition, performance measures like distribution of letter grades or performance on course specific assessments are an important part of general course assessment.These types of data are specifically lacking in the OER literature and should be included in evaluations of OER efficacy.The purpose of the present study is to add to the body of literature by examining the effectiveness of several OER adoptions that occurred in connection Open Praxis, vol. 9 issue 1, January-March 2017, pp.93-108 with Affordable Learning Georgia.As described in further detail below, I examined the results of 4950 students across 36 classes in 18 universities.My specific research question was as follows: is using OER associated with a change in student learning outcomes?

Context of the Present Study
In 2014, the state of Georgia decided to include funding in the state budget to support a University System of Georgia (USG) initiative: Affordable Learning Georgia (ALG).ALG's focus was on reducing costs of textbooks and enhancing GALILEO (GeorgiA LIbrary LEarning Online), and Georgia's Virtual Library.The initiative was implemented via Textbook Transformation Grants, which provided grants to USG faculty, libraries and librarians, and institutions to "transform their use of textbooks and other learning materials into using lower cost options" (Affordable Learning Georgia, 2016).ALG's Textbook Transformation Grants program has a three-fold objective: 1. Pilot different approaches in USG courses for textbook transformation including adoption, adaptation, and creation of Open Educational Resources (OER) and/or identification and adoption of materials already available in GALILEO and USG libraries, 2. Provide support to faculty, libraries, and their institutions to implement these approaches, and, 3. Lower the cost of college for students and contribute to their retention, progression, and graduation.
Two levels of funding are available for award.The Standard-Scale Transformation includes transformation of one or more courses with less than 500 students enrolled on average per academic year, funding a maximum of $10,800.The Large-Scale Transformation involves one or more courses/ sections or department-wide adoptions involving 500 or more students enrolled on average per academic year and funds a maximum of $30,000.Proposals can be submitted to one of four categories: No-or-Low-Cost-to Students Learning Materials, OpenStax Textbooks, Interactive Course-Authoring Tools and Software approach, (replaced the Course Pack Pilots category available in Rounds 1 and 2) and, the Top 100 Undergraduate Courses.Proposals to be submitted for funding through the Textbook Transformation Grants need to follow certain guidelines and certain activities are required to receive full funding (see https://affordablelearninggeorgia.org for more information on proposal submission and information about submission categories).To date, there have been eight calls for proposals for textbook transformation, with the latest call addressing courses for FA 2017.
The Round One Textbook Transformation Grants have already shown to be successful in that they saved students approximately $760,000 (Affordable Learning Georgia, 2015).What is not known, however, is the collective impact on student learning.Yes, students saved $760,000-but did they obtain positive learning outcomes?In the present study I will examine the learning gains or losses pre-and post-transformation in ALG Round One Grantees.

Methods
ALG's Round One call for proposals for Textbook Transformation Grants yielded the funding of 29 proposals encompassing 36 courses set to take place during the Spring 2015 academic semester.The types of data that were reported across projects varied.For example, some reported qualitative data and some reported quantitative data or both were reported.Quantitative data reporting consisted of 1) Drop Fail Withdraw (DFW) rates, 2) rates of completion, 3) numbers of students receiving a final grade of A or B, C and D, 4) numerical final grades as a percent, 5) final exam grades as a percent, and, 6) course-specific assessment grades measured in percent.These measures were not consistently reported across all groups.Some groups provided pre-transformation values for these measures.Not all groups reported qualitative data, but those that did, collected those data via surveys, focus groups or student quotes.Survey questions were not consistent across groups.Measures of student success with textbook transformations: the Affordable Learning Georgia Initiative Open Praxis, vol. 9 issue 1, January-March 2017, pp.93-108 In terms of quantitative data, one project did not report any quantitative data and seven projects did not make comparisons to pre-transformation data, leaving 21 projects and 27 courses viable for pre/post-transformation data analysis (Table 1).In each of the 21 projects, a faculty member (or members) created or utilized pre-existing OER to substitute for the traditional resources they used in previous semesters.For example, faculty members replaced the traditional bound textbook with a complete open source book (online textbook or ebook) or with individual subject-specific opensource documents or used subject specific websites.The goal of the transformation grants was to replace the costly resources used with a free version(s) and not to transform content or learning activities.Each faculty member taught both the pre-and post-transformation course and supplied the data for comparison.The information gathered provided paired data sets for analysis without instructor bias within each set.
In terms of qualitative data reported, all projects reported three quotes from their students for their respective projects.Twenty of the 29 projects provided results from surveys administered for their project.Survey questions varied, but those of interest for the present study included, whether students thought the quality of the text was comparable to a traditional textbook and whether they thought their learning experiences were enhanced.

Data Analysis
I estimated differences between pre-and post-textbook transformation across the following outcomes: 1) DFW rates, 2) rates of completion, 3) numbers of students receiving a final grade of A or B, C, and D, 4) numerical final grades as a percent, 5) final exam grades as a percent, and, 6) course-specific assessment grades measured in percent.Since the data accumulated from this study comes from different populations of students, it is necessary to check whether each data set conforms to a normal distribution (or not) to direct subsequent statistical analyses.Additionally, it is possible that variance across projects is not homogeneous (Glass, 1966).Equality of variances should be assured prior to performing statistical tests.A Levene test (Levene, 1960;Brown & Forsythe, 1974) was used to check for equality of variances.A Shapiro-Wilk test was used to check for data normality (Shapiro & Wilk, 1965).This test was chosen over other tests of normality because it works best with smaller sample sizes and has shown to be more powerful than other similar statistical tests (Razali & Wah, 2011).If the Shapiro-Wilk test shows that the data are normally distributed, parametric statistics can be performed and if the data is not normally distributed nonparametric statistics are performed.Since the data is paired (i.e.pre-and post-data), a paired t-test (data normally distributed) or a Wilcoxon signed-rank test (data not normally distributed; Wilcoxon, Katti and Wilcox (1970) will be used to test the null hypothesis that there is no difference between pre-and post-transformation student learning outcomes.Lastly, the Bonferronni correction was applied to the paired tests to adjust for multiple comparisons and control for Type I errors.

Results
I measured results based on the data that was provided by the individual reports.Sixteen projects reported information on DFW rates, seven projects reported completion rate data, seven projects reported grade distribution data, three projects reported final exam grades, three projects reported course specific assessment data and one project reported final grade data.In sum, there are twentyfour data sets for DFW rate, eight data sets for completion rate, fourteen data sets for grade distribution, three data sets reported for final exam grades, three data sets reported for course specific assessment and one data set reported for final grades.The Shapiro Wilk test indicated that most of the data was normally distributed (α = 0.05), although the paired data for completion rate and distribution of D grades was not at that same alpha level (Table 10).That being said, the Levene's test indicated that there was equality of variance across all the data (α = 0.05; Table 10).
As a result, a paired t-test was performed for all analyses except for completion rate and distribution of D grades, in which a Wilcoxon signed rank test was performed.All analyses, parametric and non-parametric alike, was not significant after Bonferronni correction (α = 0.008; Table 10).Hence, the null hypothesis that there was no difference pre-and post-transformation was supported.
Pre-and post-transformation data sets compiled for DFW rate resulted in the analysis of 24 courses/sections of courses (Table 2).Twenty-four data sets were included affecting 3133 students.DFW rate was provided for paired courses/sections of courses.Inspection of the data showed some individual variation from course to course.For example, some courses showed changes in DFW rate in favor of pre-transformation (N=11) and others showed changes in favor of post-transformation (N=12).In one case, there was no change (N=1).A Shapiro Wilk test indicated that the data was normally distributed and a Levene's test indicated that there was equality of variance across the data (α = 0.05; Table 10).A paired t-test showed that the results were not statistically significant (α = 0.008; Table 10).Hence, the null hypothesis that there was no difference pre-and posttransformation was supported.Pre-and post-transformation data sets compiled for completion rate resulted in the analysis of eight courses/sections of courses (Table 3).Eight data sets were included affecting 329 students.Completion rate was provided for paired courses/sections of courses.These data also showed some individual variation from course to course.Two courses showed changes in completion rate in favor of pre-transformation and four courses showed changes in favor of post-transformation.In two cases, there was no change.A Shapiro Wilk test indicated that the data was not normally distributed (α = 0.05; Table 10) but a Levene's test indicated that there was equality of variance across the data (α = 0.05; Table 10), so a Wilcoxon signed rank test was performed and was found to be not significant (α = 0.008; Table 10).Hence, the null hypothesis that there was no difference pre-and post-transformation was supported.Pre-and post-transformation data sets compiled for grade distribution rate resulted in the analysis of 14 courses/sections for A/B grades (Table 4, affecting 828 students), 12 courses/sections for C grades (Table 5, affecting 733 students), and eight courses/sections for D grades (Table 6, affecting 403 students).Grade distribution data was provided for paired courses/sections of course in each table.Variation from course to course was evident.For A/B grades, six courses showed changes in favor of pre-transformation, seven courses showed changes in favor of post-transformation and there was no change in two courses.For C grades, seven courses showed changes in favor of pre-transformation, four courses showed changes in favor of post-transformation and there was no change in one course.For D grades, three courses showed changes in favor of pre-transformation, three courses showed changes in favor of post-transformation and there was no change in two courses.Separate Shapiro Wilk tests implemented for A/Bs, Cs and Ds indicated that the paired data for A/Bs and Cs was normally distributed but that the paired data for Ds was not (α = 0.05; Table 10).However, separate Levene's tests for numbers of A/Bs, Cs and Ds all indicated that there was equality of variance across the data (α = 0.05; Table 10).As a result, paired t-tests were performed for A/Bs and Cs and a Wilcoxon signed rank test was performed for Ds.All tests were found to be not significant (α = 0.008; Table 10).Hence, the null hypothesis that there was no difference pre-and post-transformation was supported.Pre-and post-transformation data sets compiled for final exam grade rate (Table 7, affecting 186 students) and assessment grade (Table 8, affecting 328 students) resulted in the analysis of three courses for each.All courses showed changes in favor of pre-transformation (Tables 7 and 8).A Shapiro Wilk test indicated that the data was normally distributed and a Levene's test indicated that there was equality of variance across the data (α = 0.05; Table 10).Although the raw scores were higher pre-transformation, these results were not statistically significant (α = 0.008; Table 10).Hence, the null hypothesis that there was no difference pre-and post-transformation was supported.Measures of student success with textbook transformations: the Affordable Learning Georgia Initiative Open Praxis, vol. 9 issue 1, January-March 2017, pp.93-108 Data analysis was not performed within the final grade data category since only one paired course data set was provided (Table 9).However, in the sample of 68 students, final grades pre-transformation was favored.The Shapiro Wilk Test and Levene Test was tested at α = 0.05 and the Paired T-Test and Wilcoxon Signed Rank Test was tested at α = 0.008 (Table 10).The qualitative data that was provided varied.Quotes provided by students were generally uninformative with regards to their perception of the quality of the text.The vast majority of comments were about textbook cost (or lack thereof).Responses to survey data were more informative; however, since questions were different for each project, standardizing responses is impossible.That being said general insight can be gleaned from these data.Of the 20 projects that provided survey data, 16 (80%) were on average positive or neutral with regards to OER quality and perceived learning, three provided an overall negative perception of OER (15%), and one was uninformative with regards to OER quality and enhancement of learning (5%).The three projects that had negative OER survey data related to specific chapters of the OER rating lower or the entire book rating lower in terms of quality in comparison to the traditional textbook.In these surveys quality in general was perceived as organization, helpfulness with coursework or visual appeal.

Discussion
The null hypothesis that there would be no differences between pre-and post-transformation rates of DFW, rates of completion, distribution of letter grades, final exam grades and course specific assessment grades was supported (p values ranged from 0.51-4.30).Thus, this study demonstrates that the USG's ALG initiative helped students save money without negatively impacting learning outcomes.Non-significant results are important to report (Polanin, Tanner-Smith & Hennessy, 2016) and in this case supports the utility of OER because they indicate that students did as well using an open resource as they did using a traditional resource.Furthermore, Polanin et al. (2016) suggested that not reporting non-significant results can create dissemination biases that can affect which programs or policies are continued that may or may not be effective.Additionally, the purporting of these biases may inhibit the growth of new research.
This study is the first of its kind to measure some of these learning outcomes (e.g.final exam grade, assessment grade, and distribution of letter grades) at this scale.Fischer, Hilton, Robinson and Wiley (2015) focused on course completion, final grade, and enrollment intensity measures in a multi-institution study but indicated that more replicative studies were necessary and suggested that questions pertaining to the grades individual students receive when using OER vs. traditional resources would be of value.
The overall results are not statistically significant even though some measures of student learning outcomes show small gains or decreases in student learning when OER are adopted.These results suggest a consistent level of student performance pre-and post-transformation and underscores the quality of each chosen OER.The survey data that was provided generally supports the notion that students did not perceive a difference in quality or understandability when using the OER and the demonstration that students performed equally as well with the OER supports perceived high quality.
This study indicates that the individual project investigators chose appropriate OER to substitute for the traditional text(s) and aligned their course objectives with them well.The differences between pre-and post-transformation may have been more widespread with different overall results had the OER not been chosen and developed carefully.While the overall results are not statistically significant, there were individual instances in which students did better (or worse) when OER were implemented.Future studies should examine more carefully what factors coincide with higher or lower efficacy results.For example, it is possible that the change in resources resulted in instructor anxiety, lack of confidence or disorganization relating to the alignment of teaching materials with the new resources.Furthermore, it is possible that the overall impact of curriculum materials is relatively low and that the overall influence is small because it reflects this fact.Measures of student success with textbook transformations: the Affordable Learning Georgia Initiative Open Praxis, vol. 9 issue 1, January-March 2017, pp.93-108 Moreover, further studies should examine whether there are connections between students' utilization of curriculum materials and their overall scores.While explicit quantitative data on student use was not gathered, I have implicitly assumed that had utilization decreased significantly, it would have had a significant negative impact on student measures.However, it is conceivable that curriculum materials matter less than we think, or that the relative use of materials would need to be dramatically different in order to significantly influence student outcomes.

Limitations
Selecting projects that performed pre/post-transformation analysis and further selecting for specific measures whittled down the sample size for each data point, even though the overall sample size is large.This was a result of an inconsistent rate of reporting of specific data measurements amongst researchers (i.e.some reported only DFW while another only reported assessment grades), lack of pre-transformation data reporting and limited reporting of informative data (perhaps researchers were not sure what to report).Far more data should be collected by future Textbook Transformation Grant awardees to clearly address whether students are succeeding with OER.Additionally, the data that are collected should be consistent throughout grants.For example, all grantees should collect the same types of data to form a more robust data set and this data collection should be explicitly requested by ALG in the information when the call for proposals is made and outlined in final reports.Additionally, identical surveys should be employed across grants to ensure consistency of qualitative data.
The overall non-significant differences between pre-and post-transformation may have come from the overall re-design of courses and not the OER on its own.In some cases, the OER may have necessitated a reexamination of the course, so it is possible that course objectives aligned better with the OER than the traditional text.In addition, a fresh look at course material may have clarified objectives or alignment issues that were previously undetected.However, both of these factors are positive occurrences in terms of teaching and education.

Future Directions
To date, relatively little is known about the efficacy of OER.Additional large-scale studies are needed.With so many institutions now using OER there is an opportunity to conduct research on many aspects, including those that focus on differences in outcomes between traditional and OER taught courses.Furthermore, some individual courses are taught by multiple professors, which would lend to studying the learning outcomes based on pedagogical differences.Identifying differences in pedagogy may provide insight into the instructional design measures that may enhance OER learning outcomes.
The results of this study showed no difference in expected learning outcomes, which is satisfactory.However, most teachers are looking to improve student learning.It would be important to identify if there are certain types or platforms of delivery of OER that assist in learning or whether there are specific improvements that could be made to the OER to augment learning.

Table 3 : Completion Rate Pre-and Post-OER Transformation for four Georgia Colleges and Universities College/University Number of Students Completion Rate Pre-Transformation (Percent) per section Completion Rate Post-Transformation (Percent) per section Support of Pre-or Post-?
Open Praxis, vol. 9 issue 1,January-March 2017, pp.93-108