Measures of student success with textbook transformations: the Affordable Learning Georgia Initiative

Emily Croteau

Abstract


In 2014, the state of Georgia’s budget supported a University System of Georgia (USG) initiative: Affordable Learning Georgia (ALG). The initiative was implemented via Textbook Transformation Grants, which provided grants to USG faculty, libraries and librarians, and institutions to “transform their use of textbooks and other learning materials into using lower cost options”, in other words to use open educational resources (OER) in lieu of a traditional bound textbook. The Round One Textbook Transformation Grants have already shown to be successful in that they saved students approximately $760,000.  What is not known, is the collective impact on student learning. This study examines the learning gains or losses pre- and post-transformation in ALG Round One courses where traditional resources were replaced with OER.  It estimates differences between pre- and post- textbook transformation across the following outcomes: 1) Drop Fail Withdraw (DFW) rates, 2) rates of completion, 3) numbers of students receiving a final grade of A or B, C and D, 4) numerical final grades as a percent, 5) final exam grades as a percent, and, 6) course-specific assessment grades measured in percent. Twenty-four data sets were analyzed for DFW rate, eight data sets for completion rate, fourteen data sets for grade distribution, three data sets for final exam grades, three data sets for course specific assessment and one data set for final grades. The null hypothesis that there would be no differences between pre- and post-transformation rates in these learning outcomes was supported.  Thus, this study demonstrates that the USG’s ALG initiative helped students save money without negatively impacting learning outcomes. In addition, it is the first of its kind to measure some of these learning outcomes (e.g. final exam grade, assessment grade, and distribution of letter grades) at this scale. 


Keywords


Open Educational Resources; Affordable Learning Georgia; textbook transformation; learning outcomes; higher education

Full Text:

HTML PDF XML

References


Affordable Learning Georgia (2015). Textbook Transformation Grants Round One: Spring Semester 2015 Final Report Summary. Retrieved from http://affordablelearninggeorgia.org/documents/ALG_R1_Final_Report.pdf

Affordable Learning Georgia (2016). https://affordablelearninggeorgia.org

Allen, G., Guzman-Alvarez, A., Molinaro, M., & Larsen, D. (2015). Assessing the Impact and Efficacy of the Open-Access ChemWiki Textbook Project. Educause Learning Initiative Brief. Retrieved from https://library.educause.edu/resources/2015/1/assessing-the-impact-and-efficacy-of-the-openaccess-chemwiki-textbook-project

Allen, I. E., & Seaman, J. (2014). Opening the Curriculum: Open Educational Resources in US Higher Education, 2014. Babson Survey Research Group.

Altbach, P. G., Kelly, G.P., Petrie, H.G., Weis, L. (Ed.). (1991). Textbooks in American society: Politics, policy, and pedagogy. New York: State University of New York Press.

Bliss, T. J., Hilton III, J., Wiley, D., & Thanos, K. (2013). The cost and quality of online open textbooks: Perceptions of community college faculty and students. First Monday, 18(1). https://doi.org/10.5210/fm.v18i1.3972

Bowen, W. G., Chingos, M. M., Lack, K. A., & Nygren, T. I. (2014). Interactive Learning Online at Public Universities: Evidence from a Six‐Campus Randomized Trial. Journal of Policy Analysis and Management, 33(1), 94-111. https://doi.org/10.18665/sr.22464

Brown, M. B. & Forsythe, A. B. (1974). Robust Tests for the Equality of Variances. Journal of the American Statistical Association, 69, 364-367.

Buczynski, J. A. (2007). Faculty begin to replace textbooks with “freely” accessible online resources. Internet Reference Services Quarterly, 11(4), 169-179.

Charalambous, C. Y. & Hill H. C. (2012). Teacher knowledge, curriculum materials, and quality of instruction: Unpacking a complex relationship. Journal of Curriculum Studies, 44(4), 443-466.

College Board (2016). https://collegeboard.org

Feldstein, A., Martin, M., Hudson, A., Warren, K., Hilton III, J., & Wiley, D. (2012). Open Textbooks and Increased Student Access and Outcomes. European Journal of Open, Distance and E-Learning. Retrieved from http://www.eurodl.org/index.php?article=533

Fischer, L., Hilton III, J., Robinson, T.J. & Wiley, D. (2015). A multi-institutional study of the impact of open textbook adoption on the learning outcomes of post-secondary students. Journal of Computing in Higher Education, 27(3), 159-172. https://doi.org/10.1007/s12528-015-9101-x

Florida Virtual Campus. (2012). 2012 Florida Student Textbook Survey. Tallahassee, FL. Retrieved from http://www.openaccesstextbooks.org/%5Cpdf%5C2012_Florida_Student_Textbook_Survey.pdf.

Glass, G.V. (1966). Testing homogeneity of variances. American Educational Research Journal, 3(3), 187-190.

Hewlett (2013). Open Educational Resources. Retrieved from http://www.hewlett.org/programs/education-program/open-educational-resources

Hilton III, J. L. (2016). Open educational resources and college textbook choices: a review of research on efficacy and perceptions. Education Technology Research and Development, 64(4), 573-590. https://doi.org/10.1007/s11423-016-9434-9

Hilton III, J.L., & Laman, C. (2012). One college’s use of an open psychology textbook. Open Learning: The Journal of Open, Distance and e-Learning, 27(3), 265-272. http://dx.doi.org/10.1080/02680513.2012.716657

Hilton III, J. L., Robinson, T. J., Wiley, D., & Ackerman, J. D. (2014). Cost-savings achieved in two semesters through the adoption of open educational resources. The International Review of Research in Open and Distributed Learning, 15(2). http://dx.doi.org/10.19173/irrodl.v15i2.1700

Hilton III, J. L., & Wiley, D. (2011). Open access textbooks and financial sustainability: A case study on Flat World Knowledge. The International Review of Research in Open and Distributed Learning, 12(5), 18-26. http://dx.doi.org/10.19173/irrodl.v12i5.960

Hilton III, J. L., Wiley, D., Stein, J., & Johnson, A. (2010). The four ‘R’s of openness and ALMS analysis: frameworks for open educational resources. Open Learning: The Journal of Open, Distance and E-Learning, 25(1), 37-44. Retrieved from http://www.tandfonline.com/doi/full/10.1080/02680510903482132

Levene, H. (1960). Robust tests for equality of variances. In Contributions to probability and statistics (pp. 278-292). Stanford University Press.

Lovett, M., Meyer, O., & Thille, C. (2008). The open learning initiative: Measuring the effectiveness of the OLI statistics course in accelerating student learning. Journal of Interactive Media in Education, 2008(1), Art-13. http://doi.org/10.5334/2008-14

National Association of College Stores (NACS) (2016). https://nacs.org

OpenStax (2016). https://openstax.org

Open Textbook Library (2016). http://open.umn.edu/opentextbooks/.

Paulsen, M. B., & St John, E. P. (2002). Social class and college costs: Examining the financial nexus between college choice and persistence. The Journal of Higher Education, 73(2), 189-236.

Pawlyshyn, N., Braddlee, D., Casper, L., & Miller, H. (2013). Adopting OER: A Case Study of Cross-Institutional Collaboration and Innovation. Educause Review. Retrieved from http://er.educause.edu/articles/2013/11/adopting-oer-a-case-study-of-crossinstitutional-collaboration-and-innovation

Polanin, J.R., Tanner-Smith, E.E. & Hennessy, E.A. (2016). Estimating the difference between published and unpublished effect sizes: A meta-review. Review of Educational Research, 86(1), 207-236. https://doi.org/10.3102/0034654315582067

Provasnik, S., & Planty, M. (2008). Community Colleges: Special Supplement to The Condition of Education 2008. National Center for Education Statistics. Retrieved from https://nces.ed.gov/pubs2008/2008033.pdf

Razali, N. M. & Wah, Y. B. (2011). Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests. Journal of Statistical Modeling and Analytics, 2, 21-33.

Robinson, T. J. (2015). The Effects of Open Educational Resource Adoption on Measures of Post-Secondary Student Success. Unpublished Doctoral Dissertation, Brigham Young University, Provo, UT, USA.

Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52, 591-611

UNESCO (2002). Forum on the impact of open courseware for higher education in developing countries: Final report. Retrieved from www.unesco.org/iiep/eng/focus/opensrc/PDF/OERForumFinalReport.pdf

Wilcoxon, F., Katti, S. K., & Wilcox, R. A. (1970). Critical values and probability levels for the Wilcoxon rank sum test and the Wilcoxon signed rank test. Selected tables in mathematical statistics, 1, 171-259.

Wiley, D., Bliss, T. J., & McEwen, M. (2014). Open Educational Resources: a review of the literature. In Handbook of research on educational communications and technology (pp. 781-789). New York: Springer.




DOI: http://dx.doi.org/10.5944/openpraxis.9.1.505

Refbacks

  • There are currently no refbacks.