Accepted on 31 May 2020
Submitted on 23 Jan 2020
As the price of traditional college textbooks have increased (Government Accountability Office, 2013; U.S. Bureau of Labor Statistics, 2016), so too has a movement to shift to alternatives that are free, editable, and re-mixable. These open educational resources (also referred to as OER) are defined as “teaching, learning and research materials in any medium –digital or otherwise– that reside in the public domain or have been released under an open license that permits no-cost access, use, adaptation and redistribution by others with no or limited restrictions” (Hewlett Foundation, 2020). Further, research has demonstrated that the use of OER has either no impact or a positive impact on learning outcomes when compared to traditional textbooks (Hilton, 2016).
While research into OER has expanded over the past decade, the body of research still remains limited (see Hilton, 2019b). In addition, while there has been a call for more rigorous statistical research on the impacts and efficacy of OER usage (Hilton, 2019b), there is also the need for additional research into the design and implementation of courses using OER. Research on OER, particularly North American studies aligned with the COUP framework (cost, outcomes, usage, and perceptions), have often been positioned as a straight comparison, swapping a traditional textbook for open materials and examining the effects (Hilton, 2019b). While these comparative studies improve statistical measures, it is also useful to explore how these texts can provide a radically different teaching and learning experience for both students and faculty. OER, by their very definition, provide universal access to students and are able to be reconfigured to suit a particular instructor’s needs and objectives which can radically reshape the nature of a course (see Tuomi, 2013; Chikuni, Cox, & Czerniewicz 2019).
This study examines the creation and implementation of an open textbook for an introductory-level World Regional Geography course. While most World Regional Geography textbooks and courses approach the material from a relatively traditional perspective, emphasizing a broad array of facts and figures and utilizing a lecture-based approach, shifting to OER enabled a complete course redesign and the adoption of a flipped-classroom model. Students have responded positively to these changes and overall the use of OER has revitalized the course.
Across the United States, textbook cost increases have outpaced the inflation rate, with prices rising an average of 6 percent per year from 2002 to 2012 (Government Accountability Office, 2013). These high costs can have a significant impact on students in a variety of ways. When exploring the impacts of textbook costs on course outcomes, Jhangiani and Jhangiani (2017) found that students reported taking fewer courses, not registering for a course, or withdrawing from a course as a result of textbook costs. Thirty percent of respondents in the study reported earning a lower grade as a result of the cost of textbooks (Jhangiani & Jhangiani, 2017). While textbooks are central facet of a college course experience, a 2012 survey of students in Florida found that 64% of student respondents reported not purchasing a required textbook due to its cost (Florida Virtual Campus, 2012).
Open educational resources thus provide a way for instructors to mitigate these cost challenges while providing increased access for students. Research shows that this increased access can significantly improve student grades and withdrawal rates (Feldstein et al., 2012). Course pass rates were similarly shown to increase in a study examining a basic bath course (Pawlyshyn, Braddlee, Casper & Miller, 2013). Even if exam scores or grades remained the same as a variety of studies have shown (see Lovett, Meyer, & Thille, 2008; Wiley, Hilton III, Ellington & Hall, 2012; Hendricks, Reinsberg & Rieger, 2017), the cost savings of using OER texts can be substantial. In the Hendricks, Reinsberg and Rieger (2017) study of a physics course in Canada, for example, just one year of OER use saved the students $85,000 (Canadian dollars). This study complements previous research by examining the impact of the use of an open textbook over the course of a semester on student perception and by providing a comprehensive case study on the writing, implementation, and effectiveness of an open textbook. Furthermore, this study presents a statistical analysis of student use and perceptions that is sometimes missing from more anecdotal case studies.
Open educational resources (OER) are characterized as “teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use or re-purposing by others” (Atkins, Brown, & Hammond, 2007, p. 4). While textbooks are perhaps the most commonly cited open educational resource, they also include modules, notes, software, and other course materials (Atkins et al., 2007). Furthermore, although an awareness of open educational materials among instructors has grown considerably over the past few years, it still remains relatively low, with less than half of faculty reporting an awareness of OER materials (Seaman & Seaman, 2018). Peer-reviewed research on OER, while also expanding, remains limited (Hilton, 2019a).
That said, the Open Education Group is seeking to further OER research using its COUP framework, which refers to research into the cost, outcomes, usage, and perceptions of OER materials (Open Education Group, n.d.). Much of the evidence-based research on OER can be grouped into these four areas and this project takes a similar approach, primarily exploring the categories of student usage and perceptions using both quantitative, statistical analysis as well as a qualitative approach.
When comparing the perception of the quality of open textbooks versus traditional textbooks, the literature shows that the vast majority of students report open textbooks to be of the same or better quality than the commercial textbooks they have used. Bliss, Hilton III, Wiley and Thanos (2013) reported that 94% of 490 students said they found open textbooks to be of equal or higher quality than traditional textbooks. Illowsky, Hilton III, Whiting and Ackerman (2016) reported on two surveys of students using an open textbook at a U.S. community college, one in 2013 of 231 students and one in 2015 of 94 students (the students surveyed in 2015 used a significantly-revised version of the textbook the students in 2013 had used). In the 2013 survey, 87% of students rated the quality of the open textbook to be the same or better when compared to traditional textbooks, and in the 2015 survey, 93% of student respondents did so. In a survey of over 300 students in British Columbia, 63% of respondents said the quality of the open textbook used in their course was “above average” or “excellent,” with another 33% rating the quality as “average” (Jhangiani & Jhangiani, 2017). Illowsky et al. (2016) similarly found 87% of students using an open statistics textbook rated it equal or better than a traditional text. Faculty similarly had positive views, with one study finding that 50% of faculty perceived the OER text as the same quality as traditional textbooks and 33% finding that it was better (Hilton et al., 2013). Much of the research on student perceptions focuses on either student feedback after having used open materials, or a more controlled study where students compare and rate a traditional textbook versus an open text (see Hilton, 2019a).
Another limitation of much of the existing research on OER is that traditional textbooks can be more well-suited to traditional learning goals and assessment, such as multiple-choice tests. Open textbooks, however, can be pedagogically freeing for both instructors and students, enabling instructors to reshape learning objectives and assessment metrics around entirely different goals. Research into open pedagogy have more explicitly explored these new educational landscapes, critically examining how open educational resources have enabled what could be termed “open pedagogy” (see Hegarty 2015; Wiley & Hilton, 2018) or open educational practices (OEP) (see Cronin 2017; Cronin & MacLaren 2018). Others like Weller et al. (2015) indicated that instructors are indeed more likely to innovate and experiment with their course instruction as a result of using OER. This study builds upon these foundations to examine how open educational materials could be used to enable a completely redesigned course and explores both the faculty and student perspective on how those changes impacted the teaching and learning experience.
At most universities, World Regional Geography is an introductory-level general education course typically taken by both Geography majors and a wide array of non-majors. It is famously a challenging course to teach, as a 2019 session at the annual meeting of the American Association of Geographers can attest to: “Teach the World, No problem: Challenges to Teaching World Regional Geography in One Semester.” The breadth of the course, covering in theory the entirety of the world’s people in places in one semester, combined with the vastly different interest and experience levels of students in the course can present a significant obstacle for instructors. Compounding this issue, typical World Regional Geography textbooks emphasize breadth over depth, providing a somewhat repetitive presentation of various facets of each world region, to include its key physical geographic features, political geography, economics, culture, and so on. For instructors, the course can feel a bit like teaching the encyclopedia, conveying a list of facts about the world with little over-arching structure or connections between places.
Further, for students this approach of broad, place-based geographic knowledge closely aligns with how geography is taught at the primary and secondary levels in United States public schools. Map quizzes are common in K-12 settings and are traditionally a feature of World Regional Geography courses at the college level as well. For professional geographers, though, it is the connections between places and a deeper spatial understanding that is of critical importance. What this amounts to, then, is presenting students with a novice-level approach to geography rather than how experts understand the discipline.
Within the Department of Geography at the University of Mary Washington, individual instructors have significant freedom to design and teach their courses and there is only one instructor, the author of this research paper, who teaches World Regional Geography every semester. Thus, it was the instructor’s decision, in consultation with the department chair, to shift to a team-based learning approach. Team-based learning provided a research-backed foundational structure (see Michaelsen, Knight, & Fink 2002) for shifting from a lecture-based approach to a problem-based approach where students could work in small groups to discuss and solve complex global issues. However, the traditional format of most World Regional Geography textbooks presented an impediment to “flipping” the course since they emphasized facts about specific regions rather than connections between places or larger global problems. Furthermore, students routinely commented that they were overwhelmed by the amount of information in the traditional World Regional text, and thus assigning additional news articles on top of the textbook reading, which students could then use in application activities, would be challenging.
Thus, the open textbook World Regional Geography was developed during the summer of 2016 as an alternative to more traditional geography textbooks on the market. Rather than present a broad array of information about specific places, this text emphasizes depth over breadth, focusing on a different key concept in geography for each of the world’s regions. The broad concepts of globalization and inequality are woven through each chapter, providing a more cohesive overall structure. World Regional Geography is also more concise than traditional texts, containing ten chapters rather than the more typical fourteen or fifteen, which more closely aligns with the team-based learning format and enables instructors the flexibility to assign additional readings. The text was developed to be openly and freely accessible in support of the American Association of Geographer’s initiative to broaden participation in the discipline of geography (American Association of Geographers, n.d.). If students couldn’t even access the textbook at the introductory undergraduate level, how would they go on to be active participants in the field? Providing an open and free textbook was one way to remove an early barrier to the discipline.
World Regional Geography was initially written and compiled using LaTeX, a typesetting programming language which is available for free at https://www.latex-project.org/, and posted online in both PDF and HTML format on the author’s personal domain. In the summer of 2019, the online version was shifted to Pressbooks (at https://worldgeo.pressbooks.com/) in order to make the text more accessible for students with disabilities. Pressbooks (https://pressbooks.com/) is built on Wordpress and shares its accessibility features, such as supporting screen readers and the use of alternative text. It is free to create a Pressbook, though there is a one-time cost to “publish” the book and make it available to others. At the time of this publication, the cost was $19.99 (USD) to publish an eBook or $99 (USD) to create both a eBook and generate a PDF. Again, this was a one-time cost and there are no additional charges to make edits or changes. Pressbooks also supports exporting the text in other formats compatible with print-on-demand services, so World Regional Geography is now offered as a color print edition on Amazon and Kindle through Amazon’s Kindle Direct Publishing program (https://kdp.amazon.com). World Regional Geography was published under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License (https://creativecommons.org/licenses/by-nc-sa/4.0/). There are a variety of licenses offered by the Creative Commons non-profit organization and this particular one was selected because it allows non-commercial re-use as long as the material is attributed and shared openly. Since its initial publication, World Regional Geography has been downloaded over 18,000 times in over 30 countries. The Pressbooks version of the text has been visited by over 2,800 active users since its posting in summer 2019. It has also been adopted by a number of universities and community colleges. At the University of Mary Washington alone, adopting an open textbook represents a potential cost savings to students of $84,000 over the past four years.
This study was conducted beginning in the Fall 2018 semester and continuing to the Spring 2019 semester in two different sections of a World Regional Geography course at the University of Mary Washington. The University of Mary Washington is public liberal arts university located in Fredericksburg, Virginia. The university has approximately 4,800 students, most of whom are undergraduates (around 4,400) and most of whom are full-time (88%). Of the full-time undergraduate students in academic year 2018–2019, 68% applied for need-based financial aid and 32% received need-based scholarship or grant aid (University of Mary Washington, 2019).
This class used team-based learning, a form of “flipping” the classroom where students come to class having completed the assigned reading, take a quiz to check their knowledge, and then work in groups to discuss and solve complex problems. Each section of the course had the same basic content, assignments, exams, and delivery format and used the same textbook, an open World Regional Geography textbook written by the instructor and implemented in Fall 2016. Each course had approximately 70 students enrolled and met face-to-face three times each week. Students were asked to complete an anonymous pre-semester, mid-semester, and end-of-semester survey to determine their perceptions and use of the textbook and course format.
Beginning in Fall 2016, a relatively brief survey was developed by the instructor to gauge initial student interest in the course material, delivery format, and the textbook at the start of the course with additional surveys at the mid-semester point and at the end of the semester. These surveys were anonymous and were disseminated using Canvas, our institutional course management system. In early versions of the survey, the focus was primarily on team-based learning and on whether students preferred a printed or electronic textbook. As this research project evolved, additional questions were added in Fall 2018 specifically examining student use and perceptions of the open textbook in more detail utilizing the examples provided in the open education toolkit (http://openedgroup.org/toolkit) which builds on research by Bliss et al. (2013). For example, students were asked, “How would you rate the quality of free, open access textbooks compared to traditional course texts?” As in the study by Illowsky et al. (2016), students were not provided a definition for what would constitute a “quality” textbook and instead this question was left open to students to interpret. These additional questions regarding specific issues related to student perception and textbook use are the focus of this study. The core questions regarding team-based learning and interest in the course material were also used in the later version of the survey. A total of 136 students responded to the pre-semester surveys during the Fall 2018 and Spring 2019 semesters and 116 responded to the end-of-semester survey. These results were analyzed using SPSS Statistics 24, a statistical analysis software package. Since these surveys were anonymous, independent t-tests rather than paired t-tests were utilized, a limitation with this type of anonymous research (Warne, 2018).
As discussed, this textbook was provided freely and openly online. In addition, a black-and-white printed, spiral-bound version was offered at the University of Mary Washington campus bookstore for $15. Perhaps surprisingly, most students (N=79, 69%) responded that they purchased a printed textbook (see Table 1).
Table 1
How did you access the textbook for this class? (Please select all that apply.)
| Access Method | N | Percentage of all respondents |
|---|---|---|
| PDF or HTML Viewed Online | 40 | 35% |
| Downloaded PDF or HTML | 35 | 30% |
| Purchased a printed textbook | 79 | 69% |
| Total Respondents | 115 |
This speaks to research from the National Association of College Stores which found that most college students prefer printed textbooks over digital versions (NACS, 2014). In addition, the high percentage of students downloading or viewing a web version connects to the importance of ensuring accessibility to all learners, which can sometimes be a problem with OER materials (see Navarrete & Luján-Mora, 2018).
Several questions were developed to determine student perceptions of the quality of the open textbook. Since the textbook was developed specifically with the intention of having an accessible and conversational writing style, students were asked to rate the statement, “The writing style of the textbook and its approach has enhanced my understanding of the course material,” using a five-point Likert scale ranging from Strongly Disagree to Strongly Agree. Students overwhelmingly agreed with this statement, with 92% reporting either “Agree” or “Strongly Agree” (see Table 2). Instructors considering authoring OER texts might feel they would be unable to match a traditional textbook’s stately writing style, but it would seem that a more accessible and conversational tone is valued by students and perhaps an instructor’s goal could simply be to allow their own voice to shine through.
Table 2
The writing style of the textbook and its approach has enhanced my understanding of the course material
| Rating | N | Percentage of all respondents |
|---|---|---|
| Strongly Disagree | 0 | 0% |
| Disagree | 1 | 1% |
| Neutral | 8 | 7% |
| Agree | 52 | 45% |
| Strongly Agree | 55 | 47% |
| Total Respondents | 116 | 100% |
A more general concern was overall student perception of open educational resources and how using these resources impacted their perception. At the beginning of the semester, students were asked, “How would you rate the quality of free, open access textbooks compared to traditional course texts?” and this question was repeated at the end of the semester. Students could select whether open textbooks were worse than, about the same as, or better than traditional textbooks.
At the beginning of the semester, about half of students rated open textbooks as about the same as traditional textbooks (see Table 3). By the end of the semester, over 80% of students rated open textbooks as better than traditional textbooks.
Table 3
How would you rate the quality of free, open access textbooks compared to traditional course texts?
| Rating | Pre-Semester | End-of-Semester | ||
|---|---|---|---|---|
| N | % | N | % | |
| Open textbooks are WORSE than traditional textbooks | 4 | 3% | 2 | 2% |
| Open textbooks are ABOUT THE SAME as traditional textbooks | 68 | 50% | 19 | 17% |
| Open textbooks are BETTER than traditional textbooks | 64 | 47% | 94 | 82% |
| Total | 136 | 100% | 115 | 100% |
A t-test for independent samples can be utilized to determine whether or not this difference in the perception of quality is statistically significant. Before conducting the t-test, the homogeneity of variance was tested to determine if the variance within the two groups was equal. The result of Levene’s test for equality of variances was significant (p<.01) and thus the variances are not equal. A t-test was calculated based on unequal variances and the result was highly significant (t=5.594, p<.001). The practical significance of the difference was then analyzed using Cohen’s d yielding a value of .696, which is considered to be a medium effect size. In general, an effect size of .2 is considered small, .5 is considered medium, and .8 is considered large (Cohen, 1988). In other words, there is both a statistically significant and practically significant difference between student perception of open textbook quality before and after using an open textbook.
Students were given an opportunity to provide general comments at the end of both the pre- and post-semester surveys. This course utilized a team-based learning format which was unfamiliar to many students, so much of the comments expressed concern about the particular course structure. Of the pre-semester comments, few discussed perceptions of the textbook’s quality. Only one, for example, directly compared the text to another OER text used previously:
“The course textbook seems to be of much better quality than the open textbook I used for a math class last semester.”
Others based their perception of quality on the instructor’s in-class framing of the textbook and how it was developed:
“I love that we are going to learn out of a textbook that gives you exactly what you need to know and does not give you un needed super detailed material to mess you up.”
“I really like how instead of using a traditional textbook with lots of information we won’t use, you created your own book to concentrate on the information you deem important.”
The in-class discussion of how the textbook differed from traditional World Regional Geography texts, taking a concise approach rather than presenting a broad survey of geographic information, was clearly reflected in how students approached the text.
At the end of the semester, students were overwhelmingly positive about the textbook, with only one negative comment:
“I think the textbook was somewhat dry at times but there isn’t really anything that can be done about that.”
All of the other comments on the textbook were positive, for example:
“I really enjoyed the textbook. The language is easy to comprehend, not so much that it’s like ‘easy,’ but I understood what was being said even though I’ve never taken a geography class before this. It was also an extremely interesting textbook!”
“The textbook was absolutely wonderful compared to some other textbooks that I’ve had to use.”
“This was the best class textbook I have ever used. Information was well-written and easy to read. Unlike traditional textbooks that are unrelated to a specific class, this textbook did not include extraneous information. Therefore, I read each chapter to completion, rather than skimming or skipping reading altogether.”
“The textbook is well written and much more intriguing than a traditional book.”
Before authoring and adopting the open text, previous university-wide course evaluation comments rarely included mentions of the textbook, and when the textbook was discussed, it was generally negative. Overall, student perception of the quality of the textbook was positive, particularly after using the book throughout the semester and understanding more about its development and approach.
At the beginning of each semester, students were asked how often they use the required course texts in a typical course. Consistent with previous research (see Burchfield & Sappington, 2000; Clump, Bauer, & Bradley, 2004), students are not using required textbooks as much as instructors might think and this can have significant impacts on learning outcomes. Around two-thirds of respondents reported using the required texts frequently, two to three times each week or more (see Table 4). Ten percent of respondents reported only using the required texts two to three times each semester, and four respondents never used the required texts at all.
Table 4
For a typical course, how often do you use the required texts?
| Frequency of Use | N | Percentage of all respondents |
|---|---|---|
| Never | 4 | 3% |
| 2-3 Times a Semester | 13 | 10% |
| 2-3 Times a Month | 28 | 21% |
| 2-3 Times a Week | 79 | 58% |
| Daily | 12 | 9% |
| Total Respondents | 136 | 100% |
At the end of the semester, students were asked the same question but about the World Regional Geography textbook, and results were noticeably different. 85% of respondents reported using the World Regional text two to three times each week or more, with no respondents reporting that they never used it (see Table 5).
Table 5
For this course, how often did you use the required text?
| Frequency of Use | N | Percentage of all respondents |
|---|---|---|
| Never | 0 | 0% |
| 2-3 Times a Semester | 1 | 1% |
| 2-3 Times a Month | 16 | 14% |
| 2-3 Times a Week | 91 | 78% |
| Daily | 8 | 7% |
| Total Respondents | 116 | 100% |
These differences were then tested using an independent-samples t-test. The results of Levene’s test for homogeneity of variance was significant (p<.01), meaning that the variance between the groups was not equal, so a t-test was calculated based on the assumption of unequal variance. There was a significant difference between how often students used the required texts in a typical course compared to how often used the required text in World Regional Geography (t=3.511, p<.01). This difference further analyzed using Cohen’s d, yielding a value of .433, or a small to medium effect size.
Student comments pre-semester expressed a general appreciation for the free online textbook:
“I… really appreciate that it is available online for free, as textbook costs can add up quick.”
“The times that I’ve used a textbook written by the professor of the course, the textbook seemed much more relevant and I enjoyed (and used) the textbook way more.”
Other students appreciated having access to a low-cost printed version:
“I’m very excited that the textbook is not expensive in the bookstore and there is an online version for free!”
Another expressed frustration that the textbook was not available at online retailers:
“It would be been more convenient if the textbook was available on Amazon.”
As a result of the high percentage of students who purchased a printed version of the textbook, and the generally low quality of the black-and-white spiral version, a full color print edition was developed over the summer of 2019 and is now for sale through Amazon’s Kindle Direct Publishing platform for less than $35 (US).
At the conclusion of the semester, students again expressed appreciation for the use of an open textbook:
“I think it is great for the textbook to be available for free online or for a low cost in print, as it makes it more accessible for students struggling financially and already having to pay for many other expenses at school.”
“It was a breath of fresh air that the class did not have to buy an expensive textbook.”
“It’s also the only college class I’ve taken in which I’ve read the textbook in its entirety!”
Research indicates that students do not read textbooks as often as instructors would like to believe (see Clump et al., 2004), and thus one way to tackle this issue is to ensure access for all students. Furthermore, several students specifically mentioned the desire to highlight and underline texts, which is not possible if a student is renting the book or needs to resell it in pristine condition. Overall, open textbooks allow for a valuable combination of instructor customization and universal access which, for this case study, clearly impacted the frequency with which students engaged with the course text.
Implementing an open textbook in a course can be a daunting task. Rarely do open educational resources come with as robust an array of ancillary materials as a traditionally published textbook. However, OER provides an opportunity for instructors to dramatically redesign and rethink their courses. Even if they do not embark on authoring an entirely new open text, existing open material can be freely remixed, edited, and reconfigured to suit an instructor’s particular needs and objectives. For World Regional Geography at the University of Mary Washington, authoring and utilizing an open textbook allowed a shift from a more traditional, lecture-based course that emphasized a novice-level understanding of global geography to a much more in-depth and nuanced discussion of critical global issues. This course redesign would not have been possible with a traditional textbook. Certainly from an instructor perspective, this course has become much more enjoyable to teach and student feedback is consistently positive. Furthermore, this research has enabled additional revisions to the course textbook. As a result of the high number of students purchasing the printed text, and the printing limitations of other institutions, a printed, full-color edition was developed during the summer of 2019 and is now available for purchase on Amazon for under $35.
It is clear from both the quantitative and qualitative data collected as part of this study that actually using an open textbook significantly improves a student’s perception of OER. Students also used the open textbook significantly more frequently than they would in a typical course. Furthermore, if an instructor can frame why OER is being used, and explain to students how instructor-authored or remixed content fits within the course goals and objectives, students will likely respond positively as numerous comments in this study demonstrate. This study did not have a control group or pre-OER adoption data to compare to, since everything in the course changed as a result of OER adoption, and this represents a limitation to the study. However, what this research does demonstrate is that using OER can have a significant impact on both the teaching and learning experience and can help an instructor create and meet complex and dynamic learning objectives.
American Association of Geographers (n.d.) Diversity and Inclusion. Retrieved from http://www.aag.org/cs/programs/diversity
Atkins, D., Brown, J. S., & Hammond, A. L. (2007). A review of the open educational resources (OER) movement: Achievements, challenges, and new opportunities. Retrieved from http://hewlett.org/wp-content/uploads/2016/08/ReviewoftheOERMovement.pdf
Bliss, T. J., Hilton III, J., Wiley, D., & Thanos, K. (2013). The cost and quality of online open textbooks: Perceptions of community college faculty and students. First Monday, 18(1). https://doi.org/10.5210/fm.v18i1.3972
Burchfield, C.M., & Sappington, J. (2000). Compliance with required reading assignments. Teaching of Psychology, 27, 58–60. Retrieved from https://psycnet.apa.org/record/2000-07173-017
Chikuni, P. R.Cox, G., &Czerniewicz, L. (2019). Exploring the Institutional OER Policy Landscape in South Africa: Dominant Discourses and Assumptions. International Journal of Education and Development using Information and Communication Technology, 15(4), 165–179. Retrieved from https://files.eric.ed.gov/fulltext/EJ1239627.pdf
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum. https://doi.org/10.4324/9780203771587
Clump, M. A., Bauer, H., & Bradley, C. (2004). The Extent to which Psychology Students Read Textbooks: A Multiple Class Analysis of Reading across the Psychology Curriculum. Journal of Instructional Psychology, 31(3). Retrieved from https://eric.ed.gov/?id=EJ774105
Cronin, C. (2017). Openness and praxis: Exploring the use of open educational practices in higher education. International Review of Research in Open and Distributed Learning: IRRODL, 18(5), 15–34. https://doi.org/10.19173/irrodl.v18i5.3096
Cronin, C., & MacLaren, I. (2018). Conceptualising OEP: A review of theoretical and empirical literature in Open Educational Practices. Open Praxis, 10(2), 127–143. https://doi.org/10.5944/openpraxis.10.2.825
Feldstein, A., Martin, M., Hudson, A., Warren, K., Hilton III, J., & Wiley, D. (2012). Open textbooks and increased student access and outcomes. European Journal of Open, Distance and E-Learning, 15(2). Retrieved from https://eric.ed.gov/?id=EJ992490
Florida Virtual Campus (2012). 2012 Florida student textbook survey. Tallahassee. Retrieved from https://www.openaccesstextbooks.org/pdf/2012_Florida_Student_Textbook_Survey.pdf
Government Accountability Office (GAO) (2013). College Textbooks: Students Have Greater Access to Textbook Information. Washington, D.C . Retrieved from https://www.gao.gov/-assets/660/655066.pdf
Hegarty, B. (2015). Attributes of open pedagogy: A model for using open educational resources. Educational Technology, 55(4), 3–13.
Hendricks, C., Reinsberg, S. A., & Rieger, G. W. (2017). The Adoption of an Open Textbook in a Large Physics Course: An Analysis of Cost, Outcomes, Use, and Perceptions. The International Review of Research in Open and Distributed Learning, 18(4). https://doi.org/10.19173/irrodl.v18i4.3006
Hewlett Foundation (2020). Open educational resources. Retrieved from https://hewlett.org/strategy/open-educational-resources/
Hilton III, J. L., Gaudet, D., Clark, P., Robinson, J., & Wiley, D. (2013). The adoption of open educational resources by one community college math department. The International Review of Research in Open and Distributed Learning, 14(4). https://doi.org/10.19173/irrodl.v14i4.1523
Hilton, J. (2016). Open educational resources and college textbook choices: A review of research on efficacy and perceptions. Educational Technology Research and Development, 64(4), 573–590. https://doi.org/10.1007/s11423-016-9434-9
Hilton, J. (2019a). Recent Research on OER. Presented at OpenEd 2019, Phoenix, AZ.
Hilton, J. (2019b). Open educational resources, student efficacy, and user perceptions: a synthesis of research published between 2015 and 2018. Educational Technology Research and Development, 68, 1–24. https://doi.org/10.1007/s11423-019-09700-4
Illowsky, B., Hilton III, J., Whiting, J., & Ackerman, J. (2016). Examining student perception of an open statistics book. Open Praxis, 8(3), 265–276. https://doi.org/10.5944/openpraxis.8.3.304
Jhangiani, R. S., & Jhangiani, S. (2017). Investigating the perceptions, use, and impact of open textbooks: A survey of post-secondary students in British Columbia. The International Review of Research in Open and Distributed Learning, 18(4). https://doi.org/10.19173/irrodl.v18i4.3012
Lovett, M., Meyer, O., & Thille, C. (2008). JIME - The Open Learning Initiative: Measuring the Effectiveness of the OLI Statistics Course in Accelerating Student Learning. Journal of Interactive Media in Education, 2008(1), Art. 13. http://doi.org/10.5334/2008-14
Michaelsen, L. K., Knight, A., & Fink, L. D. (2002). Team-based learning: A transformative use of small groups. Westport, CT: Praeger.
National Association of College Stores (NACS) (2014). College Students Still Prefer Print Textbooks to Digital . Retrieved from https://www.nacs.org/advocacynewsmedia/pressreleases/collegestudentsstillpreferprinttextbookstodigital.aspx
Navarrete, R., & Luján-Mora, S. (2018). Bridging the accessibility gap in Open Educational Resources. Universal Access in the Information Society, 17(4), 755–774. https://doi.org/10.1007/s10209-017-0529-9
Open Education Group (n.d.). The COUP framework. Retrieved from https://openedgroup.org/coup
Pawlyshyn, N., Braddlee, D., Casper, L., & Miller, H. (2013). Adopting OER: A case study of crossinstitutional collaboration and innovation. Educause Review. Retrieved from https://er.educause.edu/articles/2013/11/adopting-oer-a-case-study-of-crossinstitutional-collaboration-and-innovation
Seaman, J., & Seaman, J. (2018). Freeing the Textbook: Educational Resources in U.S. Higher Education, 2018. Babson Survey Research Group. Retrieved from http://www.onlinelearningsurvey.com/oer.html
Tuomi, I. (2013). Open educational resources and the transformation of education. European Journal of Education, 48(1), 58–78. https://doi.org/10.1111/ejed.12019
University of Mary Washington (2019). 2018–2019 Common Data Set. Retrieved from https://academics.umw.edu/iae/institutional-research-2/common-data-sets-ms-excel-files/common-data-set-2/
U.S. Bureau of Labor Statistics. (2016, September 12). Consumer price index—all urban consumers: College textbooks. Retrieved from http://www.bls.gov/data/
Warne, R. T. (2018). Statistics for the Social Sciences. New York: Cambridge University Press. https://doi.org/10.1017/9781316442715
Weller, M., De los Arcos, B., Farrow, R., Pitt, B., & McAndrew, P. (2015). The impact of OER on teaching and learning practice. Open Praxis, 7(4), 351–361. http://dx.doi.org/10.5944/openpraxis.7.4.227
Wiley, D., Hilton III, J. L., Ellington, S., & Hall, T. (2012). A preliminary examination of the cost savings and learning impacts of using open textbooks in middle and high school science classes. The International Review of Research in Open and Distributed Learning, 13(3), 262–276. https://doi.org/10.19173/irrodl.v13i3.1153
Wiley, D., & Hilton III, J. L. (2018). Defining OER-enabled pedagogy. International Review of Research in Open and Distributed Learning, 19(4). https://doi.org/10.19173/irrodl.v19i4.3601