Literature Review

Open Educational Resources (OER) can benefit students in myriad ways, some of which include eliminating the cost of the textbook, ensuring earlier access to course materials and enabling students to engage with course materials when and where they choose. The high cost of textbooks can be an impediment to academic success, especially among students with pressing economic concerns (Broton & Goldrick-Rab, 2016; Colvard, Watson & Park, 2018). Students from low-income families experience food and housing insecurity and have difficulty paying their bills. In a broadly cited survey of 22,000 public college and university students in Florida, 66.6% of students report not purchasing the required textbook for the course (Florida Virtual Campus, 2016). In another survey (n = 3115), students at Brigham Young University indicated that they would spend money saved from not purchasing a textbook on housing (28.86%) and food (28.32%) (Martin, Belikov, Hilton, Wiley & Fisher, 2017). Students report that they have delayed or decided against making a textbook purchase despite anxiety over how that choice might negatively affect their course grade (Senack, 2014; Hilton, 2016; Martin et al., 2017; Stein, Hart, Keaney & White, 2017; Jhangiani & Jhangiani, 2017).

Beyond eliminating the cost of the textbook, OER materials are free and available on the first day of class and then continuously accessible at any time. First-day access to required course materials is extremely important to student learning. Research shows that when students have first-day access they do much better in a course, and when they have to wait (for a book to be delivered online, or for a financial aid check or book voucher to be processed) to access materials, they do not do as well in the course (Agnihotri, Essa & Baker, 2017). The Open University found that educators consider cost and access as critical factors influencing student retention (de los Arcos et al., 2014).

Relevant research has been done on student experience of OER, such as Cooney (2017), which found Health Psychology students at New York City College of Technology (one of CUNY’s senior colleges) held positive opinions of the access afforded by and the quality of OER, but large-scale, cross-campus analysis has yet to be performed. The access to course materials that OER provides is particularly important for students who are commuters, which most CUNY students are. Regalado and Smale’s (2015) research indicates that CUNY students spend 30-60 minutes each way commuting to school, meaning they likely spend “considerably more than the average length reported by NSSE of five hours per week” in transit to school. Students interviewed by Cooney also relayed that access to portable course materials is of crucial import for urban students, juggling multiple responsibilities, including long commutes (Cooney, 2017, p. 175).

The Hewlett Foundation defines OER as “teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use and re-purposing by others” (Hewlett Foundation, 2018). As such, OER does not have a standard look or feel that might be easily compared. Furthermore, an evaluation of Oregon’s OER designation requirement revealed that students do not understand the terminology of OER and have a low awareness of OER offerings at their institution (Freed, Friedman, Lawlis & Stapleton, 2018). No consensus exists nationally regarding how to designate zero textbook cost sections. The recommendation from Freed et al. (2018) is to use a simple phrase that is easy to understand. The phrase “Zero Textbook Cost” is used as a course registration designation across CUNY through the CUNYFirst student information system for course registration. The guidelines developed by CUNY define Zero Textbook Cost course sections as “those that do not require students to purchase a textbook” (CUNY, 2018a). Instructions describe the requirements for ZTC courses and the procedures faculty should follow to have their courses indicated as ZTC via the Registrar’s Office and the Bookstore. At CUNY, a ZTC course may include OER materials, as well as library materials or materials provided for free within copyright by the instructor, or simply be a course that does not have a textbook assigned. For the purposes of this survey and in this article, ZTC is the nomenclature used since it is both more easily understood by students and a more accurate reflection of the diverse materials used by CUNY ZTC course instructors.

Early and Convenient Access to ZTC Materials

When asked when they first accessed the textbook and other materials for their ZTC course, 90% of respondents indicated they accessed materials either before the semester started (20%) or during the first week of classes (70%), as shown in Figure 1. 7% of respondents reported they never accessed their assigned ZTC materials during the course.

Not only did most students access the materials for the course early on, when asked to compare their ZTC class with most other classes they have taken, an overwhelming majority found the materials for their ZTC class easier to access than materials for prior courses, as shown in Figure 2. Of 887 responses to this question, 76% rated their ZTC materials easier to access, with an additional 21% saying they were about the same as accessing materials from most of their previous courses. Only 3% said the ZTC materials were more difficult to access than their previous materials. If we compare these results with the 2016 Florida Virtual Campus study, where 66.6% of students reported never having purchased their textbooks at all, this level of engagement is impressively high. Additionally, the 20% of students accessing their course materials before the semester begins demonstrates to CUNY faculty that our students wish to be proactively engaged with their learning materials.

Students were also given the option to expand on their answer to the question of ease of access if they wished to do so. Their responses were coded for common themes. Of the 188 students who offered an explanation for why they found their ZTC materials easier to access, the most frequent explanation (76 students, or 40%) was convenience and the overall ease of access for ZTC materials. Students appreciate being able to access their ZTC materials anywhere, from any device, at any time, as this student points out:

I could see the course material pretty much anywhere I went, unless I was underground on the subway. I forget everything so I usually forget textbooks, but millennials these days NEVER forget their phones. So I always had my art history readings right in my front pocket!

Forty-three students (23% of respondents on this open-ended question) said their ZTC materials were easier to access because all of the materials were online and/or on Blackboard, while 25 students listed cost as the primary explanation for why they found their ZTC course easier to access. The 3% of students who responded that the materials were more difficult to access gave answers to the open-ended question reporting difficulty with their logins, materials that were slow to load, or a preference for a physical copy.

65% of respondents indicated they did all of the required reading/viewing (including texts, videos, podcasts, etc.) for their ZTC courses. Thirty-two percent said they did some of the required reading/viewing, and only 2.5% said they did none. Taken together, these results clearly show that when students have convenient, easy, cost-free access to their learning materials, they are likely to do more of their assigned reading and viewing work for their courses.

Where Students Accessed Course Materials

We asked students where they accessed ZTC materials and on what devices. Their responses are summarized in Figure 3; 48% of respondents indicated three or more locations while 30% of students reported using only one location. While almost all students, 823 of 890 (92.5%), responded that they did their work at home, there were 1684 responses characterizing ZTC coursework in other places, including 267 responses about doing their schoolwork in transit, on a train or bus. Two hundred and eighteen students reported doing their ZTC course reading, writing, and studying while at their job. Many students reported doing their work somewhere on campus, either in the library (413), in a campus study room (207), in a computer lab on campus (190) or in a common space on campus (200). One hundred sixty five students said they did their work in a public place off campus, such as a Starbucks or McDonalds. This evidence of CUNY students’ mobility and habit of studying on the go is consistent with previous research, and highlights the need for ZTC materials to be portable and mobile-device optimized.

Technology Students Use to Access ZTC Materials

When we combine the results of the location question above (where did you access materials?) with a question asking what technology students used for ZTC access, we find even more evidence about the need for mobility and optimization for delivery across multiple devices. All 890 respondents provided at least one answer to the question, “What technology did you mostly use to access the course materials (readings, textbook, videos, etc.) for this course?” and 66% of responders gave 2 or more answers for what technology they primarily used. Figure 4 shows the 2058 responses students gave to the question.

Students most often reported accessing course materials on their own computers, followed by their phone, and then library computers. These results are consistent with the findings of Smale, Regalado and Amaral (2018), who found that while most students owned smartphones and used them for some of their online and hybrid coursework, they largely relied on their home and campus computers for primary access. They summarized their study respondents’ opinion as “It was clear they considered a smartphone to be insufficient as the sole device for their academic work” (p.8), which is also a fair summary of the present study’s findings. Knowing how and where students access their ZTC materials provides useful data for those allocating infrastructure and tech fee budgets, as well as those designing course materials and/or selecting platforms for delivery.

Digital Learning

While the literature is divided regarding the effectiveness of learning from reading on paper vs. reading electronically (Niccoli, 2015), those who are skeptical about OER frequently express concern that OER means students will exclusively learn with digital materials (Green, 2016). We asked students to rate their own learning with digital materials and print materials. We phrased this question very intentionally to ask students whether they felt they learned as well with digital materials, rather than with OER or ZTC materials, partly because this question speaks directly to questions teaching faculty tend to ask when considering OER, and partly because even in their traditionally-published textbook courses, students frequently turn to digital versions as the cheapest means of obtaining required books. A majority of respondents, 66%, said they feel they learn as well with digital materials as they do with print. Seventeen percent were not sure, and 17% of students reported that they do not learn as well with digital materials.

Printing

Students were asked to report how much of their ZTC materials they printed. This survey item was chosen to solicit additional student feedback on digital and analog methods of accessing the course materials, and is a relevant question addressing faculty concerns about the efficacy of learning with digital versus traditional print resources. While OER licensing permits printing and utilizing materials in formats other than electronic, students do not always have access to a printer and are not always aware that they can print the materials; some OER materials are not printable at all, such as video and audio content. Fifteen percent of respondents reported printing all of their ZTC materials, 43% printed some of their ZTC materials, and 41% of respondents printed none. Respondents who chose to print selected from multiple-choice options to indicate that they primarily wanted to take notes on the page and also that they prefer reading on paper. Even though many students appreciate the convenience and ease of digital materials, student motivations for printing tend to be nuanced and context-specific.

While 41% of students reported printing none of their readings, 43% reported printing some with 15% saying they printed all of their assigned readings for their ZTC course. Students were asked to select as many reasons as they liked for why they printed their ZTC materials, which are summarized in Figure 5. There are important insights to draw from students’ reasons for printing. The top choice at 353 responses is “I wanted to take notes on the page,” with “I prefer reading on paper” a close second at 322 responses. Since skills and tools for reading and annotating online and digital texts exist in a variety of formats, there is an opportunity to improve student learning by expanding faculty support for reading in digital spaces and exploring supporting technology including digital annotation tools.

Student Perceptions of ZTC Courses: Benefits

To understand student perspectives on their ZTC courses, we asked two open-ended questions to elicit feedback about the benefits and drawbacks of the open/free materials used in their course. Among the benefits, ease of access was clearly a major recurring theme in students’ feedback. To provide better context about students’ perceived benefits, each response was coded for one theme.⁶ Saving money and ease of access were the overwhelmingly dominant responses, as indicated in Figure 6.

The majority of respondents (55%) expressed cost savings as a benefit, followed by 27% for ease of access. Representative comments include, “I had access to them whenever I needed them” and “I was able to access the materials any where [sic] I wanted. If I wasn’t home, I was still able to access the materials and do the work I needed to. I was not limited to a textbook and did not have to take the textbook everywhere I went in order to do my work.”

Students utilized the advantages of digital resources, such as the ability to electronically search their materials. Some noted the searchability of the materials as well as the ease of finding definitions of unknown words. These affordances can have positive implications with student populations, and is a significant benefit to students for whom English is not their primary language.

Improved pedagogy was another theme in the open-ended responses about benefits. As one student wrote, “I believe the benefits for having open materials for this course was that I felt the teacher was more engaged with what I was learning and attentive to understanding the written material given to us as students.”

In reviewing the responses to open-ended questions, the issue of stress came up consistently. When asked what the benefits of their ZTC course was, one student said “Less Stress. Much, Much Less Stress. Trust me I can tell I’m in college. I’m always stressed”, and another echoed the sentiment, “With various things in college that can stress you out from relationships to school work, having open/free materials helps mitigate it.” A third student pointed out both the lack of stress and the benefit of being able to retain their learning materials even after the end of the course, stating, “less stress and I now have it forever.”

Several respondents also mentioned the lowered environmental impact of their ZTC courses, as in “Saving money, saving paper,” “Save papers, reduce the weight we carry daily, able to do anything that we want to the content,” “It save [sic] money and save paper,” and “The better for the environment and cheap.”

Student Perceptions of ZTC Courses: Drawbacks

We also asked students to tell us about the drawbacks of their ZTC courses, but in their comments answering this question, approximately half of students responded that there were no drawbacks to the use of ZTC in their course. The other half of students questioned the quality of materials and mentioned their difficulties with the Internet or accessing their course materials, as well as paper and printing concerns, which they also associated with highlighting and note-taking practices. The responses on quality of materials included a wide range of criticism that might be found on any end-of-course evaluation, such as a lack of student-teacher contact, that readings were primarily journal articles and did not provide a more textbook-like overview, or the perennial “too much reading.”

Mirroring some findings from the multiple choice survey questions, a fairly large percentage of drawbacks mentioned (27%) were related to the lack of print materials, as in “I don’t like studying off of a computer,” “I like more printed materials,” and “I prefer physical copies.” Students also noted challenges of highlighting, annotating, and note-taking digital learning materials; this subset accounts for 5% of the 304 actual drawback responses. The nine responses that mentioned eye strain from reading digital materials as a drawback of their ZTC course can be seen as related to the print-related criticisms as well, though they were coded separately. Issues with the technology, devices, or Internet connection required to access ZTC materials accounted for some drawbacks, such as “if there was no Internet access, it was difficult to learn the information” or “if you have no wifi it would be hard to get work done on time.”

Notably, only about a third of the students who took the survey mentioned any kind of drawback to the course at all. This is not surprising, perhaps, given that when students asked if they would recommend a course using zero-cost materials to other students, almost all respondents--95% of 883 responses--said that they would, and only 47 respondents said they would not. Respondents were able to provide an explanation if they wished, and 480 respondents provided an explanation for why they would recommend a ZTC course; these explanations were coded for common themes, which are summarized in Figure 7.

Sixty percent of responses regarding why students would recommend their ZTC course to a friend focused on the cost savings, the high cost of textbooks, or the financial pressures faced by students. Further echoing comments related to benefits of ZTC courses, 23% of responses mentioned the convenience of accessing their course materials from wherever they were as the reason for positive recommendations to friends, and 11% said their positive recommendation would be based on the ZTC materials being good for learning. One student’s response highlighted the benefits of their ZTC materials for students with disabilities:

The material is easier to read it being online or on your phone you can zoom in and out making the letters and print bigger to see. Which is awesome for me because I have vision problems and cannot see tiny prints if they were to be on a regular book or printed out. I can make it the size I am comfortable with reading and do not have to strain my eyes more.

This student saw the flexibility provided by digital materials as a major benefit, because it allowed them to manage their learning in the way that was best for them, without having to disclose their need to a professor or disability services office.

Only 47 respondents of 883 said they would not recommend their ZTC course to friends.

Length of Time

At the time of writing this paper, the survey had run for only one semester, the first full semester of participation in the Open Educational Resources (OER) Initiative. The team will continue collecting results from the survey for an additional 2.5 years, as per IRB approval. During that time, we expect the results to remain largely static, but additional research will be done to see if this prediction bears out. The open-data set will be updated at the end of each future semester that the survey runs.

Inconsistent Faculty Participation

As described, the survey was distributed only to students enrolled in the OER course sections associated with the OER grant. Rather than distributing the survey through a faculty blast, or another non-specific distribution method, the researchers sent the survey to OER coordinators at each campus, who distributed the survey to the faculty course designers and section instructors involved in the grant, who then shared it with students. This delivery method was logistically feasible for a survey on such a large scale, and allowed us to increase the likelihood that only students who had actually taken a ZTC course would take the survey, but it did mean that we depended on faculty to offer the survey, which they may or may not have done. Future plans for distribution include face-to-face survey tables in libraries, and more collaboration with faculty stakeholders across the CUNY system.

Sample Size

This study was implemented with a convenience sample, though in this case purposefully so. The population was intentionally limited to students enrolled in OER courses associated with the OER grant so that we could study the targeted effects of the grant and how using OER materials instead of commercial materials affected our students’ experience. The issues regarding inconsistent faculty participation created a situation where we did not reach every single student, however, the number of responses (890) captured during this first semester of study demonstrates the value of the findings.

Lack of Control or Comparison Group

Many of the questions in the present survey ask students to compare their experiences in ZTC classes to their own experiences with commercial materials. A future research initiative will compare the experience of students using OER and ZTC materials to that of students in a control group. However, the present findings are valuable as a measure of student experience with an innovative practice.