Sociology Course in Canada : Student Views and Completion Rates

Open educational resources (OER), including open textbooks, are free, adaptable learning resources. The integration of these materials in place of commercial textbooks allows for considerable financial savings for students and creates opportunities for more active and engaged learning. The growing interest in the use of OER at a Western Canadian university led to the chance to survey students for their feedback on using OER instead of traditional commercial textbooks. This paper focuses on the views of students in an introductory sociology course for which an instructor adopted an open textbook and otherwise left the course unchanged from when it was taught with a traditional textbook. In addition, completion rates for the offerings with the open textbook are compared to previous offerings with a commercial textbook.


Introduction
The cost of textbooks for post-secondary education in the United States rose more than 1,000% between 1977 and 2015, a rate three times that of inflation (Popken, 2015).As a result, students appear to be less likely to purchase required resources and more likely to register in fewer courses or drop courses because of the cost of these materials (Florida Virtual Campus, 2016).In an effort to deal with this problem, and improve learning outcomes, post-secondary institutions throughout North America are exploring the use of open educational resources (OER) in place of commercial textbooks.
The United Nations Educational, Scientific, and Cultural Organization (UNESCO) defines OER as "any type of educational materials that are in the public domain or introduced with an open license.The nature of these open materials means that anyone can legally and freely copy, use, adapt and re-share them.OERs range from textbooks to curricula, syllabi, lecture notes, assignments, tests, projects, audio, video and animation" (UNESCO, n.d.).
Not only does the integration of OER in place of commercial textbooks save students money, but the nature of the licensing of these resources allows educators to modify them to meet local needs.Textbooks can be shaped around the course instead of the other way around.The potential benefits of OER go beyond those that directly affect learners and instructors, but also extend to the institution as a whole.If students can afford to enrol in and complete more courses, universities and colleges may retain more tuition dollars.As Wiley, Williams, DeMarte and Hilton (2016) note, given that most costs associated with teaching courses are fixed, if using OER in more courses could lead to students being able to take more courses or drop fewer courses, this could potentially provide enough tuition revenue to offset the cost of adopting and sustaining OER in those courses.This article will explore the integration of OER at a public university in Western Canada, focusing on a first year sociology course.Researchers were able to compare iterations of this course with an instructor using a commercial textbook and those by the same instructor using an open textbook in a later term.The results show promise, which is encouraging as the adoption of OER in higher education throughout North America and in other parts of the world continues to grow.

Impacts of commercial textbook costs
In Florida, surveys were conducted in 2016 with over 20,000 post-secondary students in 40 institutions.The aim of the research was to determine the effect of textbook costs on the students' access to post-secondary education as well as on student achievement.The surveys showed that the rising cost of these materials was negatively affecting student outcomes: students reported lower grades and a reduced ability to enrol and complete courses in a timely manner that would enable them to finish their programs.The researchers found that 67% of students reported not purchasing a required textbook for a course at least once, 46% indicated that they avoided taking certain courses because of the cost of the required textbooks, while 48% said that they had to register for fewer courses because of this expense (Florida Virtual Campus, 2016).In addition, 46% of students said that they had at some point either dropped or withdrawn from a course due to the cost of the text, and 38% of students responded that due to their inability to afford the text for a course, they had earned a lower grade in the course.
Researchers in Canada have also found post-secondary students reporting similar problems with the costs of commercial textbooks.Jhangiani and Jhangiani (2017) conducted a survey of 320 undergraduate students in 12 post-secondary institutions in British Columbia, in which 54% of students said they had decided not to purchase a required textbook for a course due to cost, at least once.Students also reported doing the following things because of textbook costs (either rarely, sometimes, often, or very often): 26% said they had chosen not to register for a particular course, 27% had taken fewer courses, 17% had dropped a course, and 30% reported earning a lower grade in a course.The authors note that those who reported working more hours outside of their studies were more likely to report dropping courses and earning lower marks.
In a smaller survey of students in a first-year, undergraduate Physics course at the University of British Columbia Vancouver, Hendricks, Reinsberg and Rieger (2017) reported fewer percentages of students taking similar actions due to textbook costs.Out of 150 respondents, 57% said they had not purchased a required textbook for a course at least once.In addition, 25% said they had rarely, sometimes, often or very often chosen to register in a different section of a course because of costs, 19% said they had taken fewer courses, and 16% said they had dropped or withdrawn from a course (the survey did not ask students whether they had earned a lower grade in a course).It is important to note that the respondents in that survey were mostly first year students (85%), and thus probably did not have much post-secondary education experience to report on (though they were surveyed at the end of their first year).
Research has shown that instructors are increasingly recognizing that students have difficulty affording textbooks (especially given the rising cost of tuition and housing that many students face), leading to a growing interest in OER (Harley, Lawrence, Acord & Dixson, 2010;Petrides, Jimes, Middleton-Detzner, Walling & Weiss, 2011;Pitt, 2015;Belikov & Bodily, 2016;Martin, Belikov, Hilton, Wiley & Fischer, 2017;Ozdemir & Hendricks, 2017).Open textbooks in particular have been a key element in lowering student costs.The open textbook project from British Columbia's BCcampus, the leader of the OER movement in Canada, has resulted in students in British Columbia saving approximately $9 million since that projects inception in 2012 (BCcampus, n.d.).

Student perceptions of OER
Multiple studies show that most students using open textbooks or other OER believe they are of equal or better quality than commercial textbooks (see Hilton, 2016 for a meta-analysis).In a large survey of 1299 students in two first-year Biology courses at the University of Georgia that used an OpenStax open textbook, Watson, Domizi and Clouser (2017) report that 64% of respondents rated the quality of the open textbook about the same as other textbooks they had used, and 22% rated it higher.Clinton (2018) reports on a survey of students in two versions of an introductory psychology course taught by the same instructor in the U.S.: one using a commercial textbook and another in a later term using an open textbook.Students rated the quality of textbooks similarly, with visual appeal being rated slightly more highly in the commercial textbook, and the quality of writing being rated slightly more highly in the open textbook.
In the Canadian post-secondary education context, Jhangiani and Jhangiani (2017) reported that, out of 320 undergraduate students across British Columbia using various open textbooks in their courses, 33% rated their open textbooks as of "average" quality, while 63% rated them above average or excellent.When asked if they would have preferred to purchase a traditional textbook for their course, 56% of students in this study disagreed slightly or strongly, and 24% neither agreed nor disagreed (20% agreed slightly or strongly).According to Hendricks, Reinsberg and Rieger (2017), 93% of 143 students in a first-year Physics course in British Columbia said that the quality of their open textbook was about the same (72%) or better (21%) than that of textbooks used in their other courses.The majority of students (64%) disagreed somewhat or strongly that they would have preferred to buy a traditional textbook for the course, with 18% neither agreeing nor disagreeing (18% agreed somewhat or strongly).
Finally, Jhangiani, Dastur, LeGrand and Penner (2018) surveyed 178 students in seven sections of an introductory psychology course in British Columbia, some of which used a commercial textbook while others used an open textbook.Across the different sections, students using the open textbook (in print or digital format) rated it the same or better as those using the commercial textbook, across multiple dimensions, including clarity of writing, helpful examples, visual appeal, and more.
The present study also considers students' ratings of the importance of various non-content features of their open textbook, following up on similar research by Jhangiani and Jhangiani (2017).The latter found that the following were rated by students as very important or absolutely essential: immediate access (70%), cost savings (68%), convenience/portability (54%), ability to print (41%), ability to share the book with others (34%).They also reported that most students accessed their open textbook in digital format, with 72% downloading a PDF, 50% reading it online and 43% using a print version.

Efficacy of OER (grades, persistence)
Several recent studies have examined the efficacy of open textbooks across multiple sections of the same course, finding that in most sections of a course where an open textbook was used students performed at least as well academically as those in the sections using a commercial textbook.Colvard, Watson and Park (2018) evaluated student grades (including DFW-grades of D, F, or Withdrawal) in eight courses at the University of Georgia taught by the same instructors over multiple semesters, including those where traditional textbooks were assigned and those using open textbooks instead.They found that across the courses there were higher grades, including lower DFW, for the sections using the open textbooks than for the previous sections that did not.The reduction in DFW was highly significant for students eligible for Pell grants (a marker for financial need), non-White students, and part-time students.Clinton (2018) studied 520 students in an introductory psychology course taught by the same instructor using a commercial textbook in one semester and an open textbook in another.Course grades were slightly better for the students using an open textbook, though Clinton notes that the average high school GPA for that section of the course was also higher.There was a statistically significant reduction in students withdrawing from the open textbook section, even though the high school GPA's for the students who withdrew in the two semesters were similar.
Allen, Guzman-Alvarez, Molinaro and Larsen (2015) focused on Chemwiki, an open textbook in the format of a wiki.They compared academic performance across multiple assessments (two midterms and final) between two sections of the same chemistry course (one using the ChemWiki and the other using a commercial textbook), with both having the same instructor, teaching assistants, questions, and rubrics.The researchers found no statistical difference between the academic performance on these assessments between the two sections.
In Fischer, Hilton, Robinson and Wiley (2015), the researchers looked at overall academic performance and completion rates for 15 courses across multiple institutions, comparing sections of a given course (at the same institution), with at least one section using a commercial textbook (the control group) and at least one section using open educational resources (OER) (the treatment group).In nine of the courses, there was no statistical difference between the academic performance of students in the control group compared to the treatment group.In five of the courses "students in the treatment condition were more likely to pass the course than students in the control condition" (Fischer et al., 2015, p. 167).Only students in one course in the control group were more likely to pass and had higher marks than those in the treatment group.The treatment section, however, did have a significantly higher completion rate than the control group.
In Canada, Jhangiani et al. (2018) reported that students in sections of an introductory psychology course using an open textbook scored higher on one of three non-cumulative exams than those in sections using a commercial textbook.There were no significant differences on the other two exams.Controlling for instructor difference, the same result was found for sections taught by the same instructor when using a commercial and an open textbook.
Most studies of student perceptions and efficacy of open textbooks in colleges and universities are from the United States; the present study adds to and builds upon the relatively small amount of literature on these topics so far in Canadian post-secondary education.

Context for the present study
The University of Saskatchewan is a medical doctoral university in Western Canada.The first large adoptions of open textbooks were in 2015, in an economics course and a chemistry course, Other adoptions followed that year and the number of students benefiting from the integration of open textbooks rose to more than 900 for the 2015-2016 academic year.That same year the provincial government began providing a small amount of money to the province's three largest post-secondary institutions (University of Saskatchewan, University of Regina, and Saskatchewan Polytechnic) for the purpose of creating or adapting open textbooks and other OER.This funding was used in 2016, for the first known adaptation of an open textbook at The University of Saskatchewan.Graduate students were paid to adapt an existing open textbook for a first year course in the Edwards School of Business.From 2016-2017, over 2700 students benefited from the open textbook initiative at the University of Saskatchewan, in 22 courses (total number of students in courses, not unique students).
Given the increase in the number of students benefiting from the use of OER at the University of Saskatchewan, the growing interest in the use of OER among instructors, and the need for ongoing support from the institution, the centre for teaching and learning decided to begin evaluating the initiative by surveying students about their views on the open textbooks being used in their courses.We adapted a survey based on one by Bliss, Robinson, Hilton and Wiley (2013) and during the 2016-2017 academic year we requested that instructors using open textbooks distribute the link to the online survey to the students in those courses.While most instructors agreed to do this, the response rates were very low in all but one course, Introduction to Sociology.
The instructor for this introductory sociology course had taught the course for several years prior to adopting the Introduction to Sociology -2 nd Canadian Edition (Little, 2017) open textbook from BCcampus in the Fall of 2016.This provided the opportunity to not only survey students to obtain their views of the open textbook, but also to compare overall course grade averages and completion rates with previous offerings of this same course taught by the same instructor using a commercial textbook.

Methods
At the end of 2016, after the term in which students used the open textbook, the instructor emailed the survey link to the students in both sections of the Introduction to Sociology course, and sent periodic reminders during the month that followed.The email and preamble to the survey both stated that completion of the survey was voluntary, and that the researchers and the instructor valued their feedback about the open textbook used in that course.
The survey consisted of 14 items, a combination of multiple-choice, likert-style, and open-ended questions.It asked for students' views of the quality of the open textbook, by asking how they would rate the quality of the open textbook compared to other textbooks they've used previously, and whether they would have preferred to purchase a commercial textbook for the course instead of use the open textbook.Students were also asked to rate the level of importance to them of the following features of the book: cost savings, immediate access, convenience & portability of the digital format, ability to print pages, ability to keep forever, ability to share with others.There were also open-ended questions asking students to explain their ratings and whether there are any other features they liked or disliked.Finally, there were questions about the format in which students read the textbook (e.g., PDF, epub, print), and whether they were aware of the printing services offered by the campus bookstore.
Of the 343 students who completed the course in the Fall of 2016, 129 answered some survey questions and 119 answered all survey questions, for a response rate of 35%.Once term grades were submitted, researchers were granted access by the instructor and Department Head to overall course grade averages and completion rates for both sections, and for the sections the instructor had taught during the previous two years for comparison.

Survey results
The survey asked students to rate the quality of the textbook compared to traditional commercial textbooks they had used in other courses.Students gave the open textbook high marks, with 72.8% rating it as excellent or above average compared to traditional textbooks they had used in other courses (Table 1).It is important to note that as this is an introductory course; most students were likely in their first or second year of study and may not have yet had many commercial textbooks to compare to the open textbook in sociology.
Students were also asked to rate their agreement with the statement, "I would have preferred to purchase a traditional textbook (at approximately $100) for this course."The vast majority of students indicated that they would not have preferred to buy a commercial textbook for this course, with 82.7% of respondents saying that they either "strongly disagreed" or "slightly disagreed" with that statement (See Table 2).The $100 figure, which is used by BCcampus, the U of S, and other institutions and organizations working with open textbooks is an estimated average that each student in a class saves when OER is adopted in place of a commercial textbook.As this survey was sent to students in multiple courses, the $100 was used for all courses regardless of the actual price for the books being replaced by OER.Instructors of the various sections of the introductory sociology course at the U of S have the freedom to choose their own book, with prices for the required commercial textbooks ranging from $85 to more than $150 in Canadian currency.One survey question asked whether certain non-content features of the open textbook were important to students.Overwhelmingly, they indicated that cost savings from the use of an open textbook was important, with 89% saying that this was either "very important" (74%) or "moderately important" (15%) to them.Immediate access was rated by 84% of participants as very or moderately important, with 72% saying the same about the convenience and portability of the digital format.The ability to print the materials was considered very or moderately important by 56% of those who answered the question, while 54% indicated this was the case for the ability to share the materials with others.
Open textbooks are often available in multiple digital formats.Introduction to Sociology -2 nd Canadian Edition is available to download as a PDF, ePub file (for use on many eReaders), or Mobi file (for use on a Kindle), or can be read as a website.The freedom of the open license also allows for the book to be printed.Students were asked which format they preferred for using the open textbooks.Most students (57.9%) indicated that they downloaded and read from the PDF, with another 18.2 percent saying they read the book straight from the website and 15.7% saying they had used a printed version.
The University of Saskatchewan bookstore is enthusiastically involved in the OER initiative at the university and offers a print-on-demand service for open textbooks, partnering with the University of Saskatchewan student union's print shop to offer the service.Students can email a particular contact in the bookstore with the course code and instructor name and a copy of the book will be printed and hole punched mush as a course pack would be, and is generally available within 48 hours.We suggested that instructors using open textbooks in their courses notify students of this service at the start of the term.
One of the survey questions was, "Were you aware of the printing services offered by the University of Saskatchewan bookstore for open textbooks?"Of the 122 students in the introductory sociology course who answered this question, 68.9 percent answered "yes."Eleven students indicated that they had used the service for a course using an open textbook (not necessarily this course).Finally, students were asked, if they did print the open textbook for the introductory sociology course, did they print it all at once or chapter or pages as needed.Seven students said "all at once", 18 students said "chapters or pages as needed", and 94 (79%) said this question did not apply to them.

Course grades and completion rates
The researchers also considered the overall course grade average and the completion rate for the introductory sociology courses, for the term in which an open textbook was assigned, and two previous terms as well (all taught by the same instructor).As this course uses normative grading, however, looking at course averages is not a good indicator of whether student learning is affected by the use of the open textbook.It is not surprising that the course average did not change significantly across the two terms when a commercial textbook was used compared to the term when the instructor assigned the open textbook instead.As shown in Table 4, there was no statistical difference between the overall course averages in these courses.
We did not have access to withdrawal rates for this course, so we determined completion rates for these classes by dividing the number of students who were enrolled on the first day of class by the number of students enrolled on the last day of classes for the term.Table 5 shows the details for the three terms for which we examined the data.The course taught by the instructor using the open textbook in the fall of 2016 had a statistically higher completion rate compared to the previous two times that she taught the course.We conducted a one-tail Z-test to determine these results (z=2.53 and p=.0057).This is an important indicator as The University of Saskatchewan charges tuition based on the number of courses in which students enrol, and students are eligible for refunds or partial refunds of tuition for several weeks into the term.
Columbia who reported using a print version of their open textbooks, reported in Jhangiani and Jhangiani (2017).
Regarding efficacy, though there was no significant difference in end-of-course grades between the students in the Introduction to Sociology course using a commercial textbook and those in the same course using an open textbook, the fact that this course uses normative grading means we cannot draw a conclusion that the open textbook had any effect on course grades.However, there was a statistically higher course completion rate in the term that the instructor used the open textbook compared with previous terms with a commercial textbook.

Conclusion
This study adds to and supports the literature on student perceptions and efficacy of open textbooks in the Canadian post-secondary education context.We were able to compare grades and completion rates for the same course taught by the same instructor over three terms, two using a traditional textbook and one using an open textbook.To our knowledge, there are only two other studies on open textbooks in Canada that report on efficacy (Hendricks, Reinsberg & Rieger, 2017;Jhangiani et al., 2018) with only one of those (Jhangiani et al., 2018) comparing the same course by the same instructor taking place across more than one term.
While the results of the survey and the course completion rates are encouraging, they are limited to one term of one course, making it difficult to make any broad assessments as to whether students prefer the open textbooks in general to commercial texts due to cost, perceived quality, or both, or if completion rates in other courses at the University of Saskatchewan would also rise with the change to an open textbook.Both of these questions require further study.Unfortunately, the instructor for the introductory sociology course is no longer teaching and currently no other instructors in that department are interested in using the open textbook so we are unable to study that particular course moving forward at this time.
As the open textbook initiative at the University of Saskatchewan continues and more faculty members adopt or adapt open textbooks, further courses will be studied with similar measures as have been used here.While the survey response rates among students in courses other than the introductory sociology course described in this paper were low, an effort to increase the response rate for the student surveys will be made by including a chance to win gift cards.
We will also look for another course where an instructor is adopting an open textbook for a course they have previously taught with a commercial textbook to look at possible changes in grades and completion rates for other courses.If completion rates are higher in courses using open textbooks, this could provide an important incentive for the university to further invest in the OER initiative, as it may help increase revenue.