Institutions of higher learning are using innovative options to meet the needs of their diverse student population, many of whom struggle academically, manage a disability, or have learned English as a second language (ESL). When educators rethink and retool their traditional lecture methods, all students can ultimately benefit. Questions emerge during the process, such as “how do we teach?” and “what should 21st century higher education look like?”
As educators who support students through critical writing processes, we strive to come down from the traditional “sage on the stage” role to become the “guide on the side” (King, 1993, p. 30). As distance learning becomes more prevalent, online courses utilize carefully designed learning management systems (LMS) rich with multimedia, accessible resources, authentic activities and assessments. Despite the many learning successes these environments have supported, more can be done to increase the number of students who meet course expectations in a positive learning environment.
Technological applications and add-ons outside of the LMS are an essential aspect of identifying potential responses to the questions posed above. Many tools surface that appeal to diverse learning styles and allow for enhanced skill development, easier research and greater understanding of assignments. Miller (2014) reported that “technology allows us to amplify and expand the repertoire that effective teachers use to elicit the attention, effort and engagement that are the basis for learning” (p. 11). In addition, Tcherepashenets (2015) observed the “liberating” power of technology lies “in its ability to provide personalized, individualized approach to learning” (p. 258). Equally important, individualized feedback is a significant part of the personalized approach implemented by innovative educators in the 21st Century.
The purpose of our study was to examine tools for individualizing feedback that have the potential to unleash the “liberating power” identified by Tcherepashenets (2015, p. 258). Our primary research question was “what is student perception of formative feedback when audio is added to traditional text?” We chose Kaizena and Adobe Acrobat Reader as two available technologies to implement in our exploration. After careful experimentation, we regularly used them in several instructional scenarios and surveyed our students to learn more about their experiences. Mainly, we wanted to know how easily students could access the audio feedback and if they had a preference for formative audio feedback over text alone.
Though feedback is a complicated issue with many variables, the value of formative feedback for students is well established (Shute, 2008). In contrast to group feedback, or grades alone, individualized formative feedback provides specific comments that highlight the strengths and weaknesses of student work. Moreover, numerous studies indicate that grades alone do little to promote learning improvement (Campbell & Cabrera, 2014; Kohn, 2011; Pulfrey, Buchs, & Butera, 2011). Individualization and attention to quality take learning further, regardless of the instructional mode. Whether feedback is face-to-face, synchronous, or asynchronous, attention to the affective domain using positive commentary is necessary to ensure that a student will persist in his or her efforts to learn (Bastian, 2017). Non-confirming feedback can be effective as well if it contains specific corrective information. For example, praise for the student’s personal attributes without specific examples should be avoided (Hattie & Timperley, 2007). One of the many recommendations from Shute’s (2008) extensive review of formative feedback is to take advantage of the potential of multimedia in crafting such feedback to minimize cognitive overload (p. 179). Johanson (1999) found that voice comment feedback on written assignments was a welcome addition to the practice of student-teacher conferences because of its ability to personalize instruction. Shute’s (2008) multimedia recommendation may enhance the feedback process substantially in distance learning, where face-to-face conferencing around individual assignments and tutoring is often impossible.
One of the greatest challenges for learning institutions and instructors when designing and implementing online courses is to “provide a sense of community with constructive feedback and provide open forthcoming communications as well as recognizing membership and feelings of friendship, cohesion, and satisfaction among learners” (Desai, Hart, & Richards, 2008, p. 333). Walther, Anderson, and Park (1994) determined as early as 1994 that “when cues are filtered out, communication becomes more task oriented, cold and less personal [online] than face-to-face communication” (p. 461). Casey (2008) also reported that “one of the major complaints about computer-mediated communication in general is the lack of social cues” (p. 50).
More specifically, a goal of utilizing enhanced feedback may be to establish student perception of social presence. Short, Williams, and Christie (1976) defined social presence as “the degree to which a person is perceived as a real person in a mediated situation” (p. 427). When an asynchronous instructor is available in the form of a recorded voice, feedback is more easily understood to be coming from a “real” person. Therefore, a lack of social presence might affect learners’ performance and outcomes because they don’t recognize instructor feedback as real or genuine. Social presence has also been defined as “the perceived presence or salience of others in online discussion” (What is Social Presence?, n.d.). Desai, Hart and Richards (2008) observed, “…social presence is a strong communication component that reduces isolation between the distant learner and other learners and instructor” (p. 328). Ice, Curtis, Phillips and Wells (2007) found that development of social presence was one of the key advantages of using audio feedback. In their study,
“over 450 students in courses taught by these instructors have now received audio feedback. According to these instructors, approximately one third of their students have submitted unsolicited feedback expressing a strong preference for this technique over text-based feedback. No negative feedback has been received” (p. 19).
The authors noted that such a preference was “significant” (p. 18).
Aragon, Johnson and Shaik (2002) and Young (2006) established that student outcomes in well-designed online courses are generally similar to face-to-face classes. Indeed, Driscoll, Jicha, Hunt, Tichavsky and Thompson (2012) concluded that the quality of learning is not determined by the medium, but the course delivery pedagogy which includes feedback modalities. Instructional feedback can be as brief as a grade, or as extensive as a page of paragraphs describing the student’s strengths and weaknesses in assignment efforts. According to Ice et al. (2007), students receiving voice feedback remotely feel more involved because of a perceived decrease in transactional distance, as first identified by Moore (1997). At the onset of online education, Moore addressed the problem of the separation caused by distance between learners and instructors. He concluded, “there appears to be a relationship between dialog, structure and learner autonomy” (2007, p. 24). He further observed the benefit of increasing dialogue through the use of teleconference. At the time, teleconferencing was not easily available to students and instructors at a distance.
Student-instructor interaction has been shown to be the most important factor in student satisfaction with online learning in an investigation by Eom and Ashill (2016). When audio feedback is used to supplement feedback in the face-to-face classroom, student learning is enhanced with the additional advantage of being able to listen more than once (Johanson, 1999). Sloan, Stratford and Gregor (2006) maintain that students who need academic support will benefit when there is more than one mode to convey information across learning mediums. They especially noted the benefit of using voice comments along with video support as an opportunity for distance educators to meet diverse learning needs in a personalized manner that increases social cues and personal interaction (Sloan, Stratford & Gregor, 2006).
Students perceive and implement audio feedback in different and more meaningful ways than written feedback. Why? 1) it is easier to understand because handwriting is often illegible; 2) it has more depth because possible strategies for solving problems are included and the instructor can exhibit caring more easily (Merry & Orsmond, 2008).
Before the advent of online learning, Pearce and Ackley (1995) explored a number of studies of audio feedback; they were unanimous in agreement about their effectiveness and positive results. Clark recommended audiotaped feedback as “an alternative to personal conferences” not only because it saved time, but also because it is “more effective than sprinkling the essay with undecipherable, anxiety-producing marks” (qtd. in Pearce & Ackley, 1995, p. 122). They also reported that instructors recorded their oral assessments via taped recordings and sundry methods. Dental students and 9th graders, undergraduate business students and graduate students all expressed strong preferences for voice comments. Although it took a bit of time to refine their skills, instructors found that the method of recording lessons on cassette tapes was “adaptable and can be focused on important features about responding to writing…” (Pearce & Ackley, 1995, p. 32). This expanded on the earlier finding by Kirschner (1991), whose research found that the time difference spent recording the feedback compared with text was minimal, but the time instructors spent in preparing for it was significant given the available technology. Later studies found such preparation investment to be worthwhile, even as advancements in technology reduced teacher training. As recently as 2015, trends were changing: McCarthy (2015) found that “it took slightly longer” to provide written feedback than to record audio feedback (p. 166).
Klammer (1973) and King, McGugan and Bunyan (2008) testified to satisfaction on the part of students receiving recorded feedback via qualitative data. Benefits identified by the instructors included: a) time saved because speaking is faster than writing; b) avoidance of stress related to structuring a written argument; c) softened criticism; and d) encouraging tones employed. Student observations have also been positive: less cryptic feedback; motivating vocal intonations; and greater permanence compared to physical meeting outcomes (Kirschner, 1991). Pearce and Ackley’s (1995) studies confirmed similar revelations and findings. With voice comments, the tone and inflections provide additional meaning. For example, if a student writes a paragraph with two direct quotes, but cites only the first, the instructor/tutor could reply, “What is the source here?” Depending upon how the words “source here” are uttered, this could be interpreted in the text as an exasperated tone, or an acknowledgement that the instructor/tutor is sincerely interested. Wolsey (2008), found evidence to support that students prefer formative feedback embedded in their written work rather than a separate document. Research by Ice, Swan, Diaz, Kupczynski and Dagen (2010) substantiated that there is student preference for diversity in feedback since
“a marked majority or a plurality of respondents indicated a preference for a combination of written and audio feedback. However, it should be noted that at the micro level, the preference for written feedback increased significantly, indicating that perhaps a small amount of audio and a large amount of written feedback is most effective…” (p. 126).
Among the pedagogical approaches recommended by the National Postsecondary Education Cooperative (NPEC) is the encouragement of “various forms of electronic technologies” (Kuh, Kinzie, Buckley, Bridges & Hayek, 2006, p. 67). The addition of recorded voice technology supplies another mode of educational technology to help all students discover their strengths and weaknesses with an advantage over face-to-face conferencing: hearing the feedback more than once. Student preferences and ability to make use of the varied feedback is of interest for all educators but holds special application for educators of adults. As Knowles (1980) observed,
“…the main thrust of modern adult-educational technology is in the direction of inventing techniques for involving adults in ever-deeper processes of self-diagnosis of their own needs for continued learning, in formulating their own objectives for learning, in sharing responsibility for designing and carrying out their learning activities, and in evaluating their progress toward their objectives” (p. 56).
According to Olesova, Weasenforth, Richardson and Meloni (2011), asynchronous audio feedback is highly effective for ESL learners in addition to native speakers. In teaching English composition to ESL students, Johanson (1999), described using audio files as “Rethinking the Red Ink” to help students construct meaning that complements face-to-face conferencing regarding written assignments (p. 33). In the process of creating the audio file, Johanson (1999) found he was becoming more of a “coach” as if in an office conference. In addition, as noted above, audio files can be played back more than once to allow multiple reviews, which is a distinct advantage over an in-person office conference.
By using voice comments, instructors provide another platform outside of the LMS to expand student learning and assessment experience. While the addition of voice comments to text would seem to be appreciated and desired, according to our review of the literature, there is a need for more empirical evidence to explore student reasoning for this preference, including perhaps a preference for a certain tool over another. Noting strategies to improve online student’s writing skills, Straub and Vasquez (2015) used the advantages of Google Docs and Adobe Connect (voice conferencing application) for synchronous instruction, without adding an asynchronous voice feedback option. However, as the technological options for sharing audio comments expands rapidly, our research focuses on the use of two programs that have proven suitable for asynchronous academic applications: sound files in Adobe Acrobat Reader and Kaizena voice comments (Skylar, 2009; Trust, 2018).
Kaizena and Adobe Acrobat Reader embed feedback by the use of recorded audio voice comments linked to specific ideas or writing style issues within the text. In addition, they can provide nuances linked to instruction that are most easily communicated through voice while decreasing the transactional distance between the student and the instructor. Both tools have the potential to provide feedback containing a more personal tone and positive emotion. With training, feedback via these tools may also be delivered more rapidly and include additional information without increasing the time spent in the process.
Our study was partly designed to address recommendations made by Delante (2017): “another stimulating research direction to take is to do a comparative study between online written feedback and voice/video or chat feedback” (p. 28). We initiated the study with the assumption that all surveyed students had experienced written feedback at this point in their educational studies. Therefore, student opinion was solicited on their use of one of two types of recorded audio feedback for improving writing skills within college writing courses. Along with students in regular academic online classes, students at the college who sought writing assistance through the Office of Academic Support were also asked to participate.
The three primary investigators collaborated, offering formative feedback to students between January 2016 to December 2017. The investigators used one of two technologies during the student writing process: Kaizena voice comments or Adobe Acrobat Reader, in addition to text-based feedback. As students completed the term in which they received audio feedback, they were sent a survey using Google Forms. A total of 125 students were sent the form as part of the study. The results of all survey respondents (n=44) were compiled at the end of the research period. Although differences emerged as we used them, our primary research question could be answered by students exposed to either technology: “what is student perception of formative feedback when audio is added to traditional text?”
Adobe Acrobat Reader is a program for opening and reading PDF files and has proven to be useful in sharing one-way audio feedback files with students. The program is a free download that allows limited capabilities without purchase of the full Adobe Acrobat program. No account set-up or additional online hosting is required. After an instructor saves a submitted student document as a PDF file and opens with Adobe Acrobat Reader, recorded comments may be inserted where desired. The comments are represented by small speaker icons throughout the written assignment. When saved, the document and audio files become a single packaged document and may be shared back as an attachment via email or LMS.
Kaizena, a web-application and add-on for Google Drive, evolves and enhances the interaction between student and instructor. Kaizena voice comments are based on “conversations” around a text which is easily available and linked within an email notification. As a result, a two-way dialogue becomes accessible online, replacing traditional face-to-face office hours as a writing assignment is reviewed in-depth. Instructors initiate feedback via text and/or audio attached to a document and students may respond in either format as well.
Kaizena audio feedback tools use the SAMR Model (Puentedura, 2014), providing four modes of learning, while Adobe Acrobat Reader features modes one, two and four without conversational replies.
Subjects (n=125) were undergraduates in a variety of regular academic online classes at SUNY Empire State College. The participants included international and stateside students, many seeking writing assistance through the school’s Office of Academic Support. Three investigators sent email invitations to students asking them to complete a voluntary online survey created with Google Forms regarding audio feedback on an assortment of written assignments. All participants were enrolled in one or more online courses.
The survey was developed using a Likert Scale with six questions for each, ranging from two to five choices. Of the total, 12 students were identifiable as ESL. The survey was anticipated to take about ten minutes to respond to questions. No compensation was offered, 44 students responded, and 64 students did not respond. Seventeen of the 64 non-responses were the result of invalid email addresses. A second request was sent to the valid email addresses for those who did not initially respond. Our final response rate was 35%. With the use of three investigators sending surveys to students regarding feedback on written assignments for a variety of online courses, our study incorporated both investigator and environmental triangulation, as defined by Guion (2002).
The following charts represent the results of survey questions regarding student feedback preference received between January 2016 and December 2017.
The data in Figure 1 illustrates the number of times that the voice comment tool was accessed by survey respondents. Some respondents became more familiar with the technology than others, as access increased. Students who received voice feedback more than once were also more likely to participate in the survey over students who received voice feedback only once.
Figure 2 shows that of 44 students who responded, 13.3% used Adobe Acrobat Reader sound files. A larger number (80%) of respondents accessed Kaizena voice comments. A small percentage (6.7%) could not identify the program utilized.
Figure 3 shows the majority of students accessing voice comments (55.6%) found the process easy or mostly easy. A much smaller percentage (13.3%) found the process difficult. As we teased out more detailed responses from the survey data, we found that 38 of the 44 respondents identified as Kaizena users while 6 identified as Adobe Acrobat Reader users. While the Kaizena user experience was similar to the chart below (Figure 3), the smaller number of Adobe Acrobat Reader responses skews their experience when reported as percentages. Of the 6 Adobe Acrobat Reader users, 3 reported difficulty with the process. (See Appendix 1: Ease of Access and Student Preferences).
Figure 4 indicates most students who accessed the voice comments did so with a PC or Mac computer or laptop. A much smaller percentage used an alternative personal electronic device. The identified iPad user was not able to access Kaizena via the device.
Figure 5 suggests there may be an incompatibility with using the Chrome browser to open up the Adobe Acrobat Reader PDF containing voice comments, via certain online Learning Management Systems. Four of the Adobe Acrobat Reader users could not access their audio feedback via the LMS at all using Chrome. Saving the file and opening it outside the browser with the Adobe Acrobat program provided a workaround but increased the steps and complications for feedback access between these programs.
Figure 6 indicates a strong preference for personalized formative audio feedback specifically related to students’ written work. Of the 44 students who responded, 66.7% preferred audio feedback to accompany their text-based feedback, while 33% preferred written feedback alone.
Our study included an optional written feedback section for comments (Appendix 2: Qualitative Responses).
Examples of positive views found in the comment section of the survey:
I thought it was great. I had a choice of recording or writing a comment. I was also able to see what specific areas you were referring to.
Its great. The fact that I can hear your voice makes online courses that much more interactive.
Ok I just did it and it was a lot better than reading the comments on the side it worked well and I more fully understand the comments that you made!
I loved working with the Kaizena App and hearing your remarks one-by-one was so useful when making corrections to my paper.
The voice message is understood better by me. English is my second language and the wording could be slightly difficult to understand.
Actually felt more like one-on-one learning.
Examples of negative views:
A total of three students noted that it would be easier if the voice comments were embedded in the document.
I prefer written comments because it is easy to edit my work as I read through.
It is hard to edit my work while listening to voice comments.
Would prefer if voice appeared in the Google Doc.
Note: Since these results were compiled, the Kaizena capabilities have extended to include embedded voice comments.
Several negative responses complained about the “glitches” or lack of clear audio:
Some were difficult to understand.
Speakers can become muffled.
Another student noted “glitches” as well as the following addition about providing both text and voice comments: “a bit cumbersome, but both are appreciated.”
Note: It can be expected that future advances in Internet audio transmission would make the problem of “glitches” less troublesome.
Reasons to use voice feedback abound in the literature and in the positive data results of our study. However, weaknesses were documented. In our case, “glitches” were primarily focused on access to functioning technology. Drawbacks noted by Henderson and Phillips (2015) in their study utilizing video feedback were similar: some students lacked access to videos because their computers and devices weren’t equipped to do so. In addition, some students were concerned that they lacked the privacy required for paper revision when feedback was accessed in open office settings. Others felt some anxiety about seeing and hearing the feedback from the video, which was perceived as more direct and confrontational than written feedback by some students (Henderson & Phillips, 2015). Ice et. al (2007) stressed the importance of using the student’s name in recorded feedback, which is something often overlooked with the press of time and numbers of assignments to be assessed. A panoply of dialectics using humor and personal examples as well as generally respectful discourse can more easily personalize, encourage, and affect perspectives on student work with the addition of voice for formative feedback. Kolowich (2015) observed that video voice feedback might provide too much information that is not easily reviewed more than once. The negative comment collected in our qualitative data set echoed this observation, though we are not sure whether the comment refers to a student preference to make written edits based on written feedback or trouble accessing the audio feedback repeatedly.
Instructors who use today’s digitally provided feedback options are likely to find they spend the same amount of time delivering audio feedback as written comments alone. However, they can provide more details directed toward individualized needs within that time frame. Ice et al. (2007) also found that the time spent in creating asynchronous feedback for individual writing assignments seems to take the same duration, but in addition to noting all the errors, more positive comments and suggestions for improvement may be included via additional resources such as charts and YouTube videos. Using the Kaizena Lessons feature, more resources directed toward specific writing issues can be supplied with ease. For example, many students need extra help to avoid run-on sentences; therefore, an embedded web video lesson can provide additional instruction for that particular composition issue.
Our study supports Pearce and Ackley’s (1995) finding that “students liked or were motivated better by taped feedback” (p. 32). In addition, they noted that audio feedback was more detailed than written. According to Boling, Hough, Krinsky, Saleem and Stevens (2012), “…high levels of interaction typically need to be present for learners to have a positive attitude and greater satisfaction” (p. 119). Henderson and Phillips (2015) identified themes in their data set, including student interpretation of recorded feedback as “individualized and personal” as well as “supportive: seen as caring and felt to be motivating” (p. 58). Students appreciated the degree of detail provided in terms of how their work was evaluated.
Positive student perceptions of the feedback technology are further anchored by rapidly advancing technological capabilities. Newer tools like Kaizena voice comments are saved directly in a Google Document where students can apply the recommendations immediately in their written assignments. A major hurdle may be in keeping abreast with new versions of digital feedback as they occur. Accordingly, despite the mounting evidence to support use, few instructors are providing technically enhanced feedback (Khalil, 2013; Kolowich, 2015). Harvey and Broyles provided a list of 20 resistance factors along with an antidote for each that should be considered (qtd. in Khalil, 2013). As more accessible tools become available, greater instructor usage could be expected. In fact, Lunt and Curran (2010) predicted that enthusiasm toward the use of audio feedback tools would be used to substitute for face-to-face conferencing. The Kaizena add-on voice comments agree to the extent that they characterize the feedback and replies as “conversations”. Feedback as dialogue means that the student not only receives initial feedback information, but also has the opportunity to engage the teacher in discussions about that feedback.
Crews, Bordonada and Wilkinson (2017) urged online instructional designers and instructors to encourage student feedback regarding their online content and courses as well as for their own assignments. Tait (2014) predicted that, given the newer technologies, “new entrants to e-and online learning will leapfrog their predecessor with an improved student experience” (p. 15). While there may be some minor technological “glitches,” as with all internet tools and personal computing, our study results strongly suggest that voice feedback tools are a positive addition to the instructional process. Exploring more than one tool provided further support for this suggestion. Kaizena voice comments and Adobe Acrobat Reader are both able to provide formative feedback with detailed instruction in grammatical or essay structures for written assignments. Truly, enhanced feedback with recorded voice comments has been shown to be an asset to learning.
Developing formative feedback through voice comments can be a powerful tool to develop greater self-motivation for online learning. Our investigation supported the advantages of voice comment feedback over text alone. Formative audio tools are timely and efficient and provide authentic high-quality observations encouraging dialogue that is engaging and rich in quality as well as quantity (Merry & Osmond, 2008). In this study’s survey comment section, one student saw the voice tool as saving time for both students and instructors, noting, “It is quick…a new approach on hearing both in voice, but not taking up time of both when one or the other is occupied…” Audio files are permanent, enabling a student to review the original feedback as needed, which is a distinct advantage over face-to-face conferencing. The asynchronous voice feedback format applied in our study also met the requirements of Chickering and Ehrmann’s (n.d.) seven principles for delivery methods requiring technology to:
Our findings have provided direction for future research. Specifically, the following questions can be rich investigative opportunities.
Timely formative feedback can benefit future “thoughts, feelings and/or actions” (Getzlaf, Perry, Toffner, Lamarche & Edwards, 2009, p. 4). Indeed, audio feedback may propel students forward into a series of overall positive and successful learning experiences. In reflecting on his own research comparing the kinds of feedback given, Delante (2017) recommended “a comparative study between online written feedback and voice/video or chat feedback” (p. 28). This study partially fulfilled that inquiry and supports the increasing effort by instructors to use recorded voice comments for more effective formative feedback. Our results point to a preference regarding the use of audio feedback specifically related to text in written assignments. Additional investigations into the effectiveness of using voice comments to personalize formative feedback should yield similar results, further supporting our assertion and the continued goal of promoting social presence and narrowing transactional gaps in online learning.
Aragon, S.R., Johnson, S.D., & Shaik, N. (2002). The influence of learning style preferences on student success in online versus face-to-face environments. The American Journal of Distance Education, 16(4), 227-243. https://doi.org/10.1207/S15389286AJDE1604_3
Bastian, H. (2017). Student affective responses to ‘bringing the funk’ in the first-year writing classroom. College Composition and Communication, 69(1), 6-34. Retrieved from www.ncte.org/-library/NCTEFiles/Resources/Journals/CCC/0691_sept2017/CCC0691Student.pdf
Beldarrain, Y. (2006). Distance education trends: Integrating new technologies to foster student interaction and collaboration. Distance Education, 27(2), 139-153. https://doi.org/10.1080/01587910600789498
Boling, E.C., Hough, M., Krinsky, H., Saleem, H., & Stevens, M. (2012). Cutting the distance in distance education: Perspectives on what promotes positive, online learning experiences. The Internet and Higher Education, 15(2), 118-126. doi:https://doi.org/10.1016/j.iheduc.2011.11.006
Campbell, C.M., & Cabrera, A.F. (2014). Making the mark: Are grades and deep learning related? Research in Higher Education, 55(5), 494-507. https://doi.org/10.1007/s11162-013-9323-6
Casey, D.M. (2008). A journey to legitimacy: The historical development of distance education through technology. TechTrends: Linking Research and Practice to Improve Learning, 52(2), 45-51. https://doi.org/10.1007/s11528-008-0135-z
Chickering, A.W., & Ehrmann, S.C. (n.d.). Implementing the seven principles: Technology as lever. Boston University School of Public Health Office of Teaching, Learning and Technology Teaching Resources. Retrieved from sphweb.bumc.bu.edu/otlt/teachingLibrary/Technology/seven_principles.pdf
Crews, T.B., Bordonada, T.M., & Wilkinson, K. (2017). Student feedback on quality matters standards for online course design. Educause Review, 5. Retrieved from er.educause.edu/articles/2017/6/student-feedback-on-quality-matters-standards-for-online-course-design
Delante, N.L. (2017). Perceived impact of online written feedback on students’ writing and learning: A reflection. Reflective Practice, 18(6), 772-804. https://doi.org/10.1080/14623943.2017.1351351
Driscoll, A., Jicha, K., Hunt, A.N., Tichavsky, L., & Thompson, G. (2012). Can online courses deliver in-class results? Teaching Sociology, 40(4), 312-331. https://doi.org/10.1177%2F0092055X12446624
Eom, S.B., & Ashill, N. (2016). The determinants of students’ perceived learning outcomes and satisfaction in university online education: An update. Decision Sciences Journal of Innovative Education, 14(2), 85-116. https://doi.org/10.1111/dsji.12097
Getzlaf, B., Perry, B., Toffner, G., Lamarche, K., & Edwards, M. (2009). Effective instructor feedback: Perceptions of online graduate students. Journal of Educators Online, 6(2). Retrieved from https://files.eric.ed.gov/fulltext/EJ904070.pdf
Guion, L.A. (2002). Triangulation: establishing the validity of qualitative studies. University of Florida Extension: Institute of Food and Agricultural Sciences, Sept. Retrieved from sites.duke.edu/niou/files/2014/07/W13-Guion-2002-Triangulation-Establishing-the-Validity-of-Qualitative-Research.pdf
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. https://doi.org/10.3102%2F003465430298487
Henderson, M., & Phillips, M. (2015). Video-based feedback on student assessment: Scarily personal. Australasian Journal of Educational Technology, 31(1), 2015, 51-66. https://doi.org/10.14742/ajet.1878
Ice, P., Curtis, R., Phillips, P., & Wells, J. (2007). Using asynchronous audio feedback to enhance teaching presence and students’ sense of community. Journal of Asynchronous Learning Networks, 11(2), 3-25.
Ice, P., Swan, K., Diaz, S., Kupczynski, L. & Dagen, A. (2010). An analysis of students’ perceptions of the value and efficacy of instructors’ auditory and text-based feedback modalities across multiple conceptual levels. Journal of Educational Computing Research, 43(1), 113-134.
Johanson, R. (1999). Rethinking the red ink: Audio-feedback in the ESL writing classroom. Texas Papers in Foreign Language Education, 4(1), 31-38. Retrieved from http://files.eric.ed.gov/fulltext/ED467865.pdf
Khalil, S.M. (2013). From resistance to acceptance and use of technology in academia. Open Praxis, 5(2), 151-163. http://dx.doi.org/10.5944/openpraxis.5.2.5
King, D., McGugan, S., & Bunyan, N. (2008). Does it make a difference? Replacing text with audio feedback. Practice and Evidence of Scholarship of Teaching and Learning in Higher Education, 3(2), 145-163. Retrieved from dur.ac.uk/pestlhe.learning/index.php/pestlhe/article/viewFile/136/163
Kolowich, S. (2015, January 26). Could video feedback replace the red pen? The Chronicle of Higher Education, 26. Retrieved from www.chronicle.com/blogs/wiredcampus/could-video-feedback-replace-the-red-pen/55587
Kuh, G.D., Kinzie, J., Buckley, J.A., Bridges, B.K., & Hayek, J.C. (2006). What matters to student success: A review of the literature. National Post-Secondary Education Cooperative (NPEC), 2. Retrieved from nces.ed.gov/npec/pdf/kuh_team_report.pdf
Lunt, T., & Curran, J. (2010). Are you listening please? The advantages of electronic audio feedback compared to written feedback. Assessment & Evaluation in Higher Education, 35(7), 759-69. https://doi.org/10.1080/02602930902977772
McCarthy, J. (2015). Evaluating written, audio and video feedback in higher education summative assessment tasks. Issues in Educational Research, 25(2), 153-169. Retrieved from http://www.iier.org.au/iier25/mccarthy.pdf
Merry, S. & Orsmond, P. (2008). Students’ attitudes to and usage of academic feedback provided via audio files. Bioscience Education, 11(3). https://doi.org/10.3108/beej.11.3
Olesova, L.A., Weasenforth, D., Richardson, J.C., Meloni, C. (2011). Using instructional audio feedback in online environments: A mixed methods study. MERLOT Journal of Online Teaching and Learning, 7(1), 30-42. Retrieved from http://jolt.merlot.org/vol7no1/olesova_0311.pdf
Puentedura, R.P. (2014). Building transformation: An introduction to the SAMR model. Retrieved from http://www.hippasus.com/rrpweblog/archives/2014/08/22/BuildingTransformation_-AnIntroductionToSAMR.pdf
Sloan, D., Stratford, J., & Gregor, P. (2006). Using multimedia to enhance the accessibility of the learning environment for disabled students: Reflections from the skills for access project. Research in Learning Technology, 14(1).
Straub, C., &, Vasquez, III, E. (2015). Effects of synchronous online writing instruction for students with learning disabilities. Journal of Special Education Technology, 30(4), 213-222. https://doi.org/10.1177/0162643415618929
Tait, A. (2014). From place to virtual space: Reconfiguring student support for distance and e-learning in the digital age. Open Praxis, 6(1), 5-16. http://dx.doi.org/10.5944/openpraxis.6.1.102
Trust, T. (2018). Assessment-centered Tools. Online tools for teaching and learning (Blog). Retrieved from https://blogs.umass.edu/onlinetools/assessment-centered-tools/
What is Social Presence? (n.d.). IGI Global (Idea Group Inc.) [US]. Retrieved from www.igi-global.com/dictionary/social-presence/27463
Young, S. (2006). Student views of effective online teaching in higher education. American Journal of Distance Education, 20(2), 65-77. https://doi.org/10.1207/s15389286ajde2002_2
|Kaizena ease of access||number of students||prefer voice comments||prefer written comments|
|Neither difficult nor easy||4||3||1|
|Not able to access (iPad)||1||0||1|
|Adobe Acrobat ease of access||number of students||prefer voice comments||prefer written comments|
|Neither difficult nor easy||0||0||0|
|Not able to access||2||0||2|
Note: Responses to survey questions 7 and 8 are reported together for each student, identified by a number for data purposes. #15 was the only student using an iPad and #21 and #22 were the respondents who used Smartphones.
Responses to questions:
7. Additional comments about receiving Kaizena voice feedback on your writing.
8. Could you explain why you prefer one or the other?
#7 less intrusive
#15 written feedback was much easier, because the voice feedback is difficult to download in the iPad
#17 If editing could be done on the same page with the comments, it will be better than the written comments. I prefer written comments because it is easy to edit my work as I read through. It is hard to edit my work while listening to voice comments as I have to alternate the browser.
#21 (Student identified with English as a Second Language) I enjoyed it. It is quick in my point of view on the professor and the student. A new approach on hearing both in voice but, not taking up time of both being on the phone. When one or the other is occupied they can both be responsive on their time. The voice message is understood better by me. English is my second language and the wording could be slightly difficult to understand.
#25 Some of the voice comments were difficult to understand
#26 I would prefer to have the written comments on my document because then I could save the document to look at afterwards. The comments on Kaizena also don’t attach the voice comments on the document’s intended location. For example, I have noticed I would receive a comment on how I should reword a certain sentence, but I don’t know where to look since I only have her voice to show me. The feedback is helpful but I would prefer the comments to be written. If there was anyway to save the feedback to my computer with the locations being discussed, then I wouldn’t mind using Kaizena
#28 Easier to read then listen to because speakers can become muffled.
#29 They were helpful but, there were some issues. (Glitches) So I can see and process the comments better.
#30 Written comments were Easy for me to understand.