Effective web videoconferencing for proctoring online oral exams: a case study at scale in Brazil

Alexandra Okada, Peter Scott, Murilo Mendonça

Abstract


The challenging of assessing formal and informal online learning at scale includes various issues. Many universities who are now promoting “Massive Online Open Courses” (MOOC), for instance, focus on relatively informal assessment of participant competence, which is not highly ‘quality assured’. This paper reports best practices on the use of a web videoconferencing application to quality control student assignments through online oral examination at scale. In this case study, we examine the use of a simple online conferencing technology FlashMeeting (FM) by a Brazilian University to provide ‘quality assurance’ in the assessment of twelve online postgraduate courses in Law for 20,000 students. Our research questions investigate the benefits and recommendations of using FM in online oral exams at scale. Our qualitative and quantitative data analysis centres on 3,462 short format interviews through FM conducted for this purpose by a group of around fifty assessors from September 2008 to September 2012. The effective use of FM provided evidence with respect to high quality assurance recognised by the Institution with respect to: students’ identity, their knowledge and ownership of written work. The key benefits identified from the perspective of assessors and students were: reliable examination, credible technology, authentic assessment, interactive e-Viva, low cost, scalable process and practical testing in terms of time, effort and money.


Keywords


Assessment at scale; quality assurance; videoconferencing; Massive Online Open Courses; Higher Education; oral online exams; e-Viva

Full Text:

PDF

References


Canessa, E., Tenze, L. & Salvatori, E. (2013). Attendance to Massive Open On-line Courses: Towards a Solution to Track on-line Recorded Lectures Viewing. Bulletin of the IEEE Technical Committee on Learning Technology, 15(1), 36-39. Retrieved from http://www.ieeetclt.org/issues/january2013/Canessa.pdf

Cross, S.; Whitelock, D. & Galley, R. (2014). The use, role and reception of open badges as a method for formative and summative reward in two Massive Open Online Courses. International Journal of e-Assessment, 4(1). Retrieved from http://oro.open.ac.uk/40593/

Gaytan, J. (2005). Effective assessment techniques for online instruction. Information Technology, Learning, and Performance Journal, 23(1), 25-33.

Harmon, O. & Lambrinos, J. (2008). Are Online Exams an Invitation to Cheat? The Journal of Economic Education, 39(2), 116-125. http://dx.doi.org/10.3200/JECE.39.2.116-125

Hollister, K. K. & Berenson, M. L. (2009), Proctored Versus Unproctored Online Exams: Studying the Impact of Exam Environment on Student Performance. Decision Sciences Journal of Innovative Education, 7(1), 271–294. http://dx.doi.org/10.1111/j.1540-4609.2008.00220.x

Hopkins, J. (2011). Assessed real-time language learning tasks online: how do learners prepare? eLC Research Paper Series, 2, 59-72. Retrieved from http://elcrps.uoc.edu/index.php/elcrps/article/view/n2-hopkins/n2-hopkins

Kim, J., & Craig, D. A. (2012). Validation of a videoconferenced speaking test. Computer Assisted Language Learning, 25(3), 257-275. http://dx.doi.org/10.1080/09588221.2011.649482

Koller, D., Ng, A., Chuong, D., & Chen, Z. (2013, June 3). Retention and intention in massive open online courses. Educause Review, 48(3), 62–63. Retrieved from http://www.educause.edu/ero/article/retention-and-intention-massive-open-online-courses

Kroeck, K. G. & Magnusen, K. O. (1997). Employer and job candidate reactions to videoconference job interviewing. International Journal of Selection and Assessment, 5(2), 137-142. http://dx.doi.org/10.1111/1468-2389.00053

Okada, A. (2008). Knowledge Media Technologies for Open Learning in Online Communities. The International Journal of Technology, Knowledge and Society, 3(5), 61-74.

Okada, A., Tomadaki, E., Buckingham Shum, S. & Scott, P. (2008). Fostering Open Sensemaking Communities by Combining Knowledge Maps and Videoconferencing. UPGRADE, The European Journal for the Informatics Professional, 9(3), 27-36. Retrieved from http://www.cepis.org/upgrade/files/2008-III-scott.pdf

Perna, L. W., Ruby, A., Boruch, R. F., Wang, N., Scull, J., Ahmad, S., & Evans, C. (2014). Moving through MOOCs: Understanding the progression of users in massive open online courses. Educational Researcher, 43(9), 421-432. http://dx.doi.org/10.3102/0013189X14562423

Picciano, A. G. (2002). Beyond student perceptions: Issues of interaction, presence, and performance in an online course. Journal of Asynchronous Learning Networks, 6(1), 21-40.

Robles, M., & Braathen, S. (2002). Online assessment techniques. The Delta Pi Epsilon Journal, 44(1), 39-49.

Scott, P., Castañeda, L., Quick, K. and Linney, J. (2009). Synchronous symmetrical support: a naturalistic study of live online peer-to-peer learning via software videoconferencing. Interactive Learning Environments, 17(2), 119-134. http://dx.doi.org/10.1080/10494820701794730

Scott, P.; Tomadaki, E. & Quick, K. (2007). The Shape of Live On-line Meetings. The International Journal of Technology, Knowledge and Society, 3(4). Retrieved from http://flashmeeting.open.ac.uk/research/publications/tks07.pdf

Tiller, D., O'Mara, D., Rothnie, I., Dunn, S. Lee, L. & Roberts, C. (2013). Internet-based multiple mini-interviews for candidate selection for graduate entry programmes. Medical Education, 47(8), 801-810. http//dx.doi.org/10.1111/medu.12224

Wellman, G. (2005). Comparing Learning Style to Performance in On-Line Teaching: Impact of Proctored v. Un-Proctored Testing. Journal of Interactive Online Learning, 4(1), 20-39. Retrieved from http://www.ncolr.org/jiol/issues/pdf/4.1.2.pdf

Wodehouse, A; Breslin, C; Eris, O.; Grierson, H.; Ion, W; Jung, M.; Juster, N.; Leifer, L.; Mabogunje, A. & Sonalkar, N. (2007). A reflective approach to learning in a global design project. International Conference on Engineering and Product Design Education, 13-14 September 2007, Northumbria University, Newcastle Upon Tyne, United Kingdom.

Wynne, L. & Lopes, S. (2006). Implementing Large Scale Assessment Programmes. In M. Danson (ed.). Proceedings of the 10th CAA International Computer Assisted Assessment Conference, 4 & 5 July 2006, Loughborough University (pp. 525-526).




DOI: http://dx.doi.org/10.5944/openpraxis.7.3.215

Refbacks

  • There are currently no refbacks.