Introduction

Interest in online learning and teaching has been on the rise for some time and accelerated by the 2019 CORONA virus pandemic. Many of these online courses also include online assessment activities which raises a number of issues and challenges in relation to plagiarism and academic integrity on the whole. One of the ways of coping with some of these challenges is the adoption of online proctoring tools for online assessments.

Online proctoring involves the use of virtual tools for monitoring student activities during assessment activity. These tools (as they continue to overcome their limitations) have the potential for students to take an online exam at a remote location while ensuring the integrity (security and trustworthiness) and reliability of the online exam. This includes the authentication of the student and their identity to secure and maintain the integrity of an exam and its administration (Foster & Layman, 2013).

Online proctoring has two major components. First, the availability of a web-camera on the student’s computing device needs to be activated to video record the physical learning space and everything the student does during the examination period. The examiner or the proctor is able to remotely monitor this video recording. The examiner or proctor is able to identify potential cheating, suspicious movements, and posture such as talking to someone in the room, looking at a book, mobile device, or other printed media for answers. Second, is lockdown, which will prevent the students from using any other computer applications including the Internet browser, and user-computing processes (such as copying, pasting or printing) that can lead to potential cheating during the exam. This is commonly referred to as “computer or browser lockdown” (Alessio et al., 2017). The proctoring system also records all the student Internet activities during the exam such as websites that the student tried to access. The video recording of the entire exam is made available for review by the instructors or examiners either simultaneously or afterwards.

There are four major features of online proctoring systems; (i) authentication: which is the process of ensuring the registered student is the valid student taking an online proctored exam, (ii) browsing tolerance: this is the process of setting the limit of student’s ability to use their computer for other tasks, (iii) remote authorizing and control: which is enabling the proctor to start, pause and end online proctored exam, and as well as flagging any suspicious student behaviours, and (iv) report generation: which is the creation of reports of a student’s activities during a proctored exam.

Generally, there are three types of online exam proctoring:

1. Live proctoring: This is real-time proctoring taking place during the exam with a human proctor monitoring/supervising the exam virtually, online. The human proctors are usually trained professionals to ensure the authenticity of the student and look for any red flags such as suspicious eye or facial movements or the appearance of any unverified device that could indicate possible cheating (Gautam, 2007). This requires the exams to be scheduled at a specific time depending on the availability of the proctor on a given date and time. This has equal human involvement as traditional offline exam supervision. However, unlike live proctoring, online proctoring will require competence in the use of technology, and as such much closer vigilance on the approaches of online proctors will be required (Mitra & Gofman, 2016).
2. Recorded proctoring: This involves the video recording of camera images and logs of the student taking an online proctored exam, where the proctor reviews the recording at a later time and assesses the integrity of the exam (i.e. whether or not any fraud/cheating was committed during the exam by the examinee). This allows students to take an exam at any time hence allowing multiple exams to take place simultaneously. But, this too requires human intervention for reviewing the recordings, and that can be expensive and difficult to scale as well.
3. Automated proctoring: In automated proctoring – human proctors do not monitor (or review) the entire exam, instead, the proctoring system identifies key events of possible fraud or cheating. The proctor is alerted to review these events to determine if fraud or cheating has been committed by the student (Sietses, 2016). This form of online proctoring is generally considered more convenient for the students as they are not required to arrange live proctors for their tests and exams, as there is no schedule, location and human proctor constraints. It is also very scalable as the human component is replaced by artificial intelligence or algorithms. Hence, it is considered more cost-effective (Jose, 2016). However, students’ familiarity with this proctoring system may spawn evasive strategies for fraud prevention. This form of proctoring can also easily produce false positives such as flagging innocent events as potential fraud (Sietses, 2016).

There are many online proctoring systems available that offer the three types of online exam proctoring services mentioned earlier. But, institutions in the midst of choosing and implementing an online exam proctoring system need to consider several factors first. These include (but are not limited to): ease and flexibility of integration with the existing institutional learning management system, technical performance and robustness of the proctoring system (sometimes over low internet bandwidth, poor hardware capabilities or electrical power failures), level of efficient task automation, and reporting capabilities. Privacy protection and management, security and anti-fraud measures, and their associated cost are also other key issues that need to be examined when considering an online proctoring system (Sietses, 2016).

The research that is reported in this paper sought to:

1. Identify online proctoring systems;
2. Test and evaluate selected online proctoring systems;
3. Select and dummy trial of selected online proctoring systems; and
4. Develop procedures and guidelines for online exam proctoring.

Literature Review

Interest in the affordances of technology for learning and teaching is on the rise. This is leading to a growing interest in online learning and teaching. When used effectively, online learning is able to provide higher education institutions with flexible options to expand their offerings into the global market (Casey, 2008). However, as institutions continue to grow their online education, there is a commensurate rise in concerns about how best to ensure academic integrity (Barnes & Paris, 2013). The distance or flexibility between students and instructors in an online learning environment may, in fact, contribute to the challenges of maintaining the integrity of online assessment. This was also highlighted by Hollister and Berenson (2009) that, “the most commonly reported challenge in online assessment is how to maintain academic integrity”. While proctored exams remain a common tool for assessing student learning, ways of facilitating them continue to evolve from online exams facilitated via learning management systems (LMS) to other online testing platforms (Prisacari & Danielson, 2017). This has raised both academic and non-academic issues, such as the designing and administering of online exams, and monitoring students’ behaviour during exams (Cramp et al., 2019). These behaviours include dishonest and unethical practices by the students such as cheating and fraud.

In their study, King et al. (2009) reported that the majority of students surveyed felt that cheating was easier in an online environment compared to a traditional face-to-face classroom. Similarly, Berkey and Halfond (2015) reported that 84% of the students surveyed in their study agreed that student dishonesty in online test-taking was a significant issue. In a study of 635 students, Watson and Sottile (2010) also noted that students indicated that they would be more than four times more likely to cheat in an online class. Several other studies also found higher rates of cheating online (Lanier, 2006; Harmon & Lambrinos, 2008; Grijalva et al., 2006) and prevalence of cheating online compared to in a face-to-face environment (Etter et al., 2006; Watson & Sottile, 2010).

Ensuring and maintaining academic honesty and integrity in any learning environment is vital and significant. When putting this in the context of an online learning environment, Moten et al. (2013) explained that students in these learning environments work independently with relative autonomy and anonymity, and instructors may be uncertain who is taking exams or how best to validate student learning. Therefore, online learning must address issues and challenges of honesty and integrity in student assessment and evaluation. Online proctoring is one way to address this challenge. With technology-based aides, such as computer/system lockdowns, keystroke monitoring, the ability to stop/start a test, and many other assistive proctoring processes (Foster & Layman, 2013) now easily integrated into the monitoring process, online proctoring has now become a viable solution.

Moreover, online proctoring offers both instructors and students other significant advantages. Kinney (2001) noted that online proctoring is a valuable option for students who are geographically dispersed with time differences. Several studies (such as Bedford et al., 2009; Harmon et al., 2010; Rose, 2009; Watson & Sottile, 2010) found that when compared with traditional face-to-face settings, the technologies associated with monitoring of the online examination can provide better exam security and integrity. Karim et al. (2014) in their study found that the use of remote online proctoring decreases instances of student cheating. Similarly, Kolski and Weible (2019) posited that the importance of academic integrity could be reinforced when students are aware of the instructors reviewing their recorded exam sessions. Likewise, Tao and Li (2012) highlighted that online proctoring reduces instructional time dedicated to testing allowing instructors and students to engage more with the course content.

However, there are mixed findings in terms of student performance in online-proctored exams. Schultz et al. (2007) in their study reported that students who took the non-proctored online exams performed significantly higher than did those in the proctored settings. Similarly, Alessio et al. (2017), Richardson and North (2013), Wellman and Marcinkiewicz (2004) and Carstairs and Myors (2009) reported the same findings with non-proctored test scores being significantly better than proctored test scores in their respective studies. But, other studies (such as Ladyshewsky (2015), Yates and Beaudrie (2009) and Beck (2014) found no significant difference between the test scores in proctored versus non-proctored online tests.

For institutions, selecting the fit-for-purpose online exam proctoring technology can be challenging. While there are not many studies on how institutions selected and integrated online proctoring systems, Brown (2018) describes three factors that can impact the selection of an online exam proctoring solution: cost, security and, instructor and student comfortability with the use of technology highlighting that involving the faculty in the selection of the online proctoring technology would be beneficial. She further identifies technology support staff, teaching staff and students as the three most important stakeholders in the selection process of the fit-for-purpose online exam proctoring technology of an institution (Brown, 2018).

Moreover, Foster and Layman (2013) developed a comparison matrix that describes online proctoring functionality, and compares that functionality across various online proctoring services/products such as proctoring features (human-proctor availability, data transfer encryption, proctor management, recorded review, automated proctoring, incident logs, etc.), lockdown features (browser lockdown, computer operations lockdown, keystroke alerts, etc.), authentication options (facial recognition, photo comparison, keystroke analytics, biometrics, etc.) and webcam features (camera view angles, panning, etc.). This matrix could be useful for institutions in the process of identifying and selecting the right online exam proctoring system.

The purpose of this investigation has been to add to this body of literature with a preliminary investigation, identification, and selection of an online proctoring solution, specifically addressing the following research questions:

1. Which are the most prominent online proctoring systems?
2. How effective and efficient are they for wide-scale adoption in higher education settings?
3. What are the recommended procedures and guidelines for online exam proctoring?

Methodology

This investigation was carried out at USP (the University of the South Pacific) which is a regional University that is owned and governed by twelve nations of the southwest Pacific region. These include the Cook Islands, The Republic of Fiji, Kiribati, Marshall Islands, Nauru, Niue, Solomon Islands, Tokelau, Tonga, Tuvalu, Vanuatu and Samoa. The University has campuses in all of the member countries. Its main campus is located in Suva, the Republic of Fiji where the majority of its academic Schools are based, except for the following – the School of Agriculture and Food Technology, which is situated at the Alafua Campus in Samoa, and the School of Law at the Emalus Campus in Vanuatu. The USP region spreads across 33 million square Kilometres of ocean, an area three times the size of Europe, with a total land mass about the size of Denmark. Population masses in the region vary from 2,000 in Tokelau to more than 800,000 in the Republic of Fiji. For island nations, this widely spread and sparsely populated, online learning and teaching methods, including flexible approaches to the assessment of learning, has had to feature prominently in its educational operations.

The adoption of flexible approaches to the assessment of learning has required a thorough investigation of contemporary online proctoring tools. A 4-phased approach was adopted as part of this process. The elimination of a system in every phase was based on the ‘survival of the fittest’ approach with each phase building upon the milestones and deliverables from the previous phase as per the table 1.

Results from Mock-trials

Results in Figure 1 illustrate that user experience with Proctorio was positive for students from the respective campuses. The Samoan students did report a slightly less enjoyable experience and the reason became clear when their understanding of the applications was looked at in Figure 5.

Results in Figure 2 illustrates that the completion of test with Proctorio was positive for students from the respective campuses. 14% of Samoan students did report that they were unsuccessful in completing the test. This was largely due to connectivity issues that they faced during the test.

A majority of the students from the four campuses had very little to no navigation issues throughout their quiz attempt. It was however noted that there were a few challenges (12% and 14% at Labasa and Samoa campus respectively) and these were evident during ID verification and when adding the Proctorio Chrome Extension.

The high number of positive responses in all the four campuses in Figure 4 signifies that the instructions provided to students which they followed before the actual test commenced were stated clearly and in detail. Students are quite familiar with doing online quizzes on Moodle and with Proctorio embedded to the quiz, there are hardly any major changes except for the ID verification which students have to undergo before they attempt the quiz.

It can be seen that students overall experience while doing a proctored quiz is challenging and uncomfortable. This is quite understandable as this is the first time for them to verify themselves using web cameras and their identification cards. Most of the students indicated during the focus group discussions that they had never used a web camera before and they faced problems trying to verify their ID cards as they had to hold it properly before the picture could be taken off the card. This process indicated discomfort for most of the students across the campuses.

Key Findings

1. Students incur a great deal of time and money to travel to campuses to sit for tests.
2. This cost can be eliminated with online proctoring. The team tested the Proctoring system with very low Internet speed and received positive results.
3. Online proctoring can easily be integrated into Moodle without additional infrastructure.
4. Students are generally positively disposed towards the use of online proctoring.

Discussion of the Findings

Conclusion and Recommendations

Online proctoring has its challenges. Unlike a live examination, online proctoring requires students to have access to suitable technological infrastructure, without which the option will not work reliably. Naturally, this creates a divide between those with, and without access to this technological infrastructure. Then there are those students with disabilities who may require a lot more assistance than is possible while taking online-proctored exams. There are also concerns around how the recorded video is interpreted, and used by others. These issues are not likely to go away, which means that online proctoring can only be offered as just another solution alongside other options. It ought not to be promoted as the only solution and should be adopted and used carefully and selectively in contexts and situations where it is the best solution.

In light of these concerns and considerations, the following recommendations ought to be considered in the adoption of online proctoring as part of examination processes.

1. Prepare recommended online examination procedures. Having a university-wide recommended online examination procedure would help the lecturers to facilitate the online tests in a uniform manner. This will also provide clarity around roles and responsibilities of the lecturer and that of the student.
2. Trial the proctoring system with live courses that have large regional student numbers. At the moment we have results from mock trials, but it will be beneficial if there are test results from live courses (and during peak periods).
3. Ensure a computer lab (equipped with the hardware and software requirements) is designated to students who do not have their own laptops. Not all students are able to find a quiet, well-lit room to sit for the examination.
4. Ensure that the hardware and software requirements for the proctoring system are met. There are certain requirements for using the proctoring system. These are a web camera and browser plugins...

Furthermore, meeting these expectations ought not to be seen as a one-time-fix. All of these recommendations will require ongoing monitoring and maintenance. Students with different types of disabilities would require additional assistance taking online-proctored exams. Furthermore, with online exams, we will be pushing students to procure tools that they would not need if they just sat for a paper-based exam. These will include such things as digital cameras, headphones, extra lighting in the room, laptops or desktop computers. We must ensure that the end-users of the system know how to use the system, and are comfortable in using the system before rolling it out.