The usability of augmented reality in open and distance learning systems: A qualitative Delphi study

Hakan Altınpulluk, Mehmet Kesim, Gulsun Kurubacak

Abstract


The main purpose of this research is to determine the usability of augmented reality in open and distance learning environments in accordance with universal design principles, and to make predictions for the future by gathering expert opinions on this subject using the Delphi technique. The Delphi technique was applied to 14 expert participants for 3 rounds as the primary data collection tool with open ended questions based on the theoretical framework. Structured interview questions were used as a secondary data collection tool and were applied during an academic exchange in China. In the Delphi technique used as the primary data collection tool, 92 themes were evaluated by experts and accepted as usability principles by end of this research. Therefore, 92 themes under 21 titles were presented for the use of augmented reality within the framework of universal design principles in open and distance learning. This research may be the first unique study on the usability of augmented reality not just as the convergence of this technology with open and distance learning environments but also incorporating the learning and communication dimensions of this convergence, thereby contributing to the literature of the field. It is imperative to determine the usability of augmented reality in open and distance learning, along with the “how” of this use in application. In this regard, the findings of this study are significant in shedding light to the enrichment, diversification and increased interaction of open and distance learning environments in accordance with universal design principles, bringing a new perspective to how a different technology convergence may be conducted, providing further accessibility.


Keywords


Augmented reality; open and distance learning; universal design principles; Delphi

Full Text:

HTML PDF XML

References


Adler, M., & Ziglio. E. (1996). Gazing into the oracle: The delphi method and its application to social policy and public health. London: Jessica Kingsley Publishers.

Akçayır, M., & Akçayır, G. (2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educational Research Review, 20, 1-11. https://doi.org/10.1016/j.edurev.2016.11.002

Alsina-Jurnet, I., & Guardia-Ortiz, L. (2015). Augmented reality in online educational contexts: the UOC case study. EDEN 2015 Proceedings. Barcelona, Spain. Retrieved from https://www.researchgate.net/publication/279538875_AUGMENTED_REALITY_IN_ONLINE_EDUCATIONAL_CONTEXTS_THE_UOC_CASE_STUDY

Altinpulluk, H., & Eby, G. (2016). Theoretical framework regarding the usability of augmented reality in open and distance learning systems. In EdMedia+ Innovate Learning (pp. 487-492). Association for the Advancement of Computing in Education (AACE).

Altinpulluk, H. (2019). Determining the trends of using augmented reality in education between 2006-2016. Education and Information Technologies, 24(2), 1089-1114. https://doi.org/10.1007/s10639-018-9806-3

Andujar, J. M., Mejías, A., & Marquez, M. A. (2011). Augmented reality for the improvement of remote laboratories: an augmented remote laboratory. IEEE Transactions on Education, 54(3), 492-500. https://doi.org/10.1109/TE.2010.2085047

Arici, F., Yildirim, P., Caliklar, Ş., & Yilmaz, R. M. (2019). Research trends in the use of augmented reality in science education: Content and bibliometric mapping analysis. Computers & Education, 142, 103647. https://doi.org/10.1016/j.compedu.2019.103647

Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., & MacIntyre, B. (2001). Recent advances in augmented reality. Computer Graphics and Applications, 21(6), 34-47. https://doi.org/10.1109/38.963459

Bach, B., Sicat, R., Beyer, J., Cordeil, M., & Pfister, H. (2017). The hologram in my hand: How effective is interactive exploration of 3d visualizations in immersive tangible augmented reality?. IEEE transactions on visualization and computer graphics, 24(1), 457-467. https://doi.org/10.1109/TVCG.2017.2745941

Barfield, W. (2015). Fundamentals of wearable computers and augmented reality. CRC Press.

Bau, O., & Poupyrev, I. (2012). REVEL: tactile feedback technology for augmented reality. ACM Transactions on Graphics (TOG), 31(4), 1-11. https://doi.org/10.1145/2185520.2185585

Benko, H., Ofek, E., Zheng, F., & Wilson, A. D. (2015). Fovear: Combining an optically see-through near-eye display with projector-based spatial augmented reality. Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (pp. 129-135).

Billinghurst, M., & Duenser, A. (2012). Augmented reality in the classroom. Computer, 45(7), 56-63. https://doi.org/10.1109/MC.2012.111

Chang, S. C., & Hwang, G. J. (2018). Impacts of an augmented reality-based flipped learning guiding approach on students’ scientific project performance and perceptions. Computers & Education, 125, 226-239. https://doi.org/10.1016/j.compedu.2018.06.007

Chatzidimitris, T., Gavalas, D., & Michael, D. (2016). SoundPacman: Audio augmented reality in location-based games. In 2016 18th Mediterranean Electrotechnical Conference (MELECON) (pp. 1-6). IEEE. https://doi.org/10.1109/MELCON.2016.7495414

Choi, Y. M. (2019). Applying tangible augmented reality for product usability assessment. Journal of Usability Studies, 14(4). Retrieved from http://uxpajournal.org/wp-content/uploads/sites/8/pdf/JUS_Choi_August2019.pdf

Cohen, L., Manion, L., & Morrison, K. (2013). Research methods in education (7th Ed.). Routledge.

Connell, B.R., Jones, M., Mace, R., Mueller, J., Mullick, A., Ostroff, E., Sanford, J., Steinfeld, E., Story, M., & Vanderheiden, G. (1997). The principles of universal design. Retrieved from http://www.ncsu.edu/ncsu/design/cud/about_ud/udprinciplestext.htm

Craig, A. B. (2013). Understanding augmented reality: concepts and applications. Massachusetts: Morgan-Kaufmann Publishers. Retrieved from https://dl.acm.org/citation.cfm?id=2513619

Creswell, J. W. (2013). Qualitative inquiry & research design choosing among five approaches. Sage Publications.

Custer, R. L., Scarcella, J. A., & Stewart, B. R. (1999). The modified Delphi technique: A rotational modification. Journal of Vocational and Technical Education, 15(2), 1-10. Retrieved from https://files.eric.ed.gov/fulltext/EJ590767.pdf

Dede, C. (2005). Planning for neomillennial learning styles. Educause Quarterly, 28(1), 7-12. https://people.wou.edu/~girodm/middle/Dede_on_neo-millenials.pdf

Eby, G. (2013). Uzaktan eğitim ortamlarının tasarımı yazılım mühendisliği yaşam döngüsü yaklaşımı. Ankara: Kültür Ajans Publisher.

Essany, M. (2015). B-schools get an a for virtual reality experiments. Retrieved from http://mobilemarketingwatch.com/b-schools-get-an-a-for-virtual-reality-experiments-50923/

Fingas, R. (2016). Tim Cook says AI & augmented reality are core technologies in Apple's future. Retrieved from http://appleinsider.com/articles/16/08/14/tim-cook-says-ai-augmented-reality-are-core-technologies-in-apples-future

Frank, J. A., & Kapila, V. (2017). Mixed-reality learning environments: Integrating mobile interfaces with laboratory test-beds. Computers & Education, 110, 88-104. https://doi.org/10.1016/j.compedu.2017.02.009

Gall, M. D., Borg, W. R., & Gall, J. P. (1996). Educational research: An introduction. Longman Publishing.

Gündoğan, M. B. (2012). Uzaktan eğitime ekolojik bir yaklaşım: sürdürülebilir uzaktan bilgisayar mühendisliği eğitimi ekosistemi önerisi (Unpublished doctoral dissertation). Eskişehir, Anadolu University, Turkey.

Härmä, A., Jakka, J., Tikander, M., Karjalainen, M., Lokki, T., Hiipakka, J. & Lorho, G. (2004). Augmented reality audio for mobile and wearable appliances. Journal of the Audio Engineering Society, 52(6), 618-639.

Harr, J. (2015). Oculus rift technology may improve online learning. Retrieved from http://www.educationnews.org/online-schools/oculus-rift-technology-may-improve-online-learning/

Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 32(4), 1008-1015. https://doi.org/10.1046/j.1365-2648.2000.t01-1-01567.x

Heller, F., Jevanesan, J., Dietrich, P., & Borchers, J. (2016). Where are we? evaluating the current rendering fidelity of mobile audio augmented reality systems. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (pp. 278-282). https://doi.org/10.1145/2935334.2935365

Higbee, J. L., & Goff, E. (2008). Pedagogy and student services for institutional transformation: implementing universal design in higher education. Center for Research on Developmental Education and Urban Literacy. Retrieved from https://files.eric.ed.gov/fulltext/ED503835.pdf

Höllerer, T., & Feiner, S. (2004). Mobile augmented reality. H. Karimi ve A. Hammad (Eds.), Telegeoinformatics: Location-Based Computing and Services (pp. 1-39) . London: Taylor and Francis Books. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.111.8006&rep=rep1&type=pdf

Hsu, C. C., & Sandford, B. A. (2007). The Delphi technique: Making sense of consensus. Practical Assessment, Research & Evaluation, 12(10), 1-8. https://doi.org/10.7275/pdz9-th90

Hung, H. L., Altschuld, J. W., & Lee, Y. F. (2008). Methodological and conceptual issues confronting a cross-country Delphi study of educational program evaluation. Evaluation and Program Planning, 31(2), 191-198. https://doi.org/10.1016/j.evalprogplan.2008.02.005

Ibáñez, M. B., Portillo, A. U., Cabada, R. Z., & Barrón, M. L. (2020). Impact of augmented reality technology on academic achievement and motivation of students from public and private Mexican schools. A case study in a middle-school geometry course. Computers & Education, 145, 103734. https://doi.org/10.1016/j.compedu.2019.103734

Johnson, L., Adams Becker, S., Cummins, M., Estrada, V., Freeman, A., & Hall, C. (2016). NMC horizon report: 2016 higher education edition. Austin, Texas: The New Media Consortium. Retrieved from http://cdn.nmc.org/media/2016-nmc-horizon-report-he-EN.pdf

Jot, J. M., & Lee, K. S. (2016). Augmented reality headphone environment rendering. In Audio Engineering Society Conference: 2016 AES International Conference on Audio for Virtual and Augmented Reality. Audio Engineering Society. Retrieved from http://www.aes.org/e-lib/online/browse.cfm?elib=18506

Laviole, J., Thevin, L., Albouys-Perrois, J., & Brock, A. (2018). Nectar: Multi-user spatial augmented reality for everyone: three live demonstrations of educative applications. In Proceedings of the Virtual Reality International Conference-Laval Virtual (pp. 1-6). https://doi.org/10.1145/3234253.3234317

Lee, J., Jung, S., Kim, J. W., & Biocca, F. (2019). Applying Spatial Augmented Reality to Anti-Smoking Message: Focusing on Spatial Presence, Negative Emotions, and Threat Appraisal. International Journal of Human–Computer Interaction, 35(9), 751-760. https://doi.org/10.1080/10447318.2018.1489581

Maiti, A., Smith, M., Maxwell, A. D., & Kist, A. A. (2018). Augmented reality and natural user interface applications for remote laboratories. In Cyber-Physical Laboratories in Engineering and Science Education (pp. 79-109). Springer, Cham. https://doi.org/10.1007/978-3-319-76935-6_4

Metz, R. (2015). Augmented-reality glasses could help legally blind navigate. MIT Technology Review. Retrieved from https://www.technologyreview.com/s/538491/augmented-reality-glasses-could-help-legally-blind-navigate/

Metz, R. (2016). Augmented reality study projects life-sized people into other rooms. MIT Technology Review. https://www.technologyreview.com/s/545466/augmented-reality-study-projects-life-sized-people-into-other-rooms/

Moore, M. G., & Kearsley, G. (2011). Distance education: a systems view of online learning. Cengage Learning.

Neuendorf, K. A. (2002). The content analysis guidebook. Sage Publications. Retrieved from https://academic.csuohio.edu/kneuendorf/SkalskiVitae/SkalskiNeuendorfCajigas17.pdf

Núñez, M., Quirós, R., Núñez, I., Carda, J. B., Camahort, E., & Mauri, J. L. (2008). Collaborative augmented reality for inorganic chemistry education. WSEAS International Conference Proceedings Mathematics and Computers in Science and Engineering (Vol. 5, pp. 271-277). Retrieved from https://pdfs.semanticscholar.org/bf33/32d422c37f785ca6c73a8003cabb2cabd163.pdf

Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: An example, design considerations and applications. Information & Management, 42(1), 15-29. https://doi.org/10.1016/j.im.2003.11.002

Onime, C., & Abiona, O. (2016). 3D mobile augmented reality interface for laboratory experiments. International Journal of Communications, Network and System Sciences, 9(4), 67-76. http://dx.doi.org/10.4236/ijcns.2016.94006

Parviz, B. A. (2009). Augmented reality in a contact lens. Retrieved from http://spectrum.ieee.org/biomedical/bionics/augmented-reality-in-a-contact-lens/0

Pejsa, T., Kantor, J., Benko, H., Ofek, E., & Wilson, A. (2016). Room2Room: Enabling life-size telepresence in a projected augmented reality environment. Retrieved from http://pages.cs.wisc.edu/~tpejsa/files/16-CSCW-Room2Room.pdf

Pérez-López, D., & Contero, M. (2013). Delivering educational multimedia contents through an augmented reality application: A case study on its impact on knowledge acquisition and retention. TOJET: The Turkish Online Journal of Educational Technology, 12(4). Retrieved from https://files.eric.ed.gov/fulltext/EJ1018026.pdf

Perry, T. S. (2020). Augmented reality in a contact lens: it’s the real deal. Retrieved from https://spectrum.ieee.org/view-from-the-valley/consumer-electronics/portable-devices/ar-in-a-contact-lens-its-the-real-deal

Rauschnabel, P. A., Brem, A., & Ivens, B. S. (2015). Who will buy smart glasses? Empirical results of two pre-market-entry studies on the role of personality in individual awareness and intended adoption of Google Glass wearables. Computers in Human Behavior, 49, 635-647. https://psycnet.apa.org/doi/10.1016/j.chb.2015.03.003

Rauschnabel, P. A., & Ro, Y. K. (2016). Augmented reality smart glasses: An investigation of technology acceptance drivers. International Journal of Technology Marketing, 11(2), 123-148. http://dx.doi.org/10.1504/IJTMKT.2016.075690

Redondo, B., Cózar-Gutiérrez, R., González-Calero, J. A., & Ruiz, R. S. (2020). Integration of Augmented Reality in the Teaching of English as a Foreign Language in Early Childhood Education. Early Childhood Education Journal, 48(2), 147-155. https://doi.org/10.1007/s10643-019-00999-5

Reid, N. (1988). The Delphi technique: Its contribution to the evaluation of professional practice. E, Roger. (Ed.), Professional Competence and Quality Assurance in the Caring Professions. London: Chapman & Hall.

Ro, Y. K., Brem, A., & Rauschnabel, P. A. (2018). Augmented reality smart glasses: Definition, concepts and impact on firm value creation. In Augmented reality and virtual reality (pp. 169-181). Springer, Cham. https://doi.org/10.1007/978-3-319-64027-3_12

Rossi, D. (2018). SAR for Kids: Spatial augmented reality as tool for art education. In International and Interdisciplinary Conference on Digital Environments for Education, Arts and Heritage (pp. 355-363). Springer, Cham. https://doi.org/10.1007/978-3-030-12240-9_38

Ruiz-Ariza, A., Casuso, R. A., Suarez-Manzano, S., & Martínez-López, E. J. (2018). Effect of augmented reality game Pokémon GO on cognitive performance and emotional intelligence in adolescent young. Computers & Education, 116, 49-63. https://doi.org/10.1016/j.compedu.2017.09.002

Sahin, D., & Yilmaz, R. M. (2020). The effect of Augmented Reality Technology on middle school students' achievements and attitudes towards science education. Computers & Education, 144, 103710. https://doi.org/10.1016/j.compedu.2019.103710

Scavo, G., Wild, F., & Scott, P. (2015). The GhostHands UX: Telementoring with hands-on augmented reality instruction. Retrieved from http://www.tellme-ip.eu/sites/default/files/library/AISE19-0236.pdf

Seuring, S., & Müller, M. (2008). Core issues in sustainable supply chain management–a Delphi study. Business Strategy and the Environment, 17(8), 455-466. https://doi.org/10.1002/bse.607

Skulmoski, G. J., Hartman, F. T., & Krahn, J. (2007). The Delphi method for graduate research. Journal of Information Technology Education, 6, 1-21. Retrieved from https://www.learntechlib.org/p/111405/

Smith, R. E., & Buchannan, T. (2012). Community collaboration, use of universal design in the classroom. Journal of Postsecondary Education and Disability, 25(3), 259-265. Retrieved from https://files.eric.ed.gov/fulltext/EJ994291.pdf

Sumsion, T. (1998). The Delphi technique: An adaptive research tool. The British Journal of Occupational Therapy, 61(4), 153-156. https://doi.org/10.1177%2F030802269806100403

Sun, S. W., & Lan, Y. S. (2019). Augmented reality displaying scheme in a smart glass based on relative object positions and orientation sensors. World Wide Web, 22(3), 1221-1239. https://doi.org/10.1007/s11280-018-0592-z

Takahashi, D. (2020). Mojo Vision is putting an augmented reality screen on a contact lens. Retrieved from https://venturebeat.com/2020/01/16/mojo-vision-is-putting-an-augmented-reality-screen-on-a-contact-lens/

Tashev, I. J. (2019). Capture, representation, and rendering of 3d audio for virtual and augmented reality. International Journal on Information Technologies & Security, 11(2), 49-62. Retrieved from http://ijits-bg.com/contents/IJITS-SP2-2019/2019-SP2-05.pdf

Thangaratinam, S., & Redman, C. W. (2005). The delphi technique. The Obstetrician & Gynaecologist, 7(2), 120-125. https://doi.org/10.1576/toag.7.2.120.27071

The New Economy (2014). Using mobiles to smell: how technology is giving us our senses. Retrieved from http://www.theneweconomy.com/technology/using-mobiles-to-smell-how-technology-is-giving-us-our-senses-video

Topa-Çiftçi, G. (2011). Uzaktan eğitimde iptv’nin kullanılabilirliğine ilişkin bir delphi çalışması (Unpublished master’s dissertation). Eskişehir, Anadolu University, Turkey.

Tussyadiah, I. P., Jung, T. H., & Tom Dieck, M. C. (2018). Embodiment of wearable augmented reality technology in tourism experiences. Journal of Travel Research, 57(5), 597-611. https://doi.org/10.1177%2F0047287517709090

Vargas, H., Farias, G., Sanchez, J., Dormido, S., & Esquembre, F. (2013). Using augmented reality in remote laboratories. International Journal of Computers Communications & Control, 8(4), 622-634. Retrieved from http://univagora.ro/jour/index.php/ijccc/article/view/42/pdf_59

Westerfield, G., Mitrovic, A., & Billinghurst, M. (2015). Intelligent augmented reality training for motherboard assembly. International Journal of Artificial Intelligence in Education, 25(1), 157-172. https://doi.org/10.1007/s40593-014-0032-x

Williams, P. L., & Webb, C. (1994). The Delphi technique: A methodological discussion. Journal of Advanced Nursing, 19(1), 180-186.

Yıldırım, A., & Şimşek, H. (2013). Sosyal bilimlerde nitel araştırma yöntemleri. Seçkin Publisher, Turkey.

Yin, R. K. (2003). Case study research: Design and methods. Sage Publications.

Yoon, B., Kim, H. I., Lee, G. A., Billinqhurst, M., & Woo, W. (2019). The effect of avatar appearance on social presence in an augmented reality remote collaboration. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (pp. 547-556). IEEE.




DOI: http://dx.doi.org/10.5944/openpraxis.12.2.1017

Refbacks

  • There are currently no refbacks.