Skip to content

Advertisement

  • Research
  • Open Access

Evaluation of the ARTutor augmented reality educational platform in tertiary education

Smart Learning Environments20185:6

https://doi.org/10.1186/s40561-018-0058-x

  • Received: 10 July 2018
  • Accepted: 27 July 2018
  • Published:

Abstract

Augmented Reality (AR) has seen a significant increase in applications in a number of fields in recent years. One of the areas in which AR has been applied to is education, most commonly by means of augmenting educational books. The present paper builds on previous work by using an AR authoring environment and a mobile application, developed by the authors, in undergraduate courses at the Eastern Macedonia & Thrace Institute of Technology. Using the authoring tool, the students were able to enhance existing secondary education textbooks by adding digital content to them, and, using the mobile application, view the digital content and retrieve information from the textual content of the books by asking questions in natural language form. At the end of the semester, the students were asked to evaluate the ARTutor platform by means of questionnaires and the SECTIONS framework. This study presents and discusses the results of this evaluation exercise and proposes new directions of the research.

Keywords

  • Augmented reality
  • Mobile learning
  • Mobile application
  • Authoring tool
  • User-contributed content
  • Independent learning

Introduction

Augmented Reality is a technology which enriches human senses and mixes real and virtual environments, leading to a new, more informative and stimulating environment. The principle of an AR application is to use a device’s sensors to perceive the environment and to embed dynamic, context-aware and interactive digital content to it. AR applications can be used on many different platforms such as desktops, notebooks and mobile devices; however, AR applications are usually available through mobile devices, such as smartphones and tablets or wearable devices. Built-in cameras, GPS sensors, gyroscopes and other sensors are used to recognize objects, images and scenes. When recognition is successful, relevant digital content becomes available and is displayed on screen. The objective is to seamlessly blend the environment with the digital content. This enables the user to receive more information about his environment that would have otherwise been available. The advantage of AR is not only the increase of available information in the environment, but also the creation of a certain representation of the world which is appealing to the user. For these reasons, AR has been applied to a number of areas including entertainment, tourism, commerce, engineering and others (Bhorkar 2017).

The ability of enhancing what is perceived in the environment with interactive digital content has created an interest in using AR applications for educational purposes. In the field of mobile learning, the new possibilities for teaching and learning have increasingly been recognized by educational researchers (Wu et al. 2013), (Chen et al. 2017). AR is currently being applied across disciplines in primary, secondary and higher education, and has been found to increase the academic success levels and motivations of students (Martin Gutierrez and Meneses Fernandez 2014). In a recent systematic review of 17 studies between 2012 and 2017, it is argued that AR in education can potentially influence the students’ attendance, knowledge transfer, skill acquisition, hands-on digital experience in education in a variety of domains (Fotaris et al. 2017). A review presented in (Bacca et al. 2014) summarizes research where it is shown that AR is useful in increasing students’ motivation and presents learning gains through interaction, and collaboration. Other studies, such as (Shelton and Hedley 2002), conclude that AR is particularly suitable for the instruction of subjects that are not easily experienced and understood because of real-world constraints.

AR is not only beneficial to the individual students’ study, but with the aid of the appropriate software tools it can also be applied to collaborative learning approaches (Martín-Gutiérrez et al. 2015). For example, in (Gonzàlez et al. 2012) the work focuses on cultural heritage education, where students identify places of cultural interest using geolocation and use software to access publicly available multimedia content. In another work outlined in (Kaufmann 2003), students use specialized AR software for the collaborative study of mathematics and geometry.

Apart from mobile applications that use AR for educational purposes, there have been various authoring tools developed to allow teachers to create AR educational scenarios. For example, ARLearn is an authoring tool that supports the creation of training scenarios and serious games based on decision making (Ternier et al. 2012). Also, mLearn is a platform developed by the authors to help teachers create and manage educational field trips and enable students to access the educational material from their mobile phones (Lytridis and Tsinakos 2017).

A common practice when using AR in education is creating augmented books. These are software applications used to scan the pages of traditional printed educational material and display the digital content associated with those pages on a screen. The applications are installed in devices such as mobile phones, tablets, or even personal computers enhanced with cameras. Through the use of AR in printed book pages, books can become dynamic sources of information. Design aspects of mixed reality books have been first studied in (Grasset et al. 2008). A number of studies have shown the benefits in using AR books as learning enhancement tools for motivating children and helping with better understanding of concepts (Cheng and Tsai 2016), (Dünser and Hornecker 2007). Some examples of specialized applications for specific subjects have been introduced in (Ferrer-Torregrosa et al. 2015) for anatomy and in (Weng et al. 2016) for biology. (Karamanoli and Tsinakos 2016) created an Android application that uses the commercial Aurasma tool to provide augmentations in the Greek history book of the 4th grade of primary school. (Hung et al. 2017) developed an animated AR graphics book displaying 3D images of bacteria. It utilized a virtual reality head-mounted display with a small video camera attached at the same level as the students’ eyes and the results showed that AR improved students’ learning performance and raised their motivation. Also, (Tobar-Muñoz et al. 2017) presented an AR game inspired by traditional “pop up” books, which is played in a naturalistic environment. Each page of the book comes to life using book illustrations as AR markers.

It is obvious from the brief review of the literature presented above that there have been significant advances in the use of AR in education. However, there are still many challenges that need to be overcome especially in the area of AR books, summarized in the following:
  1. a)

    AR applications are currently being developed to be used with specific books. The development of such specialized applications clearly requires expert programmers. In the absence of suitable authoring tools, a teacher planning to introduce AR in his courses would either need to acquire fairly advanced software development skills or employ expert help in order to realize such plans.

     
  2. b)

    The educational material is, in most cases, built in an AR application and any changes to the material or the book itself would require a redesign of the application, which implies further engagement of skilled programmers to update the application. The additional costs and effort needed might not be attainable, especially for individual books.

     
  3. c)

    In several AR books today, markers are used as a way for an application to identify the pages where there is digital content. Some purpose-made educational material can indeed be created to include these markers. Existing books however, would need to be redesigned and reprinted, and this would incur a large unnecessary cost.

     
  4. d)

    AR applications are usually targeted to individual books and so the study of multiple subjects using AR by a student demands the installation of multiple applications.

     
  5. e)

    Current AR books only allow students to interact with the digital objects that appear on top of the physical books. These additional sources of information do lead to a better understanding of the learning material, but information retrieval from the textual content of the book is not aided by AR.

     

The identification of these problems has led the authors to develop ARTutor, a general-purpose AR-based educational platform, first presented in (Lytridis et al. 2018). The aim of this platform is to facilitate the creation of AR books by providing an easy to use authoring tool, and to provide a single mobile application that will work with every book created with the authoring tool.

The current paper describes the first experiences in integrating ARTutor in a higher education course at the Eastern Macedonia and Thrace Institute of Technology in Greece and the results of its first large scale evaluation. The paper is outlined as follows: In Section “The ARTutor platform” the ARTutor platform and its functionality is presented. Section “Evaluation of ARTutor” describes how ARTutor was made part of the undergraduate courses and how the software was evaluated. Section “Conclusions” discusses the experimental results and proposes future directions in the research.

The ARTutor platform

ARTutor is an educational platform that aims at integrating augmented reality in the educational process in a simple way for both educators and students. It has been developed at the Advanced Educational Technologies and Mobile Applications Research Lab at the Eastern Macedonia and Thrace Institute of Technology. Its focus is on adding digital content on traditional educational books and other texts, with the aim of assisting the students’ independent study and ultimately improving the understanding of the material.

In this section, an overview of the functionality of the ARTutor authoring tool and mobile application is presented.

The authoring tool

The authoring tool is a web-based application designed to enable teachers to create AR books (Lytridis 2017a). To access the authoring tool, the teacher must visit the ARTutor portal and register for free. The portal also allows users to browse and download existing AR books. After successful login to the system, AR books previously created by the user (with an option to edit or delete them) and the option to create a new AR book are displayed. When creating a new book, the form shown in Fig. 1 appears. In this form, the teacher needs to enter some information on the book, to select a book category and the book language (Greek and English are currently available), and upload his educational material (the book) in PDF format. The file is uploaded with a simple drag & drop action to the corresponding area.
Fig. 1
Fig. 1

a Existing books table and (b) book creation form

After the uploading of the book is complete, the teacher saves all the book information. During this process, the PDF file is stored on the server and the book information on a database. Also, the text from the PDF file is extracted and indexed in order to support an information retrieval algorithm, as it will be described later. At this point, the teacher is now able to add augmentations to the book. To add an augmentation to the book, the teacher must create an asset, i.e. an object which associates an area of some page of the book to a digital object (i.e. the augmentation). This is done through an asset creation form, where the teacher must give a name to the asset, create the trigger image and upload the corresponding augmentation. As illustrated in Fig. 2, the creation of the trigger image is achieved by opening the PDF file, selecting a page, and marking the desired area of the page. When the area is selected, it is saved as an image and displayed in the asset creation form. Then, the teacher imports the corresponding digital object, which can be an image, a video, an audio clip or a 3D model.
Fig. 2
Fig. 2

Asset creation

In the same way, the teacher can add multiple augmentations to the book. All assets are displayed in a table below the book information form, where they can be edited or deleted.

Τhe authoring tool is designed in such a way that teachers can enhance their traditional educational material, even if their IT skills are limited. Books do not need to be reprinted to include specialized markers, as is the case in other approaches. The simplicity of the process of creating an augmented book means that there is no need for specialized training in the use of the software or required software and design skills. In addition, the teacher is not expected to use other software tools in conjunction with the ARTutor authoring tool, since (a) the original books are common PDF files, and (b) the augmentations added in the books’ pages are common images, videos and audio clips that can be retrieved from various sources or created using non-specialized software. In contrast to other general-purpose augmented reality authoring tools, the trigger images are created on the ARTutor authoring tool itself, and do not need to be generated and uploaded separately for each page of the same book. This approach not only allows the easy creation of trigger images, but it also ensures that each book is treated as a single entity, and not as a collection of target images. The ARTutor authoring tool is free to use for academic purposes, and the resulting augmented books are to be visible both in the portal as well as to everyone using the mobile application.

The mobile application

The ARTutor mobile application is the tool which students use to scan the learning material used in the authoring tool, view the corresponding digital content and interact with the augmentations and book content. The application is available for both Android and iOS platforms (Lytridis 2017b).

Initially the application displays all the book categories and the number of books in each category. Selecting a category reveals the books of this category as defined in the authoring tool as seen in Fig. 3.
Fig. 3
Fig. 3

The ARTutor mobile application user interface

By selecting a book, the application retrieves all the relevant assets from the server and initiates the camera. The student can then start scanning the various pages of the book and when a trigger image is detected, the corresponding augmentation is displayed. Example augmentations after trigger image detection are shown in Fig. 4.
Fig. 4
Fig. 4

Example augmentations using the ARTutor mobile application

As it can be seen in Fig. 4, apart from the augmentations, when a trigger image is detected various additional buttons appear on screen. With these buttons the student can view which pages of the book contain augmentations and see which commands are available for each displayed augmentation. The commands are ways in which the student can interact with the augmentations either using touch gestures or by voice commands. Table 1 shows the available commands depending on the type of displayed augmentation.
Table 1

Commands for interacting with augmentations

Command

Type of augmentation

Gesture

Bigger

Image, 3D model

Pinch

Smaller

Image, 3D model

Pinch

Start video

Video

Touch on object

Pause video

Video

Touch on object

Rotate

Image, 3D model

Slide finger across

To issue a verbal command, the user must press the ARTutor button displayed on the bottom right hand corner of the screen. Voice commands allow hands-free operation and also provide an accessibility feature which enables students with limited motor skills to interact with the augmentations.

The mobile application also has a novel feature where the student can ask questions in natural language form and receive the answer based on the textual content of the book. As was mentioned earlier, when the book is created, its textual contents are extracted, indexed and stored on the server. The student can use the ARTutor button to ask a question. The question is then sent to the server where an information retrieval algorithm is used to extract the relevant information and return the appropriate answer. The answer is sent back to the application where it is converted to speech and also displayed on the screen as a popup message. The capability if the application to perform searches on the content of the educational material is a novel approach designed to reinforce the self-study practice and independent learning process by providing a virtual tutor who, based on the learning material, will provide relevant information according to the students’ questions.

The ARTutor mobile application allows students to enhance their understanding of the teacher’s material by displaying explanatory interactive digital content on top of the traditional book. The concepts presented in the various pages of the book are better visualized using the various types of augmentations and additional information can be made available. The augmentations can be activated either by pointing to the electronic or the printed version of the book. Contrary to other augmented reality applications, the ARTutor application is not developed for a specific book: the content accessible to students is dynamic and more books can be accessed by the application as more books are added to the authoring tool.

In summary, the ARTutor platform is a novel educational tool with the following advantages:
  • Easy enhancement of existing learning material by teachers

  • Easy access to and interaction with the enhanced digital content by students

  • Voice-based interaction with a virtual tutor for information retrieval

  • Consideration of students with disabilities by the introduction of accessibility features

  • Possibility of collaborative development of AR content

  • The same book can have multiple versions. This allows the possibility of each teacher/user adding different content on a book, perhaps concentrating on different aspects of its content, and the students selecting the best or more appropriate version for their study.

Evaluation of ARTutor

As reported in previous work, ARTutor has already undergone the first stage of formative evaluation by a small number of users and experts. After some initial adjustments on the functionality and the user interface of the authoring tool and the mobile application during this initial stage, the platform was ready to be assessed by a larger number of users and test the entirety of its features. In this section the experimental setup and the results will be presented.

Setup

For the task of using and evaluating the ARTutor platform, students attending two undergraduate units, namely “Learning and Teaching Theories” and “Educational Technologies” were engaged. These units are taught during the second and fourth semester of the undergraduate studies of the Computing and Informatics Department at the Eastern Macedonia and Thrace Institute of Technology. ARTutor was made part of the semester’s coursework in both units in the following way: A set of 15 books taught in secondary Greek schools were selected, in subjects such as information technology, history, mathematics and geography. The electronic versions of these books are freely available in PDF format from the Greek Ministry of Education. These subjects were selected because of their potential in enhancing the material with graphics of educational value. Students of these two units were divided in groups of five and each group was assigned a portion of one of the selected school books. The assignment given to the students was to study their allocated portion of the book, identify images that can be used as triggers, and find multimedia content (images, videos, audio clips and 3D models) that would serve as augmentations and enhance the book content. They were also required to subsequently use the mobile application to view and interact with their augmentations. In this way, the students were able to experience the platform both as teachers as well as students.

In the case of the “Learning and Teaching Theories” unit, the students were also instructed to focus on the development of original educational videos relevant to the subject of their allocated book pages. All groups of students were then required to add their multimedia content to the books that were prepared in ARTutor beforehand. It should be noted that all students used the same ARTutor account, which means that groups assigned to different portions of the same book were working in parallel and essentially developing each augmented book collaboratively.

Moreover, the students attending the “Educational Technologies” unit were given group assignments where they were required to evaluate ARTutor using the SECTIONS evaluation framework, initially proposed in (Bates and Poole 2003), in order to determine the most effective use of the technology in various subjects and in various academic settings. The students submitted reports which contained their group analysis for each of the criteria of the SECTIONS framework, i.e. Students, Ease of use, Cost, Teaching and learning, Interactivity, Organizational issues, Novelty, Speed. The objective of this exercise was to gather students’ opinion on these issues and particularly on the suitability of this educational technology for all the stages of educations as well as for adult education and professional training.

In addition, at the end of the semester, the students of both units were asked to individually complete a questionnaire designed to assess the functionality of ARTutor in terms of ease of use and usefulness according to the Technology Acceptance Model (TAM). According to TAM, the user-perceived ease of use and usefulness of a technology are two determinants of its adoption (Davis 1989). The questionnaire was based on the Software Usability Measurement Inventory (SUMI) developed in the “Metrics for Usability Standards in Computing” project by the Human Factors Research Group at the University College Cork (Bevan and Macleod 1994). It contains 50 questions that the user has to reply to with either “Agree”, “Disagree” or “Don’t Know”. More specifically, the questionnaire investigates efficiency, affect, helpfulness, controllability and learnability. Finally, at the end of the questionnaire the students were given the opportunity to comment on which aspects of the software they found useful or problematic.

Results

In this section, the results gathered from the questionnaires will be presented and discussed and a summary of the analyses based on the SECTIONS framework will be provided.

Questionnaires

In total, 205 students completed the evaluation questionnaires. Initially, the degree to which the students found the software important and their computer skills were examined with two corresponding questions, as seen in Fig. 5.
Fig. 5
Fig. 5

Importance of the technology and software skills of the students

The results presented in the charts above indicate the students’ motivation and incentive to use ARTutor and signify that the test group has average to advanced IT skills.

Questions 1–10 of the questionnaire are dealing with how the software affects users and are presented in Table 2. Overall, students find that the software made them feel comfortable while using it, with helpful prompts and instructions, reliable operation and easy-to-learn tasks. Finally, nearly half of the students would recommend the software to their colleagues as opposed to 16% that would not.
Table 2

Questions 1–10 on affect

 

Question

Agree

Disagree

Undecided

1

I would recommend this software to my colleagues.

47%

16%

37%

2

I enjoy the time I spend using this software.

32%

29%

39%

3

I sometimes don’t know what to do next with this software.

32%

44%

24%

4

If this software stops it is not easy to restart it.

25%

53%

22%

5

This software responds too slowly to inputs.

39%

31%

30%

6

The instructions and prompts are helpful.

71%

8%

21%

7

I find that the help information given by this software is not very useful.

24%

46%

30%

8

This software has at some time stopped unexpectedly.

61%

23%

16%

9

Learning to operate this software initially is full of problems.

21%

50%

29%

10

It takes too long to learn the software functions.

14%

69%

17%

Table 3 shows the questions and the student answers in questions regarding controllability. It can be seen that the users feel strongly that the user interface is well-designed and the various tasks are performed quickly. It is also reported that learning to use new functions is not difficult to do and feel confident that they can learn all the functions that the software has to offer. Users generally feel in control when using the software.
Table 3

Questions 11–20 on controllability

 

Question

Agree

Disagree

Undecided

11

I think this software has sometimes given me a headache.

35%

48%

17%

12

It is obvious that user needs have been fully taken into consideration.

42%

19%

40%

13

The organisation of the menus seems quite logical.

76%

11%

13%

14

The software allows the user to be economical of keystrokes.

65%

8%

27%

15

There are too many steps required to get something to work.

22%

59%

19%

16

There have been times in using this software when I have felt quite tense.

19%

20%

61%

17

Error messages are not adequate.

39%

22%

38%

18

Learning how to use new functions is difficult.

13%

65%

22%

19

It is easy to make the software do exactly what you want.

47%

24%

29%

20

I will never learn to use all that is offered in this software.

16%

64%

21%

In terms of efficiency, students are generally satisfied with working with ARTutor. They report to be very comfortable with the way information is presented and feel that the accompanying documentation is informative. The students also report that they prefer to use functions that they are more familiar with (Table 4).
Table 4

Questions 21–30 on efficiency

 

Question

Agree

Disagree

Undecided

21

I sometimes wonder if I am using the right function.

31%

49%

20%

22

Working with this software is satisfying.

59%

19%

22%

23

I feel safer if I use only a few familiar functions.

50%

26%

24%

24

This software seems to disrupt the way I normally like to arrange my work.

25%

38%

37%

25

I feel in command of this software when I am using it.

60%

18%

23%

26

The way that system information is presented is clear and understandable.

67%

10%

23%

27

The software documentation is very informative.

46%

14%

41%

28

There is never enough information on the screen when it’s needed.

30%

42%

28%

29

I prefer to stick to the functions that I know best.

54%

22%

24%

30

Working with this software is mentally stimulating.

37%

21%

43%

As far as helpfulness is concerned, students appear to be quite satisfied with getting help from the software itself. They are not frustrated when using the software as the information provided is adequate and that tasks are straightforward to be performed in a timely manner, and it is reported that the need to go back to the documentation is rare. The results for this category of questions are presented in more detail in Table 5.
Table 5

Questions 31–40 on helpfulness

 

Question

Agree

Disagree

Undecided

31

I would not like to use this software every day.

47%

25%

28%

32

Using this software is frustrating.

19%

62%

19%

33

I think this software is inconsistent.

27%

38%

35%

34

This software is awkward when I want to do something which is not standard.

40%

23%

37%

35

I can understand and act on the information provided by this software.

65%

7%

28%

36

Tasks can be performed in a straight forward manner using this software.

68%

14%

19%

37

The speed of this software is fast enough.

30%

44%

25%

38

The software has helped me overcome any problems I have had in using it.

33%

26%

41%

39

There is too much to read before you can use the software.

16%

66%

18%

40

I keep having to go back to look at the guides.

20%

60%

21%

As far as learnability is concerned, students feel that it is fairly easy to learn to use the platform: the majority of students do not find it awkward and think that it is easy to gradually complete tasks with information on how to proceed at each stage being available. Ease of use is also asserted by the fact that most students feel that it is hard to forget how to use the software. Results for the learnability category are presented in Table 6.
Table 6

Questions 41–50 on learnability

 

Question

Agree

Disagree

Undecided

41

The software presents itself in a very attractive way.

40%

25%

35%

42

This software is really very awkward.

22%

61%

18%

43

The software hasn’t always done what I was expecting.

47%

29%

24%

44

It is relatively easy to move from one part of a task to another.

57%

19%

24%

45

This software occasionally behaves in a way which can’t be understood.

30%

45%

25%

46

Getting data files in and out of the system is not easy.

23%

50%

28%

47

Either the amount or quality of the help information varies across the system.

38%

15%

47%

48

It is easy to see at a glance what the options are at each stage.

60%

11%

29%

49

I have to look for assistance most times when I use this software.

22%

58%

20%

50

It is easy to forget how to do things with this software.

22%

64%

14%

As far as the students’ general comments on the pros and cons of the platform, the students mostly reported that they found the use of both the authoring tool and the mobile application very easy, and for most tasks they did not need to refer to the software’s documentation. They found the ability to interact with the augmentations quite useful and they were particularly interested in the questions they could pose to the ARTutor agent in order to retrieve information from the book’s contents, although it was reported that this system did not yield good results in some cases. On the other hand, negative comments mainly focused on the fact that the application required a lot of memory for books with many augmentations assigned to it. Students also found that even though the inclusion of 3D models in an AR book is appealing, it is not an easy task mainly because it requires 3D design skills given the scarce availability of free 3D models, especially those that are explanatory of some concept. It appears that 3D models are the most promising in terms of the capability of illustrating concepts and consequently they are of an important educational value.

SECTIONS framework

The most prominent students’ views are featured here, categorized by criterion:
  • Students

    On this criterion there has been extensive analysis on the suitability of ARTutor at the various stages of education. Most students have discussed that the limitation in the adoption of ARTutor as an educational technology is the ability of students to use mobile devices. Since this happens at an early age, the possibility of using ARTutor as early as in primary education was suggested.

  • Ease of use

    This was universally recognized as a prominent characteristic of the technology for both students and teachers, even those that have limited experience with software tools. Tasks are reported to be intuitively easily achieved and there were no issues with the user interface. However, there have been some criticism as far as the reliability of the mobile application was concerned, due to some bugs that were found on the Android version.

  • Cost

    In this criterion the students stated that the registration to the authoring tool and download of the mobile application was free.

  • Teaching and learning

    Here, students have focused on the learning theories that are compatible with ARTutor as a learning tool. They report that constructionism and independent learning will be the underlying learning theories in courses where ARTutor is employed. It is highlighted that videos and especially 3D models are the types of content to impact the better understanding of concepts more significantly.

  • Interactivity

    It was generally noted that a strong interactivity feature was the ability of students to interact with the augmentations via gestures and voice commands. Some students interpreted the ability to ask questions as a form of student-teacher interaction. The lack of student-student interaction capabilities and the absence of a feedback mechanism were also noted.

  • Organizational issues

    Most students agree that there are no organizational issues that would prevent the choice of this educational technology, since it is designed for self-study and so no specialized equipment or technical staff would be required at the institute adopting it. The teachers would only need a computer and an internet connection to develop and upload his educational material, and the students would only have to download the corresponding mobile application.

  • Novelty

    Students collectively mentioned that ARTutor is a novel application suited to education and highlighted differences between other AR platforms designed for advertisement and commercial purposes.

  • Speed

    Due to the simplicity of the processes involved in creating AR books that is offered by the platform, an educator wanting to integrate ARTutor to his courses would not require a long time to adopt this technology. In addition, the fact that existing books and notes are used with ARTutor means that the educational material would not need modifications in order to incorporate AR, except for the virtual addition of the digital content.

Conclusions

This paper presented the results of an evaluation study of the ARTutor platform, carried out by students at the Department of Computing and Informatics at the Eastern Macedonia and Thrace Institute of Technology. The value of the platform in terms of user-perceived ease of use and usefulness as well as its suitability for AR-based learning were assessed. The main theme which arises from the large-scale evaluation of the software is its evident ease of use both from the point of view of the teacher adding augmentations to books, as well as from the point of view of the student that uses the mobile application to view the interactive digital content superimposed on the printed book. This reinforces the argument in favor of ARTutor’s suitability as a general-purpose AR educational tool.

The evaluation of the platform by a large number of users has led to a number of improvements of the ARTutor platform in terms of user interface and functionality. Feedback from students has been particularly useful in highlighting issues which were not evident in the first phase of the evaluation and in suggesting additions that could improve the user experience. It must be noted that the results are clearly influenced by the students’ subject of study. It therefore has to be assumed that their evaluation was more technically-oriented, rather than focused on the educational aspects.

While the usability aspect of the ARTutor platform was investigated in this paper, it is crucial to also establish the educational benefits of this educational technology and determine its effect on the learning process. The obvious next step is therefore to use ARTutor as part of the curriculum in the various stages of education: primary, secondary and tertiary. The results of such a systematic study would help establish a link between the educational use of AR and learning outcomes. For this study, the books already developed by the students could be used, or teachers could participate by developing material for their own courses. In this way, teacher attitudes towards AR and ARTutor in particular could be measured. In addition, collaborative course material development by students for their own courses can also be investigated in order to test whether their motivation is increased.

The software itself can be further developed. For example, the possibility of cloud-based recognition can be explored, in order to solve the problem with phone memory usage and large downloads. Also, the characteristics and preferences of the student can be taken into account in order to create a student profile and therefore deliver a more personalized experience when using the application. Moreover, the authors are currently investigating the incorporation of ARTutor with a Learning Management System (LMS) such as Moodle so that ARTutor is more integrated with distance learning courses. Other aspects such as support for more languages for the user interface and the information retrieval algorithm and improvements in accessibility are also considered important and will be pursued further.

Abbreviations

AR: 

Augmented reality

LMS: 

Learning management system

TAM: 

Technology acceptance model

Declarations

Availability of data and materials

The ARTutor authoring tool can be found at http://aetma.teiemt.gr/aetma/artutor. The ARTutor mobile application can be found at Google Play for the Android version and at the AppStore for the iOS version.

Authors’ contributions

CL developed the authoring tool and the mobile application and was responsible for the collection of the evaluation data. AT designed the evaluation procedure and supervised the analysis and interpretation of the evaluation results. Both authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
AETMA Lab, Eastern Macedonia & Thrace Institute of Technology, Agios Loukas, 65404 Kavala, Greece

References

  1. J Bacca, S Baldiris, R Fabregat, S Graf, Augmented reality trends in education: A systematic review of research and applications. Educ. Technol. Soc. 17(4), 133–149 (2014). https://www.jstor.org/stable/jeductechsoci.17.4.133.
  2. AW Bates, G Poole, in Effective teaching with technology in higher education: foundations for success. A Framework for Selecting and Using Technology (2003), pp. 75–105 Retrieved from https://goo.gl/pr6Wuf Google Scholar
  3. N Bevan, M Macleod, Usability measurement in context. Behav. Inform. Technol. 13(1–2, 132), –145 (1994) https://doi.org/10.1080/01449299408914592
  4. G Bhorkar, A Survey of Augmented Reality Navigation. 8(2), 73–272 (2017) https://doi.org/10.1561/1100000049
  5. P Chen, X Liu, W Cheng, R Huang, A review of using augmented reality in education from 2011 to 2016. Springer science+business media Singapore (2017) https://doi.org/10.1007/978-981-10-2419-1_2
  6. KH Cheng, CC Tsai, The interaction of child-parent shared reading with an augmented reality (AR) picture book and parents’ conceptions of AR learning. Br. J. Educ. Technol. 47(1), 203–222 (2016). https://doi.org/10.1111/bjet.12228 View ArticleGoogle Scholar
  7. FD Davis, Perceived Usefulness, Perceived Ease of Use , and User Acceptance of lnformation Technology. MIS Q. 13(3), 319–340 (1989). https://doi.org/10.2307/249008 View ArticleGoogle Scholar
  8. A Dünser, E Hornecker, Lessons from an AR book study. Proceedings of the 1st International Conference on Tangible and Embedded Interaction - TEI ‘07, 179 (2007). https://doi.org/10.1145/1226969.1227006 Google Scholar
  9. J Ferrer-Torregrosa, J Torralba, MA Jimenez, S García, JM Barcia, ARBOOK: Development and assessment of a tool based on augmented reality for anatomy. J. Sci. Educ. Technol. 24(1), 119–124 (2015). https://doi.org/10.1007/s10956-014-9526-4 View ArticleGoogle Scholar
  10. P Fotaris, N Pellas, I Kazanidis, P Smith, in Proceedings of the 11th European Conference on Game-Based Learning (ECGBL 2017). A Systematic Review of Augmented Reality Game-Based Applications in Primary Education (Academic Conferences and Publishing International Limited, Reading, 2017).Google Scholar
  11. F Gonzàlez, L Villarejo, O Miralbell, JM Gomis, How to use mobile technology and augmented reality to enhance collaborative learning on cultural and natural heritage? An e-learning experience. Procedia Soc. Behav. Sci. XX New Trends Glob. Educ. Conference North Cyprus 2012, 0 (2012)Google Scholar
  12. R Grasset, A Dünser, M Billinghurst, The design of a mixed-reality book: Is it still a real book? Proc. 7th IEEE Int. Symp. Mixed Augmented Real. 2008 ISMAR 2008, 99–102 (2008). https://doi.org/10.1109/ISMAR.2008.4637333 View ArticleGoogle Scholar
  13. YH Hung, CH Chen, SW Huang, Applying augmented reality to enhance learning: A study of different teaching materials. J. Comput. Assist. Learn. 33(3), 252–266 (2017). https://doi.org/10.1111/jcal.12173 View ArticleGoogle Scholar
  14. P Karamanoli, A Tsinakos, A mobile augmented reality application for primary School’s history. IOSR J. Res. Method. Educ. 6(6), 2320–7388 (2016). https://doi.org/10.9790/7388-0606035665 Google Scholar
  15. H Kaufmann, Collaborative augmented reality in education. Keynote Speech at Imagina Conference (2003), pp. 1–4. https://doi.org/10.1109/LaTiCE.2014.23 Google Scholar
  16. Lytridis, C. (2017a). ARTutor authoring tool. Retrieved from http://aetma.teiemt.gr/aetma/artutor/ Google Scholar
  17. Lytridis, C. (2017b). ARTutor mobile application. Retrieved from https://itunes.apple.com/us/app/artutor/id1323153650?ls=1&mt=8 Google Scholar
  18. C Lytridis, A Tsinakos, mLearn: A mobile learning platform. J. Eng. Sci. Technol. Rev. 10(4), 81–86 (2017). https://doi.org/10.25103/jestr.104.11 View ArticleGoogle Scholar
  19. C Lytridis, A Tsinakos, I Kazanidis, ARTutor-an augmented reality platform for interactive distance learning. Education Sciences 8(1) (2018) https://doi.org/10.3390/educsci8010006
  20. J Martin Gutierrez, MD Meneses Fernandez, Applying augmented reality in engineering education to Improve Academic Performance & Student Motivation. Int. J. Eng. Educ. 30(3, SI), 625–635 (2014)Google Scholar
  21. J Martín-Gutiérrez, P Fabiani, W Benesova, MD Meneses, CE Mora, Augmented reality to promote collaborative and autonomous learning in higher education. Comput. Hum. Behav. 51, 752–761 (2015). https://doi.org/10.1016/j.chb.2014.11.093 View ArticleGoogle Scholar
  22. BE Shelton, NR Hedley, in Augmented Reality Toolkit, The First IEEE International Workshop. Using Augmented Reality for Teaching Earth-Sun Relationships to Undergraduate Geography Students (Darmstadt, Germany, 2002). https://doi.org/10.1109/ART.2002.1106948
  23. S Ternier, R Klemke, M Kalz, P van Ulzen, M Specht, AR learn: Augmented reality meets augmented virtuality. J. Univ. Comput. Sci. 18(15), 2143–2164 (2012). https://doi.org/10.3217/jucs-018-15-2143 Google Scholar
  24. H Tobar-Muñoz, S Baldiris, R Fabregat, Augmented reality game-based learning: Enriching students’ experience during reading comprehension activities. J. Educ. Comput. Res. 55(7), 901–936 (2017). https://doi.org/10.1177/0735633116689789 View ArticleGoogle Scholar
  25. NG Weng, OY Bee, LH Yew, TE Hsia, An augmented reality system for biology science education in Malaysia. Int. J. Innovative Comput. 6(2), 8–13 (2016) Retrieved from http://ijic.fc.utm.my/index.php/ijic/article/view/128/58 Google Scholar
  26. HK Wu, SWY Lee, HY Chang, JC Liang, Current status, opportunities and challenges of augmented reality in education. Comput. Educ. 62, 41–49 (2013). https://doi.org/10.1016/j.compedu.2012.10.024 View ArticleGoogle Scholar

Copyright

© The Author(s). 2018

Advertisement