Educational games and the new forms of interactions

Abstract

Games are seen as environments that promote the engagement of children to the learning of Computational Thinking (CT) concepts. The new forms of interaction using body movements and gestures have increased in particular in video-games. However, few studies have been conducted to investigate how children feel and interact when using hand gestures in the context of learning tools. And this number becomes low when we look at studies about CT learning. In this article, we explored the feedback of 29 children about their interaction by using hand gestures in a game to stimulate the CT, i.e. a CT game. In the analysis, we explored how these forms of interaction impacted on the accomplishing of the game phases. Our findings revealed the children kept more engaged when using hand gestures, and consequently, this affected their concentration on the game purpose. To complement our investigation, we interviewed 8 elementary school teachers to gather their perceptions about the potential of using hand gestures and our game. The findings showed the teachers agreed that hand gestures are suitable to be used in learning tools. They also demonstrated interested in introducing the CT fundamentals in their classes. However, these teachers demonstrated some concerns about how to match the use of that technology to their teaching activities.

Introduction

The increase in the number of learning environments is not new for the area. However, these environments have evolved to become suitable to the demands of the new generation of students(Spector 2014; Boechler et al. 2014). The new generation concerns not only with the content that is delivery by these environments. The students are motivated by the different ways of interacting that these environments provide. The new information and communications technologies have pushed researchers to explore how these new technologies can be introduced into the learning activities(Hwang 2014).

The use of new technologies has boosted the emerged of smart learning environments (SLE) that give support to student activities (Spector 2014). Spector (2014) points out that a SLE should use adaptive technologies or include innovative features which promote the engagement of students. Yet Hwang (2014) (Hwang 2014) affirms that this kind of environment should offer computer mindtools, i.e., computer systems to stimulate them in developing a constructively thinking and learning.

The teaching of computational thinking (CT) has emerged as a way of motivating people in solving problems by applying computational fundamentals. The practices about computational thinking have been widely discussed in different countries to be used in levels of education (Boechler et al. 2014; Narayanan et al. 2018).

The learning environment have provided different ways of interacting with its resources. Among the various types of movements, hand gestures have explored for learning purposes in different contexts as for instance, to interact with storybooks (Kauppinen et al. 2013), and for musical notation learning (Renzi et al. 2015). Other works present experiences using touch and gestures interaction in the context of CT learning (Goyal et al. 2017; Rahman et al. 2013; Papadakis et al. 2016; Gomes et al. 2018; Falcão et al. 2017). However, little has been explored regarding the way that children interact with hand gestures in the context of CT.

Considering the discussion above, this article presents the findings of an investigation about the potential of using hand gestures to interact with a CT game. We called CT game a game that stimulates the development of CT as well as introduces its fundamentals. We explored the use of hand gestures in a learning environment from gathering data of the children interaction. We also explored from interviews the teachers’ perspective about the use of the CT game and the introduction of body and hand movements on learning activities. We considered that our investigation is relevant to the study of SLE due to body and hand movements represent innovative forms of interaction that can improve the students’ engagement on virtual learning environments (Gomes et al. 2018). Besides, these new forms of interaction have become more present on new devices and consequently in the daily of children and teachers (e.g. smartphones, tablets, videogames).

To carry out our study, we conducted an empirical study by taking the comparison method and the descriptive statistics to present our findings. Our investigation had its background on the fundamentals of Human Computer Interaction research (HCI research) presented by Lazar et al. (Lazar et al. 2017). Two versions of the game encompassing the same concepts of CT were developed. In one version, children could interact by using hand gestures, and in another with touch interaction. Our investigation covered two perspectives. First, we collected data of the children interacting in a CT game by using hand gestures and also we explored the children feedback. To do this, we carried out the study with 29 children (aged 7 to 10 years old) and compared both types of interaction in the perspective of children. After, we interviewed 8 teachers who belonged to an elementary school to collect their perceptions about both game versions and about the application of body movements activities in their classes.

To guide our study we defined three research questions (RQ):

RQ1 - What is the feedback of the children about the two game versions?

RQ2 - In which version (i.e. gestures and touch) did the children employ fewer movements to complete the game phases? Did the first form of interaction they used (i.e. gestures and touch) affect their efficiency to accomplish the goal of a phase?

RQ3 - What is the teachers’ opinion about the use of CT games based on gestures and touch in their daily teaching practices?

The main contribution of our work is on exploring the potential of using hand gestures for CT learning. Additionally, we discussed the teachers’ perception of the use of learning environments that provide such type of interaction. Similarly of our work, Rahman et al. (Rahman et al. 2013) present a similar investigation regarding different forms of interaction, but not within the context of CT. On the other hand, Goyal et al. (Goyal et al. 2017) explore the learning of CT, however, not providing the different forms of interaction. Our results revealed that the children presented more concentration on the game goal when playing in hand gestures version. From the teachers’ answers, we could found out that although they considered the body movements and gestures important to learning activities, they pointed out some concerns regarding how to use the technology in their classes.

Background

Fundamentals

Computational Thinking (CT) term arose from the works of Jeannette Wing in 2006. It encompasses the application of Computer Science fundamentals at different levels of abstraction to figuring out solutions to problems in various domains (Wing 2011). In 2011, Computer Science Teachers Association (CSTA) and ISTE (International Society for Technology in Education) proposed a set of concepts and an array of activities for the exploration of these concepts at different school levels (ISTE and CSTA 2011).

Frequently, CT fundamentals are introduced in the form of digital games within the context of schools in particular for teaching children. Digital games are considered a way in which students play and fun while facing learning challenges (Gomes et al. 2018). Frequently, children apply different mechanisms to interact in a game that can range from the use of the traditional keyboard, mouse and touch to the employment of gestures and body movements (Martin-SanJose et al. 2017; Goyal et al. 2017).

The interaction through natural interfaces, also known as NUI (Natural User Interface), is mainly used in interaction with video-games such as Kinect. The potential for applying NUI has motivated researchers to explore this form of interaction in learning context (Martin-SanJose et al. 2017). Body movements, gestures and voice are recognized as NUIs, because they represent movements and actions that humans usually do in the real world (Rahman et al. 2013; Wigdor and Wixon 2011). However, the so-called natural movements are not always understood in this way by children (Kauppinen et al. 2013), which can cause a misunderstanding of how children face the interaction and the learning goal.

Related work

The potential to use hand gestures in learning environments has been explored in some works. Kauppinen et al. (Kauppinen et al. 2013) present a study using the Kinect Stories application from which children from 3 to 6 years old interact with a storybook, using gesture and voice commands. Renzi et al. (Renzi et al. 2015) use NUI and gestures to stimulate and motivate children to learn the basics of music notation. Subramanian (Shiratuddin and Wong 2011) provides a virtual science laboratory, where the teacher and students can visualize and perform experiments using hand gestures and Kinect. Rahman et al. (Rahman et al. 2013) conduct a comparative study of forms of interaction in which children interact through natural gestures and touch. The authors adopt Kinect and iPad to compare the NUI and touch options, respectively, in a pre-existing application. The results showed that children were more engaged in the interaction when using NUI.

In Greece, an initiative involving 43 preschool children highlights that the teaching of CT fundamentals can be seen as a form of computational literacy (Papadakis et al. 2016). All activities are focused on playful scenarios, with interaction through touch on a tablet. The children interact by selecting the movements that a character should do. The results reveal that the children present issues regarding the laterality (turning right and left to make a move). These issues are also found out by Falcão et al. (Falcão et al. 2017) in a study about how children play CT games in 3D environments. Additionally, the same authors point out the children adopt the trial-and-error basis interaction when they do not understand how to interact with the game. Goyal et al. (Goyal et al. 2017) explore gestures to teach CT using Kinect. In the application, the programming language concepts are related to the gestures available to the interaction. The findings show the students feel more relaxed when interacting with gestures and consequently their engagement is improved.

Although the works above addressed the topics of gestures and CT they do not explore both concepts together. Only Goyal et al. (Goyal et al. 2017) and Rahman et al. (Rahman et al. 2013) conduct a similar study. However, Goyal et al. (Goyal et al. 2017) and Falcão et al. (Falcão et al. 2017) examine the interaction issues in CT games, but they do not compare the different forms of interaction. On the other hand, Rahman et al. (Rahman et al. 2013) present an investigation comparing the use of gestures and touch, but not in CT context. Our work provides a study joining the concepts of NUI and CT and showing the potential of using this new form of interaction.

Computational thinking game

To develop our game, we took into account some guidelines and lessons learned provided in the literature focusing on HCI research. From the use of NUI perspective, we considered the guidelines presented in (Shiratuddin and Wong 2011; Maike et al. 2015). The first one (Shiratuddin and Wong 2011) considers the issues regarding the use of NUI in the design of games. The second (Maike et al. 2015) focuses on the good practices to use NUI from HCI perspective. We also took into account the lessons learned presented in (Papadakis et al. 2016; Falcão et al. 2017) about the laterality issues. We also performed an exploratory study with the children participation from which we investigated the use of the hand gestures. Additionally, we considered the recommendation that HCI research should conduct exploratory studies involving end-users and technology (Lazar et al. 2017). CSTA recommendations covered the CT fundamentals. The game was designed by one undergraduate student in Computer Science and two senior researchers in HCI and e-learning area. All the participants had expertise in the application of HCI fundamentals and in using NUI technologies. The senior researchers followed and validated all the steps of the game design. The follow subsections reported the planning and development of both CT game versions.

Planning the game

We decided to build a 2D application taking into account that some studies reveal that children present problems on concentration when interacting on 3D environments (Falcão et al. 2017). In our game, the player controls the pirate during its missions. It has to deliver objects to the dreaded Blackbeard. The pirate can walk only to the right without the option of returning to the left. By using only this movement, we can avoid problems in the understanding of how to do the action of turning to the left and right (Papadakis et al. 2016; Falcão et al. 2017). We also decided to verify the children understanding about the icons (i.e. for tablet version) and hand gestures in the context of our game. For having an interactive conversation with the children, we produced tangible materials which represented some elements presented into the game. The artifacts we developed to support the interactive conversation were: (i) a description of the story of the game; (ii) a set of images representing the actions the character could perform; (iii) a set of images representing the scenario of the game, its obstacles and objects that the character should overcome and "catch", respectively; (iv) a group of questions to capture the children opinion about the game flow and about the images and the gestures used; and (iv) a profile questionnaire.

We conducted a study with 10 children (aged 9 to 11 years old) in a school located in Sorocaba, São Paulo. Each student took part individually and spent 6 minutes on average in the activity. First, they answered the profile questionnaire. After, we offered to the children the scenario of the game and a set of images contained the character, objects and icons representing the character’s movements (i.e. icons of the tablet version). First, the researcher accounted the pirate story and the students built the game scenes. The children indicated their preferences for images that represented the character movements and actions. In a second step, we gathered what hand gestures the participants would use to interact with. One researcher held the game scenario putting it in front of the child at the same height that the camera was to capture the gestures. Another researcher, then, accounted the same story again and asked the participant what hand gestures s/he would use for each character action. A third researcher made notes about the movements selected by the children.

Considering the images and hand gestures the children selected, first, we conducted testings to explore the accuracy of gestures recognition from the camera. Among the several toolkits of NUI technologies (e.g. Kinect and Leap Motion), we used the Intel solution. Intel loaned two RealSense¹ cameras for our research group. Unity and Visual IDE Studio supported the development of the game². For each movement that the character could take, two authors of this article performed all the gestures available on the camera, and then executed the gestures chosen by the children. The results showed limitations on the identification of some gestures, in particular for those that the movements were very similar one each other. To prevent the interaction issues, we performed other testings again to select thus the movements which did not present recognition failures. As a result, we got a mapping of the gestures and icons (for touch version) which represent the movements of the pirate (see Table 1).

Table 1 Mapping of the hand gestures and icons to the movements - (a) Pirate movements/actions, (b) Hand gestures, (c) Hand gestures descriptions, (d) Icons representing the movements/actions

Developing the game

After exploring the technological aspects, we proceeded to develop the game. We built first the hand gesture version by using the RealSense toolkit, and then we exported it to Android platform (mobile touch version). The game provided 8 phases in total to motivate different profiles of players. In all the phases, players could follow their evolution from visual and textual feedback. By selecting a sequence of movements (i.e. by making the hand gestures or by touching the icons) players conducted the pirate to achieve a phase goal.

We elaborated a tutorial to be displayed at the begin of the game following a playful approach (Gomes et al. 2018). An animation of Blackbeard character "talked" to the children and explained the game goal, and what symbols and gestures were available and their respective meaning.

Taking into account that our game focused on motivating the learning of CT fundamentals, we took some background based on educational guidelines. CT is defined as a problem-solving process. It has characteristics that can be implemented in elementary education (ISTE and CSTA 2011). Brazilian Computer Society summarises CSTA proposal in three characteristics (Raabe et al. 2017): (i) abstraction - the ability to understand and outline solutions from using suitable models and representations; (ii) automation - the ability to describe a solution in a precise sequence of instructions (algorithm); and (iii) analysis - the ability to critically analyse problems (i.e. verifying the existence of different solutions) as well as analysing solutions in the aspects of accuracy and efficiency.

In our game, we considered the three characteristics above. Gradually, the game introduced the concepts of CT to the children. For instance, in the first phase, according to the children understood how to play the game, they started to develop the concept of abstraction. This phase requires the pirate to walk forward 8 times and pick up the object in its last step. Even in a simple phrase, the child is developing the ability of abstraction. The game requires the child to understand the problem and build an abstract model for the solution. At the same time, the child is also developing the ability to automate a solution by applying a precise and straightforward language (arrow buttons or hand movements for the tablet and camera, respectively). Similarly, Phase 2 requires to the children the application of abstraction and automation skills. In Phase 3, the children find a new challenge. The player must decide which path to choose to end the phase. In this case, the children is developing the ability of analysis. They have to identify the different paths and analysed, which would be the best way to achieve the solution.

Figure 1 shows the camera (i.e. hand gestures) and tablet (i.e. touch) versions. Both have similarities in the scenarios of the phases to avoid misunderstandings on how to interact with. For the tablet version, there is a set of images (see (c) into Fig. 1) that represents the movements available for touch interaction. In the camera version, as the player performs commands by hand gestures, no icons are shown on the screen.

Fig. 1

Blackbeard’s Mission. i Camera version. iiTablet version

Although both versions make the play button available (see (a) into Fig. 1) in the camera version it is executed only by gestures (see in Table 1). The sequence of movements selected by the player is displayed at the bottom (see (b) into Fig. 1). Different from video-games in which the interaction by gestures is continuous, in our game, the players must select a sequence of commands, and only after that, they activate the play button. Yet in Fig. 1b, the icon in highlighting shows which the command is in execution. From this view, the players can follow the pirate action and associate it to the movement they have chosen.

Before conducting our study with the children, we carried out a pilot testing with 4 children (aged 9 to 11 years old) was performed. The pilot testing also allowed us to verify whether children would understand the gestures we have adapted. The children played the hand gestures and touch versions on a computer with a RealSense camera installed and on a Samsung 10.1" tablet, respectively. The results of the pilot testing revealed that children enjoyed the versions. They did not have doubts about the use of images (tablet version) and gestures (camera version). Only one of them had problems with the recognition of gestures in consequence of issues presented by the camera accuracy.

Investigating the interactions of children

We conducted a study with 29 children (aged 7 to 10 years old) who belonged to two classes of an elementary school in Salto de Pirapora, São Paulo, Brazil.

The school contacted the parents and guardians of the children to assign the ethical agreement regarding the capture of image and audio. For the study conduction, the school, provided two rooms that were located on opposite sides of the same building. A computer and a notebook, both with the RealSense cameras installed, were placed in one of the rooms. In this room, two HCI researchers (one undergraduate and one master student) carried out the observation of participants interaction. In the second room, two Android tablets were available to be used by the children; and one HCI researcher (one undergraduate student) watched out the interactions of the participants. Two children took part in each room at a time. The researchers followed the scripts of how to apply the study for mobile and camera versions. The researchers elaborated the script which was revised by the senior HCI researchers.

To avoid the bias that could be caused by the order in which the game versions were used (camera and tablet), we split the children into two groups (Lazar et al. 2017). One group started their participation by using the camera (C) and afterwards, the same group interacted with the tablet (T). For the other group, we reversed the order. Moreover, the same child did not have contact with the two versions on the same day to avoid s/he memorized the sequence of movements to accomplish the game phases.

To start, we provided a brief explanation of our study, and about the CT game. After, in the case of camera interaction, one researcher demonstrated, only once, the hand gestures and the respective actions of the character. The participants were informed they are free to ask for help. For tablet version, we recorded the screens of the game while participants interacted using LookBack³. The interaction with gestures was filmed using two smartphones. Besides, the researchers made notes about children’s behaviours and doubts. Figure 2 illustrates two children interacting with camera and tablet versions, respectively.

Fig. 2

Children interacting with the game versions. i Camera version. iiTablet version

The participants answered the profile questionnaire (questions about the children relation with Internet and use of computers and games) only once at the end of their first participation. The questionnaire answers showed us that all the children played games on the Internet and interacted with smartphones and tablets. As soon as the children finished their participation in one version, we collected their degree of pleasure individually according to the SAM (Bradley and Lang 1994). Self Assessment Manikin (SAM) is a pictograph evaluation method to measure emotional responses from some stimulus. It is recognized as a suitable instrument to gather children feedback due to it facilitates the mapping of emotion to the image that represents it (Hayashi et al. 2016). This technique considers three dimensions. The user chooses a value on a scale of 1 to 9 on each dimension by using images to represent their emotions after interactions. In our study, we used only the pleasure dimension, collecting whether the participant had a positive or negative reaction.

For the analysis, we considered only the data gathered until phase 3 of the game, which was enough to examine the CT characteristics. We recorded 58 videos (29 children x two forms of interaction) which were watched carefully. By watching the videos, we examined the hand gestures employed to interact with the camera, and images select on touch interaction (i.e. on the tablet). In both cases, we observed the movements related to accomplishing the game phases. We explored each video twice to get a more accurate analysis. Table 2 presents the results of the analysis. Blank cells represent the phases the participant was not able to complete. Some samples of tablet interaction were discarded, because videos recording failures, and when participant missed school on the day of tablet interaction.

Table 2 Number of movements (hand gestures or touch) by Participant (P) per game phase (GP) in each Device (D), (M) - minimum number of movements to achieve the phase goal

Findings and discussion on the investigation with the children

We will present the findings of the investigation conducted with the children by comparing the results of both game versions. Additionally, we will discuss the results associated them to previous works. Our results will be presented by answering the RQ1 and RQ2. The descriptive statistics was adopted in our analysis due to our sample was small and unbalanced.

Children feedback

To answer the RQ1 - What is the children feedback about the different game versions?, we considered the perspective of the children (i.e. the responses of SAM) and the perspective of the researchers (i.e. the analysis of the videos). While SAM provided the children perspective the analysis of the videos allowed us to see whether the gestures introduced some barrier to the interaction. In Fig. 3, we notice that the feedback was similar for both versions. Usually, interactions in mobile devices are well-known by children, and in our study, the participants’ profile restated this. Therefore, from the videos, we noticed that the children did not struggle on interacting with the tablet version. Although interaction by gestures was not familiar to the participants, this did not affect the positive feedback of the children for the interaction with hand gestures. Our results reinforce those found out by (Wigdor and Wixon 2011) which state that NUI has a great potential to motivate children to interact.

Fig. 3

Children feedback - SAM pleasure dimension

To complement SAM results, we examined from the videos whether the children had any issue in understanding how to use the hand gestures. We checked in the videos whether the participants were able to use and remember each gesture or not. Hence, we looked for evidence of barrier by observing whether (i) a child only asked questions to confirm that s/he was making a correct gesture (i.e. in fact s/he was remembering and understood the gesture) or whether (ii) s/he had doubts of what gestures s/he should make. Table 3 shows the number of doubts that the participants had on the use of the hand gestures to do the respective movements.

Table 3 Number of doubts for each type of movement

By considering the recommendations and guidelines presented in other works (Papadakis et al. 2016; Falcão et al. 2017; Shiratuddin and Wong 2011; Maike et al. 2015), we created a game with good playability. Hence, we got positive feedback regarding the adoption of hand gestures interaction. Positive emotions and, consequently, the acceptance of technology have a positive influence on learning (Bradley and Lang 1994). Feeling pleasure in doing activities encourage students to interact and engage in learning environments (Goyal et al. 2017).

Movements and first form of interaction

By exploring video data, we answered the RQ2. We examined whether the use of hand gestures inserted some barrier of interaction to the children. Consequently, we checked whether these barriers provoked an increase in the number of movements the children took to achieve a goal phase. The literature points out the importance of mitigating the barriers of interaction in the learning environments to avoid the negative impact that these barriers can cause on the children learning (Gomes et al. 2018).

First, we answered the sub-question of the RQ2 - In which version (i.e. gestures and touch) did the children employ fewer movements to complete the game phases? - without considering the form in which the participant has interacted first. We compared the number of movements executed by the children (P) to the minimum of movements (M) required to achieve the goals in phase 1 and 2 phase (GP) (see values in Table 2). Figure 4a shows the children made fewer movements when interacting with the camera. From the analysis of the videos, we could see that on touch interaction (i.e. tablet version), the participants adopted a trial-and-error basis interaction which explains the outliers in Fig. 4a. This behavior has already mentioned in (Falcão et al. 2017). The authors associated trial-and-error basis interaction to children difficulties in understanding the meaning of interface elements. However, by examining the videos, we could notice that in our case, this behaviour was linked to the intention of reaching the next phase of the game quickly. The outliers of the camera version referred to failures in recognising some hand gestures that happened with some participants. As a result, some children performed a more significant number of movements in an attempt to recognise their gestures.

Fig. 4

Number of movements used to achieve the goal of the phases. a Number of Movements. b Additional movements - Phase 1. c Additional movements - Phase 2. d Additional movements - Phase 3

Looking at the data of the movements in another perspective, we explored the number of additional movements the children employed regardless of which version they have interacted first. The graphs in Fig. 4b, c and d summarize the results in the three phases of the game. We analyzed the first and second attempts to achieving the goal in the same phase. We see that the number of additional movements decreases from phase 1 to phase 2, except for a few outliers. This result reveals the participants evolved their skills about the game through the phases. In phase 3, in which a more accurate logical reasoning was required, the participants showed greater difficulty to reach the final solution in the first attempt. However, phase 3 was the one in which more children used a fewer number of additional movements in the second attempt. And several of the solutions employed only the minimum number of steps.

To answer the other sub-question - Did the first form of interaction they used (i.e. gestures and touch) affect their efficiency to accomplish the goal of a phase?, we analyzed the number of movements they did in phase 3. We considered only 22 participants that reached the goal of this phase. Additionally, we considered the form they have interacted first. Figure 5a shows that the most of the participants were able to achieve the goal of phase 3 on the tablet (i.e. their second form of interaction) when the camera was their first form of interaction.

Fig. 5

Number of movements in Phase 3. a Camera - First form of interaction. b Tablet - First form of interaction

By comparing the movements employed with the minimum number required, we noticed that the interaction with the camera was not very efficient when it was the first form of interaction (see Fig. 5a). However, when the participants interacted first with camera and after with the tablet, they were able to achieve the goal of phase 3 by employing a fewer number of movements. On the other hand, most of the participants who had tablet as the first form of interaction (see Fig. 4 b) did not reach phase 3. Nonetheless, the participants who had the camera as their second form of interaction proposed more efficient solutions than those who first used the camera.

These results reveal that gestures interaction can promote greater engagement and a higher concentration of the children. These results restate the findings presented in(Goyal et al. 2017) and (Rahman et al. 2013). Additionally, these results Besides, the evidence showed the first interaction through the tablet was hampered by the excessive number of touches that the participants performed to reach the end of the phase rapidly (Falcão et al. 2017).

Exploring teachers’ perspective

In the second round of our study, we conducted structured interviews with 8 teachers. The participants belonged to the same elementary school where the investigation with children was carried out. By gathering the teacher’ viewpoint, we aimed to understand whether the teachers considered that the hand gestures interaction and our game could be relevant to the children learning. Additionally, we intended to see whether the teachers often applied activities working with body movements and gestures, and used electronic games in their classes. We did not have any influence on the selection of the participants of the interview. The coordinator of the elementary school was responsible for inviting the teachers to take part in the interviews.

The interviews followed a script which was elaborated by a master student and reviewed by a senior researcher. The first author conducted the interviews individually with each participant. All the interviews were recorded in audios. First, we explained to the teachers the purpose of the study and the aim of the interview. After, we asked the participant to confirm the agreement in participating by signing the term, which gave us permission for using the data collected to academic purposes. Subsequently, we asked a set of 8 questions in the same order, as shown in Table 3. The first 4 questions collected the teachers’ profile and the others the teachers perspective about the use of gestures and our game. Before asking question 7, we presented both versions of the CT game, pointing out the difference between them.

After gathering the data, we transcribed the audio recordings to text format to conducted a qualitative analysis. We took the open coding technique to explore the participants’ responses. Open coding relates codes to chunks of text. These codes receive denominations that give meaning to the chunks of texts they refer to (Strauss and Corbin 1998). Subsequently, these codifications were revisited and grouped when patterns in the information were identified.

Our qualitative analysis was supported by the tool Atlas.ti⁴. First, the second author explored the participants’ responses and assigned codes to them. After, the fourth author revisited the data to analyze whether other codes could be found out. In this step, 31 codes were generated. Later, these two authors refined the 31 codes by grouping them in themes. The first three questions (see Table 4) were joined in the same theme regarding the participants’ profile. On the other hand, the other five questions had 4 different themes assigned to them.

Table 4 Questions applied in the interview

Findings and discussion on the teachers’ perspective

The results found out in the interviews supported us on answering our RQ3 - What is the teachers’ opinion about the use of CT games based on gestures and touch in their daily teaching practices?. Considering the profile theme, we saw the educational background of the participants matched in three different groups: teachers of elementary school, Arts or Physical Education. The participants had 7 years of experience on average, and they reported to have more facility in teaching the mother Language and Mathematics. All the respondents said they accessed the Internet frequently, mostly for searching for exercises or supplementary materials to support their classes. However, part of them reported that the school did not have Internet access available to the teachers.

As a result of the teacher perspectives about the use of gestures and the CT game in learning activities, we got 4 different themes. For each theme, we assigned a sub-theme that grouped summarized the important statements reported by the participants. We also classified these sub-themes as positive or negative, which reflect their impact on the children learning. To complete, we associated with the theme and the sub-theme the participants’ speeches related to it. Table 5 shows our findings.

Table 5 Themes, sub-themes and teachers’ speeches

From the theme the importance of body movement activities, we notice that all the teachers recognise the relevance of performing activities that involve gestures and body movements. Most of the participants pointed out that these activities can affect not only the development of motor skills but also the ones related to the communication of the children in their social environment.

Part of the codes assigned to the theme performing activities with gestures and body movements demonstrated a negative impact on the teachers’ perspective. The teachers pointed out that they were interested in performing CT games and activities of body movements and gestures. However, the excessive number of students in a class brings difficulties to coordinate such activities. Additionally, the teachers reported that such activities request more time of planning to be effective in their purposes. And they usually cannot manage their time to do this. Otherwise, from a positive perspective, all the teachers stated they promoted practices that encourage children to make body movement and gestures, such as singing children music, performing dances and stage plays, and by practicing sports games. One of the participants said s/he often performed activities of body movement through electronic games by using a Kinect.

All the participants demonstrated the agreement that games as ours allow can support the activities that stimulate the development of students’ motor skills. In Table 5, we see this result in the theme impact of our game on the students’ learning. However, only three of them reported that our CT game (in both versions) could have an impact on the learning of CT fundamentals. These three teachers were attending continuous training about the introduction of CT in elementary school class exactly in the period when we conducted the interviews. These participants were able to describe that our game had the purpose of working with algorithm principles. And going beyond, they pointed out that the same principles could be applied in classes by taking unplugged activities. These results revealed that the CT background aided the teachers in making the relationship between our game and other activities which involve CT fundamentals. This finding restated the importance of providing continuous training to elementary school teachers on how to add the CT in teaching practices.

Taking into account the same theme, we found out positive and negative points of view regarding the use of gestures. In a positive perspective, the teachers, in general, reported they considered that (+) interaction by gestures requires the students to have more attention. The participants mentioned that touch interaction could encourage the children to take a trial-and-error basis interaction, i.e. (-) touch interaction can become automatically. In the participants’ viewpoint, the trial-and-error basis interaction does not encourage students to reflect on the meaning of the movements and their relation with the game. According to the teachers, from the hand gestures version of the game, the children should structure their thinking more accurately. These findings restated what we have found out in “Findings and discussion on the investigation with the children” section on the investigation with the children about the potential to use gestures interaction for learning purposes. Yet, we could notice that the teachers who attend continuous training about CT were the ones that feel more comfortable in making the relation between the form of interaction with the learning issues.

Finally, the last theme addresses the teachers’ answers regarding the use of our games to support their classes. Only one participant answered negatively. The others presented their doubts about how they could apply and match the games with their activities. The concerns were focused on how to use the gesture version, in particular, the concerns were about the use of the technology (i.e. RealSense camera). They also arose issues on the setting of the game to be used with other technological tools as such multimedia projector or a digital whiteboard. Overall, all the teachers showed to be excited about the possibility of applying such games in their daily teaching practices in classes.

Our results showed that the NUI was recognised as an important tool by the teachers. However, they presented some concerns regarding the adoption of the tool. Although the use of Kinect in classes was explored in other works (Shiratuddin and Wong 2011; Kauppinen et al. 2013; Renzi et al. 2015), our results showed that the number of students to deal with could impact negatively in the acceptance of technology by the teachers. The teachers’ statements reinforce the results found out in the literature that the NUI acts positively on the attention and the engagement of the children for the use of learning tools (Goyal et al. 2017; Rahman et al. 2013; Falcão et al. 2017; Martin-SanJose et al. 2017).

Limitations of the study

Our study revealed relevant results regarding the use of NUI in learning environments. However, there are some limitations that have to be pointed out. The first one was the number of children that took part in the first study. As the children played the two versions in different days, many of them were not presented in one of the sessions, which caused an unbalanced in our sample. The RealSense camera represented a barrier to our investigation due to the recognition of some movements failed.

Conclusions

Our findings revealed that both participants, children and teachers, were very welcome in interacting with hand gestures to play a CT game. Considering the investigation with the children, we noticed a natural evolution of the children in the game. The learning of CT fundamentals requires the children to have an abstraction of solutions as well as to be able to automate and analyzing these solutions. For this, the concentration and attention on the problem to be solved are essential. Our findings showed the hand gestures could motivate the participants’ engagement, and consequently, this can affect their concentration. Even though the participants were not able to do fewer movements when interacting with the camera version first, they were able to achieve the goal. The same did not happen for those who had first interaction on the tablet version.

By looking at the results from the teachers’ perspective, we conclude that both the hand gestures and our game could be adopted in classes since the teachers receive training to handle it. We noticed the teachers gave great importance to activities involving body movement and gestures on developing different skills and on supporting the students’ concentration.

Generally speaking, we could conclude that NUI can be seen as a potential form of interaction to enrich the communication between students and smart learning environments. NUI also can work as an engine to promote the engagement of students in these environments.