Examining the relationships between students’ perceptions of technology, pedagogy, and cognition: the case of immersive virtual reality mini games to foster computational thinking in higher education
Smart Learning Environments volume 10, Article number: 16 (2023)
Researchers are increasingly exploring educational games in immersive virtual reality (IVR) environments to facilitate students’ learning experiences. Mainly, the effect of IVR on learning outcomes has been the focus. However, far too little attention has been paid to the influence of game elements and IVR features on learners’ perceived cognition. This study examined the relationship between game elements (challenge, goal clarity, and feedback) as pedagogical approach, features of IVR technology (immersion and interaction), and learners’ perceived cognition (reflective thinking and comprehension). An experiment was conducted with 49 undergraduate students who played an IVR game-based application (iThinkSmart) containing mini games developed to facilitate learners’ computational thinking competency. The study employed partial least squares structural equation modelling to investigate the effect of educational game elements and learning contents on learner’s cognition. Findings show that goal clarity is the main predictor of learners’ reflective thinking and comprehension in an educational game-based IVR application. It was also confirmed that immersion and interaction experience impact learner’s comprehension. Notably, adequate learning content in terms of the organisation and relevance of the content contained in an IVR game-based application significantly moderate learners’ reflective thinking and comprehension. The findings of this study have implications for educators and developers of IVR game-based intervention to facilitate learning in the higher education context. In particular, the implication of this study touches on the aspect of learners’ cognitive factors that aim to produce 21st-century problem-solving skills through critical thinking.
Due to how affordable and portable technology has become, immersive virtual reality (IVR) technology is mediating teaching and learning in several educational contexts (Bower and Jong, 2020). IVR technology does not only create an opportunity to supplement teaching and learning but also allows students to interact with learning objects. Besides, the features of IVR technology—immersion, interaction, presence, and immediacy—contribute to its affordances in education (Makransky et al., 2019). Indeed, IVR technology is used for training and simulation of educational concepts (Lui et al., 2020), which makes it possible to represent almost everything as a learning object in a virtual environment for learners to gain experiential knowledge (Alrehaili & Al Osman, 2019).
Computational thinking (CT) is a growing field that demonstrates a pedagogical approach to fostering critical thinking and problem-solving skills among contemporary learners (Wing, 2009; Carretero et al., 2017). CT involves thought processes to solve a given problem. CT includes concepts such as algorithmic thinking, problem decomposition, problem abstraction, pattern recognition, and recursive thinking (Agbo et al., 2021b). The introduction of these concepts into schools’ curriculum allows young learners to have digital literacy and develop thinking skills that can foster their ability to computationally solve real-world problems (Aho, 2012). Developing CT competency is essential for K-12 (Wing, 2009) and higher education students (Cachero et al., 2020). Hence, the integration of CT in the higher education context is currently gaining ground nowadays (de Jong & Jeuring, 2020). Studies have shown how university students who do not have a computer programming background found programming courses difficult to understand (Agbo et al., 2019; Gamage, 2021; Liu & Zhong, 2018). This difficulty could be due to their lack of comprehension of the concepts of programming. One way to reduce the difficulty in understanding computer programming is to develop their CT knowledge. This knowledge of CT will support students’ creative thinking skills that can be applied to problem-solving (Sukirman et al., 2021).
To develop a state-of-the-art technology-based intervention to facilitate 21st-century skills including CT, IVR and game-based learning (GBL) is a viable approach. Studies have shown that educational IVR mini games can impact learners’ cognition and learning outcomes in creative thinking, reflective thinking, and self-efficacy (Lee et al., 2010; Makransky & Petersen, 2019; Kim et al., 2020; Agbo et al., 2022). Despite these positive outcomes from previous studies, there are concerns that may affect the use and adoption of IVR in education. For example, it has been reported that IVR technology could lead to a higher cognitive load, more distractions to learners, and poorer performance (Makransky et al., 2019). To address these concerns, there is a need to examine the relationship between IVR features and educational game elements on learners’ cognition. In other words, to reinforce the affordances of VR and GBL for a 21st-century learning experience, there must be a concerted effort by scholars to examine critical elements of both the technology and pedagogy using their characteristic elements. In addition, investigating the affordances of IVR and GBL vis-à-vis learner’s cognitive benefit is a continuing concern among educators in the higher education context. Therefore, game elements which include challenges, goal clarity, feedback, and adequacy of learning content, and IVR features such as immersion and interaction form the study’s variables being examined. Although studies have investigated the impact of IVR features on learning outcomes (Lin et al., 2020; Barrett et al., 2021), the effect of IVR and game elements on learners’ cognition is inadequately researched (Imlig-Iten & Petko, 2018). Hence, this exploratory study builds on the path model of Hamari et al. (2016) and Lin et al. (2020) to examine how features of IVR and game elements of an educational tool influence learners’ reflective thinking and cognition.
Technology-mediated learning theory
According to literature, technology-mediated learning (TML) refers to an environment in which learners interact with instructional materials, peers, and or instructors where information technology plays an intermediary role in connecting the different stakeholders within a learning situation (Alavi, 1994; Ryoo & Lee, 2016). Essentially, TML theoretical assumption stresses that in educational technology research, technologies themselves have no intentions but are rather featured to convey meanings between stakeholders in a learning environment (Lin et al., 2020). In evaluating the effect of a learning environment within the theoretical framework of TML, Bower (2019) has provided several scenarios to guide researchers. For example,
Does technology-mediated learning theory apply in computer-assisted learning situations where a program of instruction or software package has been written for use by a student in isolation at a stand-alone machine? Even though such learning may not occur in a social context, the digital content has been composed by educators and is mediated using technology, so technology-mediated learning theory may be useful to examine and explain effects. (Bower, 2019, p. 1043)
Nowadays, many studies are carried out to examine how teaching and learning are being mediated by IVR applications to facilitate students’ learning experience in terms of cognition (Cheng & Tsai, 2020), creative thinking, and reflective thinking (Makransky & Petersen, 2019). For example, Lin et al. (2020) recently investigated the effect of IVR application on learning outcomes using a two-path model. According to Lin and colleagues, when designing a TML intervention, technological features of VR, such as immersion, should be well designed since it strongly predicts learners’ motivation and learning effectiveness. Additionally, Cheng and Tsai (2020) investigated the effect of student learning traits mediated with immersive features of IVR on their learning attitude. Findings from Cheng and colleague suggests that students’ intrinsic value and self-regulation for learning could positively impact their sense of immersion.
Immersive virtual reality and mini games in education
IVR has varied applications in education and training. Prior research on the effect of IVR on learner’s performance exists (Agbo et al., 2021a). For instance, Makransky et al. (2019) investigated the effectiveness of an IVR application in promoting science education compared to video-based instructions. In their study, Makransky and colleagues revealed that IVR provides more interactivity, which in turn is beneficial to learning. Petersen et al. (2020) conducted a study that demonstrated how learners could travel virtually to relevant sites to learn about climate while in physical classrooms. According to Petersen and his colleagues, IVR application increases students’ interest, knowledge, and self-efficacy, while pretraining affects cognitive load, affecting learners’ performance.
Similarly, Lui et al. (2020) recently explored how to teach complex scientific concepts such as gene regulation through simulation within an IVR environment. Their experiment revealed that students who learned the concepts using the IVR approach gained better knowledge compared to those who did not learn using the IVR intervention but the traditional approach. While previous research underscores the relevance of IVR for teaching and learning, this current study investigates how students think IVR game-based application can facilitate their computational thinking by exploring the relationships between IVR features and game elements on learners’ perceived cognition.
Furthermore, educational mini games developed as an IVR application is a good approach to foster higher-order thinking skills where learners are not only motivated to learn but also engaged through game challenges that start easily and then progressively become difficult (Chaves et al., 2021; Hamari et al., 2016). Educational mini games are short types of serious games that are developed to be playable independently to gain micro knowledge (Agbo et al., 2021b). Unlike serious games, mini games are flexible, simple, and easy to learn. The benefits inherent in mini games create the opportunity to achieve a small unit of learning objectives; hence, this study conducted an experiment with IVR mini games using low-cost head-mounted displays (HMD) and hand controllers. Studies have shown how the use of IVR applications with low-cost HMD such as Google Cardboard and a Bluetooth hand controller are increasingly deployed for instructional practices and science education (Parong & Mayer, 2018; Cheng & Tsai, 2020; Agbo et al., 2022).
Computational thinking in higher education
Computational thinking was made popular by Wing in her presentation at the Communications of the ACM (Association for Computing Machinery) anniversary celebration (Wing, 2006). Since then, teaching and learning of computational thinking have been integrated into the school curriculum, mostly in K-12 settings (Grover & Pea, 2013). Several debatable topics have emerged over the years regarding, for example, how computational thinking should be taught at school (Mannila et al., 2014), who should be taught computational thinking (Lockwood & Mooney, 2018), what common grounds should be developed for computational thinking and computing education, and whether computational thinking should be integrated into higher education institutions’ curriculum (Hu, 2011; Czerkawski & Lyman, 2015; Apiola & Sutinen, 2021).
A systematic review study revealed that teaching computational thinking in higher education institutions began in 2010, where educators designed courses to infuse computational thinking concepts into their classes (Agbo et al., 2019). Furthermore, de Jong and Jeuring (2020) presented a scoping literature review of tools and interventions used to teach computational thinking in higher education and how effective these interventions are. Their findings revealed that varied disciplines, including information systems, journalism, sociology, tourism, and even engineering, have discussed or implemented some computational thinking concepts. However, programming education classes were the most likely subject area to integrate computational thinking concepts (de Jong & Jeuring, 2020). By “computational thinking concepts," we mean topics focusing on problem-solving skills, algorithmic thinking, problem decomposition, pattern recognition, problem abstraction, and recursive thinking.
Integration of computational thinking in higher education to facilitate programming education is on the rise (de Jong & Jeuring, 2020). Educators are exploring more contemporary approaches to using technology in learning and teaching. That is why we need to investigate how to do it well. Besides, the affordability of advanced technology and smart systems such as cloud computing, the internet of things, smartphones, and wearables, has caused education to be more flexible, autonomous, and ubiquitous, such that learners can gain more benefits of enhanced learning experience that is engaging and motivating (Lin et al., 2020). Hence, leveraging a GBL approach and virtual reality (VR) as state-of-the-art technology can provide an opportunity for an enhanced learning experience (Alrehaili & Al Osman, 2019) for learners in the higher education context.
Development of the research hypothesis
The relationships between the dependent and independent variables for this study were motivated by the path models of Hamari et al. (2016) and Lin et al. (2020) to develop the game element constructs, IVR technology constructs, and cognition constructs. As shown in the conceptual framework of the relationships between IVR, GBL, and perceived learners’ cognition (Fig. 1), two components, including the technology (immersion and interactivity) and pedagogy (game challenge, goal clarity, and feedback), form the independent variables. The cognition component constitutes the dependent variables (reflective thinking and comprehension), whereas learning content is the moderating variable.
Within the technology component, immersion, and interaction (Makransky & Petersen, 2019) are the constructs investigated. Whereas, in the pedagogy component, game elements such as challenge, goal clarity, and feedback (Fokides et al., 2019; Fu et al., 2009) were the underlying constructs. Figure 2 depicts the structural model for this study, which seeks to examine the relationships between the constructs and how they influence the learner’s perceived cognition.
Effect of educational game elements on learner’s perceived cognition
According to Plass et al. (2015), game elements are critical factors that determine the learning outcome of an educational game. They posit that for any game research, cognitive engagement is one of the outcomes of educational GBL, influenced by the three key elements of almost all games (i.e., challenge, response, and feedback). Feedback is a major element of a game that allows the player to be aware of the current level of achievement, determine the gap between the current stage and the expected goal, and provide relevant tips required to complete the goal (Fu et al., 2009). Players of a game can receive feedback in form of rewards, warnings, hints, or instructions. The appropriateness and timeliness of feedback provided to players of an educational game could influence the perceived level of challenge. According to Beghetto (2018), Challenge "literarily means an invitation or call to action" (p. 13). In game design, elements such as challenges can foster players’ curiosity leading to intrinsic motivation (Plass et al., 2015).
Nevertheless, an adequate level of challenge with goal clarity is required to maintain a state of flow for an effortless learning experience. Goal clarity defines the intermediate and primary objectives of an educational game (Wang et al., 2014). According to the literature, educational game challenge and goal clarity significantly impact learners' cognitive processing (Mayer, 2014), human behaviours, and learning outcomes (Fokides et al. (2019); Wang et al., 2014. Therefore, the kind of challenge perceived by a player can determine what feedback could be appropriate to overcome such challenge; hence, this study hypothesized that:
Game challenge will positively influence the feedback received when playing VR mini games.
Goal clarity will positively influence the feedback received from playing VR mini games.
Impact of learning contents of IVR mini games on learners’ perceived cognition
The central focus of any technology-mediated learning intervention is usually to enhance the learning experience with the aim of improving knowledge. Therefore, cognition in this context indicates the knowledge and capability gained after undergoing a learning process using information technology to mediate between the learner and the learning contents. The knowledge gained can be applied to solving difficult tasks. In this study, learners' perceived cognition refers to participants' self-ratings of cognitive benefits rather than measures of learning or performance. Learners' perceived cognition (reflective thinking and comprehension) has been adapted from Lin et al. (2020) model of the TML framework in the VR context. According to the literature, comprehension is a fundamental element of the cognitive learning process whose impact on learners in an IVR learning environment is explored (Zhao et al., 2020). Similarly, this study conceptualizes reflective thinking as the state of mind where learners engage in self-inquiry to understand a certain phenomenon and clarify possible doubts (Makransky & Petersen, 2019). According to Chen et al. (2019), reflective thinking is a complex skill that would require critical thinking and problem-solving skills. Based on this discussion, this study hypothesized that:
Goal clarity will have a positive effect on learners’ reflective thinking when playing VR mini games.
Game challenge will positively influence learners’ reflective thinking when playing VR mini games.
Important features of IVR on learner’s perceived cognition
As stated earlier, research has shown that IVR features which include immersion, interaction, the immediacy of control, and representational fidelity could influence learning outcomes (Lee et al., 2010; Lin et al., 2020). Immersion is a characteristic of VR technology that creates the opportunity for a user to perceive of being in a real environment yet in a virtual environment. Although the degree of immersion experienced by users of VR is subject to variation, the technological capabilities of VR devices, the sensors, screen resolutions, and even the virtual contents have a great impact on the way the technology influences user's immersion (Mütterlein, 2018). Similarly, interaction within a virtual environment is possible through the integration of different devices such as HMDs, motion sensors, and hand controllers of varied interaction capabilities. The more a technology-mediated learning tool allows for interaction between the learners, learning contents, and other objects within the virtual environment, the more positively impactful the learning experience becomes. Based on the discussion, this study proposed the following hypotheses.
Immersion will have a positive effect on learners’ comprehension when playing VR mini games.
The interaction will have a positive effect on learners' comprehension when playing VR mini games.
Reflective thinking predicts learners’ comprehension when playing VR mini games.
In the context of this study, the concept of learning content in relationship to TML refers to lesson topics, lessons, and concepts to be learnt including learning materials, instructions, and tutorials provided in IVR application to facilitate learners' comprehension. According to Lee et al. (2020), the content quality of a VR application generally refers to the appropriateness, accuracy, flow of the presentation, and completeness of learning materials. Therefore, the adequacy of learning content could moderate the learning outcome by enhancing the degree of reflective thinking a learner may possess. Therefore, this study proposed the following hypothesis.
H8. Adequacy of learning contents will positively moderate reflective thinking and can influence learners’ comprehension when playing VR mini games.
Description of the VR application and the mini games to foster computational thinking
To examine the effect of IVR mini games on higher education learners’ perceived cognitive outcomes, this study experimented with an VR application (iThinkSmart) containing mini games aimed at fostering students’ understanding of computational thinking concepts and supporting players to gain problem-solving skills. In other words, student's computational thinking competencies such as algorithmic thinking, recursive thinking, problem decomposition, abstraction, and pattern recognition can be supported by playing the mini games contained in the iThinkSmart IVR application. These mini games include (a) River Crossing, (b) Mount Patti Treasure Hunt (MoPaTH), and (c) Tower of Hanoi. Figures 3a–c shows the screenshot of these mini games interface.
The river crossing mini game, for example, is a logic puzzle that allows students to perform sequential and computational movements of objects while following certain conditions to gain analytical thinking, algorithmic analysis, and problem-solving skills (Lamagna, 2017). In addition, research shows how River Crossing puzzle can demonstrate an AI approach to solving the Breadth-first search (BFS) algorithm (Ratnadewi et al., 2018), which is important knowledge in programming. For example, the IVR app showcased in this study allows students to solve the River Crossing puzzle by applying computational concepts, such as reverse engineering, combination, and algorithmic thinking skills to move items across the river with predefined constraints. By doing so, students can demonstrate how to apply computational thinking concepts to unravel optimization problems. Moreover, finding an optimal solution to an optimization problem remains a critical problem to deal with in computer science.
The MoPaTH mini game contains challenges which are computational thinking logic puzzles adapted from Bebras Computing Challenge (2022) and the Beginners Computational Thinking Test (BCTt) (Zapata-Cáceres et al., 2020). The puzzles are presented as multiple-choice quizzes that require critical thinking and problem-solving skills for a player to resolve within the constraint of sixty seconds to unlock a hidden treasure. The MoPaTH mini game simulates a succession of fallen rocks that are directed at the player. Concurrently, there are computational thinking quizzes that display at intervals (typically every one and a half minutes). The player must provide a correct solution to the puzzle in order to stop the rock from falling on him/her. Otherwise, the player suffers from being crushed by the rock and no score point is earned. For every correct solution to a quiz, the player earns a score point and progresses in the climbing of the mountain to the top. Playing the MoPaTH mini game allow students to gain computational thinking and problem-solving skills by, for example, using random outcomes from rolling dice to backward engineer what is contained in their faces, which is one of the BCTt quizzes. The dice problem is common in teaching if-else-if conditional block statements in programming. Therefore, students can gain computational thinking that can lead to their programming knowledge by playing the MoPaTH mini game. Further, the MoPaTH mini game also facilitates the players' expedition experience of a contextual virtual environment.
The tower of Hanoi mini game is a mathematical puzzle that contains three pegs and several discs. This puzzle is well known and used in the field of cognitive psychology and mathematics to teach several concepts (Klavžar & Milutinović, 1997; Kotovsky et al, 1985). Because of the connection of these fields to computer science, Tower of Hanoi puzzle is also used to teach computing concepts including recursion (Butgereit, 2016). In the IVR application showcased in this study, the Tower of Hanoi was integrated to visualize recursion so that players can comprehend and gain recursive thinking skills, which is one of the concepts of computational thinking. Recursion is a computational thinking concept that computer science students need to demonstrate programming skills, however, understanding recursion can be difficult, which why this mini game aim to foster its understanding through a visualization approach.
Research context, ethics, participants, and procedure
This study was conducted at a federal university located in the North-central region of Nigeria. After receiving approval from the institutional head, purposive convenience samples of 60 students were invited to participate in the study. Out of 60 students who registered to participate in the study, 49 students consisting of 38 (77.6%) males and 11 (22.4%) females completed the activities. The number of students that participated in the study was logistically manageable and considered adequate for an evaluation of a VR application as shown in previous studies (Butt et al., 2018; Kim et al., 2020; Lui et al., 2020). Furthermore, physical contact was limited during the time the study was conducted due to the COVID-19 pandemic. Inclusion into the study was intentionally limited to computer science (CS) students because they may have completed CS courses that could give them initial background on computational thinking concepts. The participants gave their informed consent through an online recruitment form where we introduced the goal of the experiment, and informed them that participating in the study was voluntary and that they could withdraw from participating at any stage. In addition, the participants gave their consent to use data collected during the experiment, including the images, for research purposes.
During the experiment, the participants were first introduced to the concepts of computational thinking, HMDs, and how to set up the VR application. This initial set up for the experiment took 30 min. Next, the researchers shared the iThinkSmart VR application for Android Package (APK) with all participants through the Google drive link where they downloaded and installed the App on their smartphones. After the installation was completed, each student played the mini games. The duration for completing the games varied between participants. An average of 30 to 40 min was used for playing the IVR mini games using the HMDs and hand controllers provided by the researchers as shown in Fig. 4. Next, each player completed a survey containing a questionnaire to elicit information on their gameplay and VR experience (described in the next section).
To examine how the IVR features and game elements of an educational tool influence learners’ reflective thinking and comprehension, this study adapted instruments from existing literature. For immersion, we adapted instruments that measure the participants' experience from Hamari et al., (2016) and Makransky and Petersen (2019). For indicators that measure interaction, this study adapted instruments from Bellur and Sundar (2017). Furthermore, instruments to measure participants' experience of game element features (challenge, goal clarity, and feedback) were adapted from Fu et al. (2009) and Fokides et al. (2019), whereas the moderating construct (adequacy of learning contents) instruments were also adapted from Fokides et al. (2019). Additionally, the overall experience of the participants after the experiment was collected using an online form. The analysis of this qualitative data regarding the user’s experience is presented in the result section.
To measure the participant's perceived cognition after playing the mini game, the study adapted the instruments utilized in previous studies on IVR structural model (Lee et al., 2010; Makransky & Petersen, 2019). Although the learning outcomes of the structural model formulated for these previous studies differs from our study, we collected the IVR features and game elements respectively, which forms the independent constructs relevant to this study in examining their effects on learner’s cognition. All the items for the constructs in the structural model were measured with a 5-point Likert scale as shown in Appendix.
To test the hypothesis formulated in this study and to examine how the IVR features, and game elements of an educational tool influence learners’ reflective thinking and cognition demonstrated in Figs. 1 and 2, partial least squares structural equation modelling (PLS-SEM) analysis was conducted using the WarpPLS 7.0 software (Kock, 2020). PLS-SEM technique is increasingly used in IVR studies (Cheng & Tsai, 2020; Lin et al., 2020; Barrett et al., 2021), whereas WarpPLS software is suitable for examining both factor-based SEM and non-leaner analysis of structurally linked variables in path models (Kock, 2020).
This study assessed the measurement models based on the significance of each estimated coefficient or loading, convergent validity, and discriminant validity. We also assessed model fit as shown in Table 1, which demonstrates that the model measurement is satisfied based on the criteria for conducting SEM with WarpPLS (Kock, 2020).
The convergent validity was assessed based on the composite reliability coefficient (CRC) whose value should not be less than 0.7; Cronbach’s alpha coefficient (CRC) whose value should be preferably above 0.7; and average variance extracted (AVE) whose value is recommended to be 0.5 and above according to literature (Hair et al., 2011). On the other hand, the assessment of the discriminant validity was based on the correlations among the latent constructs whose values should be less than the square of AVE (Lin et al., 2020).
Based on the item combined loading in Table 2, all indicators load significantly on their latent constructs with values higher than the recommended thresholds whereas the convergent validity of all constructs meets the requirements as shown in Table 3.
Furthermore, the result of the discriminant validity presented in Table 4 shows that the square root of AVE for each construct examined (bold text values in diagonal) is higher than the correlation coefficients for other constructs. The results of the convergent validity and discriminant validity obtained from the SEM-PLS analysis confirmed that the adapted items in the scale were reliable and valid for examining the relationship between game elements and VR features and their impact on learners’ cognition within the theoretical framework of TML.
The structural model
The analysis of the structural model, the relationship between the constructs, and their statistical significance are presented in Fig. 5. The model consists of two parameters, i.e., path coefficient (β) and square multiple correlations (R2). While the path coefficient examines the causal effect of a variable on another connected variable, R2 explains the extent of variation of a construct by the independent constructs (Lin et al., 2020).
As shown in Fig. 5, seven out of eight path relations were found significant with path coefficient (β) ranging from (0.25 to 0.55), thus resulting in seven hypotheses supported and one not supported. Indeed, the result revealed that IMM → COMP (β = 0.48, P < 0.01) and INT → COMP (β = 0.41, P < 0.01) have direct, strong, and significant relationships, respectively. Similarly, GCL has a positive and significant direct influence on RFL (β = 0.55, P < 0.01); GCL significantly predicts FB (β = 0.43, P < 0.01); CHA has a strong and positive influence on FB (β = 0.43, P < 0.01), however, CHA does not positively influence players’ RFL (β = -0.06, P = 0.33).
To understand the moderating effect of learning contents deployed in an educational IVR game-based application, this study examined the indirect effect of adequate learning contents among the latent variables of the structural model. In other words, a moderator analysis was conducted to examine the indirect effect of adequate learning content on learners’ cognition by connecting game elements constructs fully mediated by RFL and moderated with ADQ. The moderation result shows that ADQ fully moderated RFL with a positive and significant effect on COMP (β = 0.36, P < 0.01).
Therefore, for the direct path-coefficient, immersion shows to be the strongest predictor of learners’ comprehension, whereas, in the indirect path-coefficient, goal clarity remains the strongest predictor of comprehension, respectively. In general, the proposed hypothesis (H1, H2, H3, H5, H6, H7, and H8) were supported while H4 was not supported, as shown in Table 5.
Regarding the values of the R2 of the structural model, Fig. 5 presents that the latent variables of educational game elements accounted for 62% of the variance in FB (R2 = 0.62), whereas that of the cognitive factor accounted for 26% of the variance in RFL (R2 = 0.26) and 51% of the variance in COMP (R2 = 0.51).
The qualitative analysis of the data obtained from participants in this study through open-ended form was analy4sed following the thematic coding approach, a variation entitled “structured tabular thematic analysis (ST-TA)” by Robinson (2022). The ST-TA was developed as a technique for thematic analyses when working with brief qualitative text. In Table 6, we present a summary of codes generated from the open-ended data collected from a few participants. Although the collection of users' feedback was very important to the researchers, the majority of the participant did not feel obliged to fill out the form since the field was intentionally made not compulsory. Notwithstanding, the analysis of the few data provides insights regarding users' experience in interacting with the IVR showcased in this study.
First, the data was codded, and thereafter, the codes were grouped into themes. Generally, three themes emerged from the codes which include interaction, aesthetic, and learning outcomes. Figure 6 shows a map of themes developed in this study regarding how users’ interaction with an IVR application with its aesthetical characteristics impacts their perceived learning outcome.
Although users were immersed in using the VR application, the overall feedback shows that further improvement is needed to increase the experience. In addition, the result shows that because some aesthetics features in the VR application limit users' experience, their learning outcomes did not seem to be significantly impacted. One possible reason could be that users were distracted by these limitations that the aesthetic features post such that they may not maximize the educational element of the mini games. It is not clear whether the learning content adds to the cognitive load that may affect the learning outcomes, which is contrary to the findings of Makransky et al. (2019).
This study explored how IVR features and educational game elements influence learner's reflective thinking and cognition using an SEM-PLS analysis. The analysis of the measurement model and the structural model was satisfied, indicating that the proposed model was reliable based on all the quality criteria employed. The results indicate several factors that directly and indirectly relate to the learners' perceived cognition using IVR and GBL approaches. In the structural model, feedback, goal clarity, and game challenge have an indirect relationship with learners' comprehension, while immersion and interaction have a direct relationship with learners' comprehension. This study shows the centrality of reflective thinking in the cognition model. Recent research has emphasized the importance of reflective thinking and identified that engaging learners in active reflection need further investigation (Chen et al., 2019). When the goal of learning through a game-based approach is clear to the learners, it can lead to more impactful feedback and improve the experience of playing VR mini games. Similarly, the level of learner engagement through the challenge presented in VR mini games may potentially enhance learners' reflective thinking and consequently influence their comprehension. Surprisingly, this study shows that game challenges did not predict the learner's reflective thinking. As an immersive IVR mini game, one could explain this finding by perceiving that players could find it overloading when the game challenge is increasing. Moreover, Chen et al. (2019) posit that learner's engagement in reflective thinking can generally be a daunting task.
This study differs from the earlier studies conducted by Lee et al. (2010) and Makransky and Petersen (2019), where the authors used usability and VR features to predict reflective thinking. Whereas in this study, the interrelationships of feedback, goal clarity, and game challenge are the indirect determinants of learners’ cognition. Despite the non-significance of game challenge relationship with reflective thinking, it contributes indirectly to the learner's cognition through feedback.
Furthermore, the level of immersion, which could be low or high, and interaction in educational VR applications directly predict the learner's comprehension without any third variable intervention. This study indicates that the higher the learner's immersion and interaction experience with VR mini game, the higher the learner's comprehension. A recent study corroborates the significance of immersion and interaction as features of VR technology on learning experience and learning outcomes (Makransky et al., 2020). While comparing learning through IVR and video, Makransky and his colleagues concluded that the distinguishing mark between IVR and video learning is interactivity, immersion, or a combination of the two factors. The finding from this study regarding how IVR features influence learners' cognition aligns with the findings of Lee et al. (2010), which revealed that immersion of a VR application significantly predicted cognitive benefits. On the contrary, Hamari et al. (2016) argued that immersion did not predict perceived learning outcomes in a non-VR GBL environment. In their study, Cheng and Tsai (2020) highlighted the importance of IVR and its characteristic features in science education, noting that the type of IVR devices and applications can influence users' experience and learning outcomes. Although our qualitative analysis revealed a similar result where certain aesthetic features of the VR application were below users’ expectations and in turn, did not seem to significantly impact their learning outcomes, further research may be required to better understand the conditions under which immersion and interaction influence learner experiences and learning outcomes.
Among the indirect variables, this study revealed that goal clarity is the strongest predictor of reflective thinking, suggesting that the clearer the goal of an educational mini game, the more players would reflect and demonstrate critical thinking. Similarly, the result revealed that the higher the learner's perception of goal clarity, the greater the learner's positive feedback, while the feedback grows with VR mini games challenges. This development indicates that the feedback received from VR mini games based on goal clarity may be positive and rewarding, which in turn may increase motivation for the learners to resolve more challenges.
This study also established the interaction effects of the adequacy of learning contents. The findings show that the interaction effects of reflective thinking on comprehension can be positively influenced by the value of the adequacy of learning content. The interaction effect of the adequacy of learning contents reveals higher relationship strength than when reflective thinking relates directly to learners' comprehension. Interestingly, the adequacy of learning content positively moderates reflective thinking and significantly affects learners' comprehension while playing IVR mini games. Unlike the moderating interaction effects of this study, Cheng and Tsai (2020) earlier used immersion to mediate the relationship between learning traits and learning attitudes, whereas our study delineates immersion as a predictor of the dependent variable (comprehension).
Limitations of the study
This study involved some limitations. First, there is a limitation due to the small sample size recruited to conduct this study. Future studies could increase the number of participants to obtain more data that may impact the study findings. Second, the study setting was limited to a university and a single department. Widening the scope of the study in terms of settings and disciplines could provide more insightful outcomes. Third, this study only developed its structural model around two independent variables of IVR (immersion and interaction) and three game elements (challenge, goal clarity, and feedback) to investigate the impact of IVR and GBL on higher education learners’ cognition. Future studies could consider including other IVR features and game elements to understand the behaviours of the phenomenon examined. Another important limitation is that all constructs, including learners’ cognition, were measured by student self-ratings, which may be inaccurate. Future research could utilise other measures of cognition, such as performance testing. Although these limitations exist, findings from this study provide an understanding of how an IVR and GBL application can impact learners' reflective thinking and comprehension through a structural modelling approach. In addition, the qualitative analyses provide further insights to reinforce the study’s findings. The implication of this study demonstrates how game elements and IVR features can support the design of an educational intervention to support higher education students' computational thinking skills, particularly in the context of developing countries.
Conclusions and implications
This study responds to the research gap in IVR literature by showing direct and indirect factors that influence learners' cognition using GBL within the context of higher education students. Also, the study shows the moderating effect of the adequacy of learning contents as a catalyst of learners' cognitive benefits. It is enthralling that the insights from this study show how game elements are associated with IVR features to foster learners' reflective thinking and cognition through engagement, interactivity, and immersion when playing IVR mini games. This finding will help the VR researchers, managers, and educational game developers—particularly from developing countries—to adopt the best practices related to developing IVR game-based applications to foster critical thinking required of all 21st-century learners.
Computer science educators need to pay close attention to learning content through IVR mini games and make it adequate and robust for the students to gain computational thinking competence because it can greatly influence students' reflective thinking and comprehension. Since the virtual environment is an artificial environment capable of improving the learning experience, educators need to stimulate learners' curiosity through reflective thinking. In addition, educators and practitioners should pay attention to the factors responsible for learners' reflective thinking and cognition, such as feedback, goal clarity, game challenge, immersion, and interaction. Practitioners should also consider the moderating strength of adequacy of learning contents between reflective thinking and comprehension. For the learners to maximize cognitive benefits, all six factors (feedback, goal clarity, game challenge, immersion, interaction, and adequacy of learning content) should be considered while planning to design educational IVR game-based applications.
Availability of data and materials
The datasets generated during this study are available from the corresponding author.
Immersive virtual reality
Association for Computing Machinery
Adequacy of learning contents
Mount Patti Treasure Hunt
Partial least squares structural equation modelling
Composite reliability coefficient
Cronbach’s alpha coefficient
Average variance extracted
Structured tabular thematic analysis
Aho, A. V. (2012). Computation and computational thinking. The Computer Journal, 55(7), 832–835.
Alavi, M. (1994). Computer-mediated collaborative learning: An empirical evaluation. MIS Quarterly, 18, 159–174.
Alrehaili, E. A., & Al Osman, H. (2019). A virtual reality role-playing serious game for experiential learning. Interactive Learning Environments, 30, 922.
Apiola, M., & Sutinen, E. (2021). Design science research for learning software engineering and computational thinking: Four cases. Computer Applications in Engineering Education, 29(1), 83–101.
Agbo, F. J., Sanusi, I. T., Oyelere, S. S., & Suhonen, J. (2021a). Application of Virtual Reality in Computer Science Education: A Systemic Review Based on Bibliometric and Content Analysis Methods. Education Sciences, 11(3), 1–23.
Agbo, F. J., Oyelere, S. S., Suhonen, J., & Laine, T. H. (2021b). Co-design of mini games for learning computational thinking in an online environment. Education and Information Technologies, 1–35. https://doi.org/10.1007/s10639-021-10515-1.
Agbo, F. J., Oyelere, S. S., Suhonen, J., & Tukiainen, M. (2022). Design, development, and evaluation of a virtual reality game-based application to support computational thinking. Educational technology research and development, 1–33. https://link.springer.com/article/10.1007/s11423-022-10161-5.
Agbo, F. J., Oyelere, S. S., Suhonen, J., & Adewumi, S. (2019). A systematic review of computational thinking approach for programming education in higher education institutions. In Proceedings of the 19th Koli Calling International Conference on Computing Education Research (pp. 1–10).
Bower, M. (2019). Technology-mediated learning theory. British Journal of Educational Technology, 50(3), 2215–2233.
Bower, M., & Jong, M. S. Y. (2020). Immersive virtual reality in Education. British Journal of Educational Technology, 51(6), 1981–1990. https://doi.org/10.1111/bjet.13038
Barrett, A. J., Pack, A., & Quaid, E. D. (2021). Understanding learners’ acceptance of high-immersion virtual reality systems: Insights from confirmatory and exploratory PLS-SEM analyses. Computers & Education, 169, 104214.
Bebras Computing Challenge. (2022). https://www.bebraschallenge.org/
Beghetto, RA (2018). What If? Building students' problem-solving skills through complex challenges. ASCD.
Bellur, S., & Sundar, S. S. (2017). Talking health with a machine: How does message interactivity affect attitudes and cognitions? Human Communication Research, 43(1), 25–53.
Butgereit, L. (2016, November). Teaching recursion through games, songs, stories, directories and hacking. In 2016 International Conference on Advances in Computing and Communication Engineering (ICACCE) (pp. 401–407). IEEE.
Butt, A. L., Kardong-Edgren, S., & Ellertson, A. (2018). Using game-based virtual reality with haptics for skill acquisition. Clinical Simulation in Nursing, 16, 25–32.
Cachero, C., Barra, P., Meliá, S., & López, O. (2020). Impact of programming exposure on the development of computational thinking capabilities: An empirical study. IEEE Access, 8, 72316–72325.
Carretero Gomez, S., Vuorikari, R. and Punie, Y., DigComp 2.1: The digital competence framework for citizens with eight proficiency levels and examples of use, EUR 28558 EN, Publications Office of the European Union, Luxembourg, 2017, ISBN 978-92-79-68006-9 (pdf),978-92-79-68005-2 (print), 978-92-79-74173-9 (ePub), https://doi.org/10.2760/38842 (online), https://doi.org/10.2760/836968 (print), https://doi.org/10.2760/00963 (ePub), JRC106281.
Chaves, E. M., Braga, P. B. D. A., Montenegro, Y. F. L., Rodrigues, V. B., Munguba, M. C., & de Albuquerque, V. H. C. (2021). Toc-Tum mini-games: An educational game accessible for deaf culture based on virtual reality. Expert Systems, 38(1), e12470.
Chen, M. R. A., Hwang, G. J., & Chang, Y. Y. (2019). A reflective thinking-promoting approach to enhancing graduate students’ flipped learning engagement, participation behaviors, reflective thinking and project learning outcomes. British Journal of Educational Technology, 50(5), 2288–2307.
Cheng, K. H., & Tsai, C. C. (2020). Students’ motivational beliefs and strategies, perceived immersion and attitudes towards science learning with immersive virtual reality: A partial least squares analysis. British Journal of Educational Technology, 51(6), 2140–2159.
Czerkawski, B. C., & Lyman, E. W. (2015). Exploring issues about computational thinking in higher education. TechTrends, 59(2), 57–65.
de Jong, I., & Jeuring, J. (2020, November). Computational Thinking Interventions in Higher Education: A Scoping Literature Review of Interventions Used to Teach Computational Thinking. In Koli Calling'20: Proceedings of the 20th Koli Calling International Conference on Computing Education Research (pp. 1–10).
Fokides, E., Atsikpasi, P., Kaimara, P., & Deliyannis, I. (2019). Factors influencing the subjective learning effectiveness of serious games. Journal of Information Technology Education: Research, 18, 437–466.
Fu, F. L., Su, R. C., & Yu, S. C. (2009). EGameFlow: A scale to measure learners’ enjoyment of e-learning games. Computers & Education, 52(1), 101–112.
Gamage, L. N. (2021). A bottom-up approach for computer programming education. Journal of Computing Sciences in Colleges, 36(7), 66–75.
Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43.
Hair, J. F., Ringle, C. M., & Sarstedt, M. (2011). PLS-SEM: Indeed a silver bullet. Journal of Marketing Theory and Practice, 19(2), 139–152. https://doi.org/10.2753/MTP1069-6679190202
Hamari, J., Shernoff, D. J., Rowe, E., Coller, B., Asbell-Clarke, J., & Edwards, T. (2016). Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior, 54, 170–179.
Hu, C. (2011). Computational thinking: what it might mean and what we might do about it. In Proceedings of the 16th annual joint conference on Innovation and technology in computer science education (pp. 223–227).
Imlig-Iten, N., & Petko, D. (2018). Comparing serious games and educational simulations: Effects on enjoyment, deep thinking, interest and cognitive learning gains. Simulation & Gaming, 49(4), 401–422.
Kim, K. G., Oertel, C., Dobricki, M., Olsen, J. K., Coppi, A. E., Cattaneo, A., & Dillenbourg, P. (2020). Using immersive virtual reality to support designing skills in vocational education. British Journal of Educational Technology, 51(6), 2199–2213.
Klavžar, S., & Milutinović, U. (1997). Graphs S (n, k) and a variant of the Tower of Hanoi problem. Czechoslovak Mathematical Journal, 47(1), 95–104.
Kock, N. (2020). Multilevel analyses in PLS-SEM: An anchor-factorial with variation diffusion approach. Data Analysis Perspectives Journal, 1(2), 1–6.
Kotovsky, K., Hayes, J. R., & Simon, H. A. (1985). Why are some problems hard? Evidence from tower of Hanoi. Cognitive Psychology, 17(2), 248–294.
Lamagna, E. A. (2017). Frogs+ puzzles= algorithmic thinking. Journal of Computing Sciences in Colleges, 32(6), 111–119.
Lee, E. A. L., Wong, K. W., & Fung, C. C. (2010). How does desktop virtual reality enhance learning outcomes? A structural equation modeling approach. Computers & Education, 55(4), 1424–1442.
Lee, M., Lee, S. A., Jeong, M., & Oh, H. (2020). Quality of virtual reality and its impacts on behavioral intention. International Journal of Hospitality Management, 90, 102595.
Lin, Y., Wang, G., & Suh, A. (2020). Exploring the effects of immersive virtual reality on learning outcomes: A two-path model. In D. Schmorrow & C. Fidopiastis (Eds.), Augmented Cognition. Springer.
Liu, H., & Zhong, C. (2018, October). Assessing programming education approaches for college beginners with different backgrounds. In Proceedings of the 10th International Conference on Education Technology and Computers (pp. 136–139).
Lockwood, J., & Mooney, A. (2018). Computational thinking in secondary education: Where does it fit? A systematic literary review. International Journal of Computer Science Education in Schools, 2(1), 41.
Lui, M., McEwen, R., & Mullally, M. (2020). Immersive virtual reality for supporting complex scientific knowledge: Augmenting our understanding with physiological monitoring. British Journal of Educational Technology, 51(6), 2181–2199.
Makransky, G., Andreasen, N. K., Baceviciute, S., & Mayer, R. E. (2020). Immersive virtual reality increases liking but not learning with a science simulation and generative learning strategies promote learning in immersive virtual reality. Journal of Educational Psychology, 113, 719.
Makransky, G., & Petersen, G. B. (2019). Investigating the process of learning with desktop virtual reality: A structural equation modeling approach. Computers & Education, 134, 15–30.
Mannila, L., Dagiene, V., Demo, B., Grgurina, N., Mirolo, C., Rolandsson, L., & Settle, A. (2014). Computational thinking in K-9 education. In Proceedings of the working group reports of the 2014 on innovation & technology in computer science education conference (pp. 1–29).
Mayer, R. E. (2014). Computer games for learning: An evidence-based approach. MIT Press.
Mütterlein, J. (2018). The three pillars of virtual reality? Investigating the roles of immersion, presence, and interactivity. In Proceedings of the 51st Hawaii international conference on system sciences.
Parong, J., & Mayer, R. E. (2018). Learning science in immersive virtual reality. Journal of Educational Psychology, 110(6), 785–797.
Petersen, G. B., Klingenberg, S., Mayer, R. E., & Makransky, G. (2020). The virtual field trip: Investigating how to optimize immersive virtual learning in climate change education. British Journal of Educational Technology, 51(6), 2099–2115.
Plass, J. L., Homer, B. D., & Kinzer, C. K. (2015). Foundations of game-based learning. Educational Psychologist, 50(4), 258–283.
Robinson, O. C. (2022). Conducting thematic analysis on brief texts: The structured tabular approach. Qualitative Psychology, 9(2), 194.
Ryoo, J., Lee, D., & Techatassanasoontorn, A. A. (2016). Virtual world-based information security learning: Design and evaluation. Journal of Information Science Theory and Practice, 4(3), 6–29.
Sukirman, S., Ibharim, L. F. M., Said, C. S., & Murtiyasa, B. (2021). A strategy of learning computational thinking through game based in virtual reality: systematic review and conceptual framework. Informatics in Education. https://doi.org/10.15388/infedu.2022.07
Wang, Y., Rajan, P., Sankar, C., & Raju, P. K. (2014). Relationships between goal clarity, concentration and learning effectiveness when playing serious games. In Proceeding of Twentieth Americas Conference on Information Systems, Savannah, Georgia (pp. 1–10).
Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35.
Wing, J. (2009). Computational thinking. Journal of Computing Sciences in Colleges, 24(6), 6–7.
Zapata-Cáceres, M., Martín-Barroso, E., & Román-González, M. (2020). Computational thinking test for beginners: Design and content validation. In 2020 IEEE Global Engineering Education Conference (EDUCON) (pp. 1905–1914). IEEE.
Zhao, J., Lin, L., Sun, J., & Liao, Y. (2020). Using the summarizing strategy to engage learners: Empirical evidence in an immersive virtual reality environment. The Asia-Pacific Education Researcher, 29(5), 473–482.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
See Table 7.
About this article
Cite this article
Agbo, F.J., Olaleye, S.A., Bower, M. et al. Examining the relationships between students’ perceptions of technology, pedagogy, and cognition: the case of immersive virtual reality mini games to foster computational thinking in higher education. Smart Learn. Environ. 10, 16 (2023). https://doi.org/10.1186/s40561-023-00233-1
- Computational thinking
- Immersive virtual reality
- Mini games
- Reflective thinking
- Technology-mediated learning
- Higher education