Combining virtual reality with asymmetric collaborative learning: a case study in chemistry education

Abstract

The use of Virtual Reality (VR) in education is getting more and more common, especially when hands-on learning experiences have to be delivered. With VR it becomes possible, e.g., to simulate dangerous or costly procedures that could hardly be implemented in real settings. However, engaging large classes in immersive laboratory activities may be difficult, since VR kits are still rather expensive for quantity purchases and may require powerful PCs as well as proper spaces to work. A possible way to deal with these issues could be to combine VR with so-called asymmetric Collaborative Learning (CL). CL is a particularly interesting pedagogical approach, as it make learners work in team to achieve a common educational goal, promoting critical thinking and active learning. In asymmetric CL, in particular, learners use different technologies to interact. When combined with VR, asymmetric CL could be used, e.g., to let some learners get immersed in a virtual environment, while some others actively participate in the experience using a desktop interface. This configuration could allow, in principle, to involve more learners in the same amount of time and with the same number of VR kits, while also letting them benefit of the advantages of CL. Based on these considerations, this paper investigates the impact of CL on VR-based education by leveraging an immersive virtual environment designed to support a laboratory experience in a Chemistry course. A user study was conducted by involving 46 university students enrolled in the course. Objective and subjective metrics were used to compare two education methods, i.e. one in which the students experienced the VR environment in isolation, another one in which pairs of students collaborated with an asymmetric approach. Students’ knowledge acquisition was assessed by means of theoretical quizzes, whereas practical performance was automatically measured during the VR experience. The experimental results showed that trading off VR-based, individual learning for CL may have positive effects on the acquisition of theoretical knowledge, but may be detrimental to the achievement of practical abilities if sufficient exposure to technology cannot be guaranteed.

Introduction

In the field of education, specifically in higher education (Pellas et al., 2021; Marks and Thomas, 2022), it is often necessary to link theoretical explanations with practical examples that can help to enhance the overall understanding of the tackled topics, thus increasing the students’ confidence (Adkins et al., 2023). In particular, in the case of engineering, medicine, and chemistry subjects, among others, it may be difficult to separate frontal lessons from laboratory activities; through these combined experiences, the students can learn theoretical concepts while applying them to specific problems and tasks, like operating on industrial machines (Pratticò and Lamberti, 2021), performing medical interventions (Kim et al., 2022), simulating reactions (Dinc et al., 2021), etc. These types of procedural competences cannot be achieved only through explanations provided by others, i.e. without a direct, hands-on component which is key to achieve effective mastery of involved operations; in fact, according to De Lorenzis et al. (2023), when both theoretical and practical contents are involved in a learning path, it is essential to give students the possibility to have a first-person experience with them.

The design and realization of such practical, hands-on experiences present a series of important challenges, which can make them hard to implement in real settings. As a matter of example, medical students need to learn how to perform surgical operations without making mistakes, but training these abilities can be complicated since, in many cases, they cannot practice directly on patients (Perez-Gutierrez et al., 2020). Similar considerations apply in the industrial domain, where personnel must train on the execution of certain procedures which, however, may be too risky without a sufficient experience. These issues affect also other domains; for instance, training first responders is typically difficult since it is not possible to expose them to hazardous situations or it may be too costly to let them experience these situations in a safe way (Lamberti et al., 2021).

The above examples show that, often, replicating the real procedures and training onto them may be unfeasible, due to possible dangers or costs: the common solution in these cases is to fall back on simplified, real-life simulations that try to recreate the main aspects of the original activities. Unfortunately, this solution is far from being perfect, as the limited realism of the simulation could hinder the overall quality of the training experience.

The problems that have been mentioned so far can be addressed by leveraging immersive technologies such as Augmented Reality (AR) (Tuli et al., 2022; Singh and Ahmad, 2024) and Virtual Reality (VR), which has been already used to overcome the limitations of real-life simulated exercises, both for training professionals (e.g., firefighters, policemen, etc. (Çakiroğlu and Gökoğlu, 2019; Calandra et al., 2022; Uhl et al., 2022)) as well as school or university students (e.g., in the fields of engineering, physics, etc. (De Lorenzis et al., 2023; Pirker et al., 2017)). Specifically, in multiple occasions, it was possible to observe that VR technology offers a series of advantages with respect to simulated practical activities, since it enables the creation of safe, configurable training experiences characterized by a high level of immersion and realism, the possibility to collaborate with others, and the opportunity to use automated scaffolding and assessment modules that can support autonomous operations. Overall, it was shown that VR training can be particularly beneficial for experiences concerning hands-on activities, performing better than printed material (Buttussi and Chittaro, 2021), video-based lectures (Lovreglio et al., 2021), frontal lessons without associated practice (De Lorenzis et al., 2023), and even low-fidelity simulated exercises (Calandra et al., 2022). Based on these considerations, there is an increasing interest in applying VR applications to support various instructional design methods and improve outcomes in higher education settings (Pellas et al., 2021), while reducing costs (Farra et al., 2019; Gwynne et al., 2020).

Notwithstanding, to date, adoption rates of VR technology in higher education have not been well described (Marks and Thomas, 2022). The use of VR technology, especially for the creation of laboratories activities in the mentioned context, is fragmented (Elmoazen et al., 2023) and characterized by several drawbacks often associated with organizational and logistical issues (Hidayanto and Prabowo, 2022). Specifically, on the one hand, VR kits (including a headset and a pair of hand controllers) which are typically exploited to deliver immersive VR experiences are still quite expensive for quantity purchases, and may require powerful PCs as well as proper spaces to work properly (Al Farsi et al., 2021), mainly due to the initial investment (Farra et al., 2019); on the other hand, classes in higher education institutions can be large, and every student must be guaranteed the same exposure to learning material, which may not be easy given the above limitations (Ouverson and Gilbert, 2021).

As a consequence, the adoption of VR applications in this context can be rather challenging, especially if they have to be used by each student individually. Nevertheless, there might be alternative ways to use VR that could allow to leverage its advantages while getting rid of the associated constraints. In particular, it could be worth investigating whether a well-known paradigm of traditional education like Collaborative Learning (CL) (Dahri et al., 2019) could be applied to VR education, thus letting multiple students engage in a learning experience by sharing a single (virtual, in this case) environment. Among the possible ways in which CL could be used in this context, a conventional VR application in which all the students have the same role and can connect from remote using separate headsets could address some space-related limitations, but would still require a duplication of the necessary equipment. A possible solution to this problem could be to design the application to be asymmetrical (i.e. accessible via different interfaces at the same time), thus allowing multiple users to experience it with a reduced number of VR devices.

Approaches based on asymmetric designs appear to have a positive impact on situated learning and knowledge transfer (Burova et al., 2022; Drey et al., 2022), although research on this topic is quite limited in the state of the art. The combined use of CL and immersive technologies calls for even more investigations, given the lack of studies not only on the use of asymmetric techniques in virtual environments (Thomsen et al., 2019; Lee et al., 2020), but also of pair learning methods (Pirker et al., 2020). Based on the above considerations, the aim of this paper is to investigate whether asymmetric CL in a VR-based educational experience can influence the effectiveness of the learning process. In particular, the interest is in determining how a team-based pedagogical approach combined with immersive VR can affect the acquisition of theoretical concepts and practical skills related to a hands-on procedure. To reach this goal, a user study has been carried out by considering a use case concerning a laboratory activity for a Chemistry course encompassing both theoretical and practical contents. The activity concerns the synthesis of biodiesel, which involves the use of dangerous or toxic reagents, and cannot be easily replicated by students in real life due to possible consequences of wrong operations.

Related works

This section presents a review of the literature relevant for the paper, focusing on the use of VR in education, the adoption of CL, and the combined use of VR and CL in educational contexts.

VR and education

Various studies have highlighted the benefits of using VR in educational settings (Kurniawan et al., 2019). In fact, VR is gaining popularity in informal education, due to its ability to create simulations of scenarios that are impractical to replicate in real-life. For instance, the work by Garcia Fracaro et al. (2022) showed how VR can be used for self-training and/or evaluate the performance of trainees involved in industrial operations; objective and subjective measures indicated that the considered technology could enhance the learning experience of the trained operators as well as their practical performance, by also preparing them better for possible emergencies. Nevertheless, efforts are also devoted to integrating VR-based experiences into formal education, where this technology has been showed to be particularly effective not only for improving performance, but also motivation at learning (Makransky et al., 2019). Thus, it is not surprising that the number of domains in which VR is being investigated in both informal and formal education settings is growing.

As a matter of example, in Calandra et al. (2021), Calandra et al. proposed a VR road tunnel fire simulator that functions as a serious game to train first responders and citizens on emergency procedures. Similarly, in Carrozzino et al. (2023), the authors presented a virtual simulator designed to train urban Search & Rescue operators; VR was used to recreate hazardous conditions that real rescuers typically encounter, allowing for realistic scenario-based training in the use of equipment. By using real-time ray tracing in combination with VR rendering, Pape et al. (2021) created a VR educational tool that can enable the implementation of repeatable, non-hazardous, feature-rich and cost-effective learning scenarios in the field of optical design.

Although these studies did not evaluate the effectiveness of VR for learning, they demonstrated the potential of the considered technology in various educational contexts. In contrast, other studies conducted in-depth examinations on the use of VR for the achievement of practical, hands-on abilities, also evaluating the effects of technology on learning performance.

For example, O’Connor and Rainford (2023) designed an immersive VR-based simulation to prepare medical students for clinical practice. The authors conducted a user study in which students were divided into two groups: a VR one, which received VR practice, and a control group, which did not. Assessments performed by clinical tutors to compare the practical performance of the two groups before and after the introduction of VR showed that the VR group performed significantly better than the control group.

It is worth observing that, to improve learners’ hands-on abilities, it is important to have objective and accurate ways to assess their practical performance (Reznick and MacRae, 2006). The added value of VR is that virtual environments can be easily endowed with measuring mechanisms enabling accurate and unbiased quantitative evaluations (Gallagher et al., 2005). For instance, Tai et al. (2022) presented a VR-based training environment for car detailing. In their study, the trainees’ practical performance was assessed by the devised VR application, which generated a final score based on errors made in the tasks and their completeness.

Other studies focused on academic performance, intended as the performance of higher education students evaluated by means of objective measures based on written exams focusing on the knowledge gain.

As a matter of example, Campos et al. (2022) investigated the impact of VR on students’ learning outcomes and their perception of the experience in an introductory Physics course. The study found that VR can enhance students’ attention and visualization abilities, leading to a positive attitude towards the learning experience. Like in O’Connor and Rainford (2023), the study conducted by Campos et al. encompassed a VR group and a control group to compare students’ performance. Two instruments were used: a pre-post theoretical quiz administered to both the group to assess the knowledge gain, and a subjective survey about students’ experience with VR, which was only administered to the VR group. Another study conducted in the academic context is that by De Lorenzis et al. (2023). The authors developed a VR-based application to prepare students in the execution of measurements on electrical cabinets. Specifically, they compared a traditional VR-based learning scenario (where VR was used to prepare and evaluate the study participants) against an alternative approach based on the concept of learning by teaching (where VR was used only to prepare the participants, who were then asked to guide other subjects). A pre-post quiz was used to assess the participants’ knowledge gain, showing the advantages associated to the use of VR in an uncommon way. Also Hu-Au and Okita (2020) studied students’ learning and behavior comparing two pedagogical approaches focused on a Chemistry laboratory: a VR-based experience and a traditional, real-life activity. The study used a theoretical pre-post questionnaire to analyze academic performance. The study found that academic performance was comparable across the two conditions, except for the application of knowledge, where only the VR participants showed a significant increase in their scores. There was also evidence that a VR environment may facilitate greater elaboration and reflection of the learning aspects compared to a real-life environment.

Although a number of benefits have been mentioned so far, the use of VR in education can also present downsizes. For instance, Al Farsi et al. (2021) identified three main limitations of VR-based learning that concern technology: cost, low quality computers and corresponding applications, and challenges for less technology-savvy individuals. Motion sickness is an another issues associate with the use of VR that some learners may experience during use (Bailenson, 2018). Also reading text through VR headsets may present challenges due to limited resolution, improper fit of glasses, or visual impairments, which can make it difficult to properly access written contents in virtual experiences (unless professional VR kits are used (Albus et al., 2021)). Other non-technical limitations concern learning outcomes. As outline by Makransky et al. (2019), despite its motivational properties, VR can overwhelm and distract learners, thereby reducing opportunities to achieve positive learning outcomes. However, recent studies have indicated that cognitive load when using VR can be lower that in other settings (Henstrom et al., 2023), suggesting that overload can depend on the topic and context.

Collaborative learning

If, on the one side, the use technology, including VR, is proving effective in the context of education, on the other side there are pedagogical approaches that are receiving more and more attention. One of them is CL, a teaching and learning approach that engages learners in pairs or teams for achieving a common learning goal (Dahri et al., 2019). With the growing complexity of the society, the need for thinking and working together on critical issues is continuously increasing, making ever more convenient to pass, in different contexts, from individual attempts to team work, and from autonomy to community (Laal and Ghodsi, 2012). Hence, the growing interest towards CL it not surprising, as learners get responsible for their mutual learning since the success of one learner helps the others to be successful too Laal and Ghodsi (2012). In fact, CL is recognized to come with many advantages: it can promote a learner-centered education, induce a positive attitude in learners, stimulate critical thinking, create an active learning environment, enable a seamless knowledge flow among the team members, etc. (Dillenbourg, 1999; Zhao and Zhang, 2009).

Over the years, the use of CL has progressively expanded, showing positive outcomes in various learning contexts, encompassing English language (Peng et al., 2019; Emir and Yangın-Ekşi, 2024), Information Technology (Konak et al., 2016), Chemistry (Yang and Wang, 2023), etc. In particular, CL has been proven to be advantageous when adopted in courses with collective laboratory activities, where students are asked to perform hands-on, collaborative tasks. For instance, in Shibley and Zimmaro (2002), Shibley Jr et al. determined the effect of CL on students’ attitudes and performance in an introductory Chemistry laboratory. Students were randomly assigned to either a control or a collaborative group. The control group experienced the laboratory individually, whereas the collaborative group used the CL approach. No differences were found in terms of learning gain, suggesting that CL did not affect students’ short-term performance; however, students in the collaborative group developed a more positive attitude towards Chemistry and laboratory activities, showing the advantages of this approach over a traditional one. Another successful example of CL adoption can be found in the work by Markut et al. (2023), where the students of an inorganic Chemistry course were asked to identify symmetry elements for seven molecules using common 2D representations, student-constructed 3D concrete models, and student-created drawings; the experimental results showed that the collaborative activity was associated with an increased level of engagement and fulfilled its pedagogical goals, confirming the positive impact that CL can have on laboratory experiences.

Despite the positive outcomes, CL is not without drawbacks. One of such drawbacks is the possible unequal participation among team members and the dominance of certain individuals, especially those who are particularly self-confident (Barros, 2011); for instance, Kiraly (2003) observed that weaker students tend to benefit from the more proficient ones, whereas the opposite is rare. Additionally, students may struggle to trust their team members, particularly if some of them prefer to work independently and lack motivation to collaborate. In these cases, awkward situations or misunderstandings may arise with certain team members, potentially leading to incomplete tasks. This observation is supported by the findings of Klimkowski (2006), who suggested that inadequate teamwork performance can hinder project coordination and goal attainment. Furthermore, in higher education institutions, there is often uncertainty about how to evaluate teamwork (Strom and Strom, 2002). Institutions aim to accurately assess group projects, but observing the collaborative processes among students is often difficult and it can be challenging to distinguish the individual contributions. Finally, collaboration in learning requires the design of specific and well-coordinated tasks that can allow students to practice teamwork skills; however, developing such tasks may be complex and time-consuming.

Collaborative learning and VR

Recent studies showed the importance of combining technologies, including VR, with CL, defining the so-called technology-enhanced CL (Wang and Shen, 2023; Pirker et al., 2020; Pidel and Ackermann, 2020).

VR can be particularly effective in this scenario, as it enables the creation of multi-users learning environments, which can foster remote socialization and cooperation among learners (Nguyen and Bednarz, 2020; Pirker et al., 2020). In particular, according to the systematic literature review conducted by van der Meer et al. (2023), five distinct advantages of the use of VR in the context of interest for this paper can be identified: (1) VR is an efficient tool that engages and motivates learners, (2) it can support distance learning and remote collaboration, (3) it provides multi- and interdisciplinary spaces for both learning and collaborating, (4) it helps to develop social skills, and (5) it supports CL-related paradigms and approaches.

The above review basically confirms that the combined use of VR and CL deserves further attention. However, even though, nowadays, immersive hardware is becoming cheaper and the coordination of multiple VR users in a single physical space may not be hard to achieve, the wide adoption of traditional, multi-user VR systems for CL purposes could be impractical, mainly because of the difficulties that would be faced by educational institutions in acquiring sufficient devices to accommodate all the students (Ouverson and Gilbert, 2021; Sari et al., 2023). This is especially true in higher education, where classes are usually large, and the delivery of VR experiences to a number of students can be particularly challenging (Ouverson and Gilbert, 2021).

A way to tackle the above issues could be represented by the adoption of an asymmetric approach in the implementation of VR experiences, as presented, e.g., by Burova et al. (2022), i.e. making virtual environments simultaneously accessible via diverse technologies with different immersion and control levels. In particular, in an asymmetric scenario, two roles are defined: the actor(s), provided with a high-immersion hardware (e.g., a VR headset with hand controller), and the assistant(s), interacting with the virtual environment with a different technology characterised by a lower degree of immersion. The factor that influences the degree of asymmetry is the interactivity of the assistant, which can be low, medium and high according to the taxonomy of asymmetric interfaces for collaborative immersive learning by Thomsen et al. (2019). These three levels of asymmetry are defined by the possibilities offered by the associated interaction technology: direct (e.g., control of the view direction or manipulative actions) for low asymmetry, indirect (e.g., 2D pointing, use of buttons, chat, etc.) for medium asymmetry, or none (e.g., verbal communication) for high asymmetry. Specifically, low asymmetry occurs when the assistant can directly influence the environment, such as changing perspective or manipulating objects; medium asymmetry occurs when the assistant can transfer information digitally through an interconnected digital interface; high asymmetry occurs when the assistant’s abilities are purely analogue, and information must be conveyed verbally. In experiences characterized by a medium asymmetry, such as those incorporating VR and desktop interfaces, environment awareness cues are particularly beneficial for team members (Ouverson and Gilbert, 2021).

In the state of the art, it is possible to find some studies presenting asymmetric VR experiences, focusing both on the creation of specific hardware suited to the specific scenario or the design of high-level applications. As a matter of example, Gugenheimer et al. (2017) proposed ShareVR, a proof-of-concept of an asymmetric application using floor projection and mobile displays in combination with positional tracking to visualize the virtual world for the non-VR user; this configuration allowed users to interact with the VR user and become part of the virtual experience; the results of a user study showed the effectiveness of the devised application in terms of enjoyment, presence and social interaction. Chen et al. (2015) studied the collaboration between a single VR user and other users who could join the virtual environment by hitching on the view of the former using wearable holographic displays and Skype-enabled devices; this setup presented opportunities for real time and asynchronous collaboration, that can be either proximal or remote. In the work by Ibayashi et al. (2015), a multi-touch tabletop device and several VR headsets were used in the field of architecture to make shared decisions when designing a given space. Architects and designers typically need at least two viewpoints to this purpose: a small-scale view, i.e. a first-person view of the space to see local details, and a large-scale external view, i.e. a top-down view of the entire space. In this work, by means of asymmetry users could discuss the design of the space from two viewpoints simultaneously. Another example is provided in Pan et al. (2018), where Pan et al. investigated how two users collaborate in four settings with different interfaces: AR collaborating with AR, AR collaborating with VR, AR collaborating with VR and virtual body, and AR collaborating with desktop. The study found that 3D interactions could facilitate the emergence of a leadership pattern, and that the effect of leadership was stronger in higher asymmetry settings. Specifically, users who experienced an AR interface with a high level of immersion and situational awareness exhibited a stronger effect of leadership.

Some studies focused on the use of asymmetric, collaborative VR applications in educational contexts. In fact, by fostering communication, asymmetry can have a positive impact on the experience and behavior of the learners, specifically in scenarios where collaboration is required to achieve a common learning goal (Thomsen et al., 2019). For instance, Thompson et al. (2018) and Uz-Bilgin et al. (2020) investigated methods to promote collaboration among students in a VR/tablet-based educational game. Their findings suggest that introducing different roles among students can positively influence collaboration. Similarly, Lee et al. (2020) demonstrated that roles played an important part in asymmetric collaboration, and that non-VR users can experience the same level of immersion of VR users. However, these studies did not conduct a formal assessment of the proposed methods and did not investigate how roles impacted learning. Another example on the use of asymmetry in a learning context is presented by the work in Drey et al. (2022), where Drey et al. presented two VR-based CL systems: a symmetric one where two learners use both a VR headset, and an asymmetric one where the assistant uses a tablet. The results of a user study showed that the symmetric system provided significant higher presence, immersion, and lower intrinsic cognitive load, which are all important for learning; however, the symmetric and asymmetric systems performed equally well in terms of learning effectiveness. This suggests that both of them could be actually considered for devising VR-based CL experiences.

Research gap

Looking at the current literature, it appears that there is a lack of studies investigating the possible advantages of an asymmetric approach (Agnès et al., 2023; Lee et al., 2020), and that very little is known about a combination of VR and pair learning (Pirker et al., 2020). Given also the previously mentioned difficulties in making every learner have his or her own VR headset (Al Farsi et al., 2021; Ouverson and Gilbert, 2021), it is clear that the combination of a team-based pedagogical approach with an asymmetric CL in a VR-based educational application has potential and is worth specific investigation.

Therefore, in this study, an immersive VR application was designed based on previous studies’ learning strategies, evaluation methods, and results. Its purpose is to provide a virtual laboratory experience in the context of a Chemistry course. The application can be used either individually, or by two co-located students using different interfaces. A user study was conducted to gather insights on the design of VR applications supporting asymmetric CL approaches, focusing on the impact of such approaches on the acquisition of theoretical concepts and practical skills.

Materials and procedures

This section describes the immersive VR application, by first presenting its single-user mode, in which only one student is expected to use the VR kit. Afterwards, the asymmetric mode, in which two students collaborate by using partly VR (the actor) and partly a desktop interface (the assistant), is illustrated. Finally, the evaluation functionalities embedded in the application are discussed.

A video of the VR application is included as supplemental material.

Technology

The VR application was implemented using Unity 2021.3.10 game engine and the OpenXR framework to handle the VR functionalities (i.e. the tracking of the controllers and the user interface). The final application consists of an executable file that could be deployed on consumer, VR-ready machines (PCs equipped at least with a NVIDIA GeForce GTX 1060 or equivalent), with no installation or additional download required.

Although the application supports various consumer VR kits, a configuration consisting of a Meta Quest 2 connected to a Microsoft Windows-based PC equipped with mouse, keyboard and a screen was specifically targeted. This configuration supports both the implementation of the single-user mode, with only one student wearing the VR headset, as well as of the asymmetric mode, with more than one students collaborating using both the VR- and the desktop-based interface. In the latter case, one of the students uses the mouse and keyboard to interact with the application while observing the virtual environment on the screen; another student wears the VR headset and interacts using the controllers. Despite the different interaction and visualization means, the students are connected to the same virtual environment, running on a single instance of the application without the need to rely on networking functionalities.

All the objects used in the virtual experience were modeled in Blender, mimicking the equipment available in a real laboratory at the university, and then imported into the game engine where materials and textures were added. All the textures were created using image processing software such as Gimp and Adobe Photoshop. The 3D models were optimized for immersive VR use, and the number of vertices for each model was kept relatively low (less than 5,000 vertices per object). Additional visual details (e.g., the simulation of liquid reagents) were added using custom shaders that were created using the Unity Shader Graph editor.

The logic, scripts and components that control the application were written using the MS Visual Studio suite, which can be connected to the Unity game engine to enable an easy-to-use development pipeline. Specifically, all the scripts were written in the C# programming language.

Design process

The design process of the devised application involved several meetings with professors from the Chemistry department of the authors’ university to define both educational and technological aspects. These meetings helped to establish the aspects of the biodiesel synthesis procedure that needed to be included in the virtual experience, together with the learning objectives of the application.

Initially, videos of the synthesis procedure recommended by the Chemistry department professors were analyzed to understand the key elements of the reaction. Subsequently, the professors executed the procedure step by step during a live demo in a laboratory, highlighting the main educational aspects. Photos and videos were taken to ensure the detailed reconstruction of the procedure in the virtual environment. Based on collected information, the procedure was split into three phases: in the first phase, the selection of the required glassware and the assembly of the the structure is performed; in the second phase, reagents are selected and measured; lastly, in the third phase, prepared reagents are used to perform the reaction and produce the biodiesel. Based on the discussion with the professors, each phase was associated with a set of evaluation parameters and a 0-to-100 score, presented in detail in Sect. "Evaluation Module".

The application’s design aims to challenge students both practically and theoretically. Through the virtual experience, they learn practical skills, like adopting an adequate behavior in a Chemistry laboratory, and theoretical knowledge, like the reagents required for the reaction and their quantities, as well as correct operations and their order. The professors requested to let the students be allowed to make mistakes, as long as these mistakes did not prevent them from continuing the procedure (e.g., allowing the students to freely choose and measure reagents, regardless of whether they are correct or not).

From a technical viewpoint, it was decided to allow interaction with virtual objects via VR controllers. This choice makes the application scalable and compatible with other consumer VR kits, since not every kit provides native hand-tracking capabilities like the Meta Quest 2. Another important technical choice was to reduce unnecessary movements. Hence, locomotion was not implemented in the virtual experience. Students can interact with all the objects in the virtual laboratory by remaining stationary at the center of a virtual space that is 1.5$\times$1.5m wide. These objects are positioned at close range and can be reached by physically turning around. This choice should improve the usability of the application for users with no previous VR experience. In addition, reducing motion in VR should also prevent motion sickness (Chang et al., 2020). Finally, certain interactions in the virtual experience were simplified. For instance, some laboratory objects were enlarged and some operations were modified to make them less difficult to replicate in VR (e.g., all the heating plates have a redesigned user interface that approximates the behavior of the real instrument). This choice was made since the focus of the experience should be on understanding and learning the operations to be performed, rather than on realistically reproducing specific movements.

Biodiesel synthesis procedure

The main objective of the real synthesis procedure is to produce biodiesel starting from three different reagents: an oil, an alcohol and a catalyst. Reagents can be either correct, partially correct or incorrect. Correct reagents are an oil, an alcohol and a catalyst that lead to the maximum yield of the reaction, whereas partially correct reagents still allow the reaction to take place but imply a lower yield. Incorrect reagents are any three reagents that do not include an oil, an alcohol and a catalyst (e.g., two oils and a catalyst), and lead to no reaction. To perform the reaction and produce the maximum amount of biodiesel, reagents must be used in specific amounts (namely, the molar ratio between oil and acid must be 1/6); otherwise, an excessive presence of water could result in the production of free fatty acids that could react with the catalyst and produce soap.

The necessary steps to safely and correctly perform the procedure are the following:

• Wear the adequate protections (gloves, coat, goggles) to operate in the laboratory;

• Select the correct glassware and assemble the structure in which the reaction will take place. Namely, the reaction requires a round bottomed flask with at least two necks, placed inside a container filled with water and heated by a heating plate. The flask must be connected to a condenser refrigerated by flowing water to avoid the complete evaporation of the reagents. The structure must also contain a separating funnel to separate the final product into biodiesel and soap (if present);

• Turn on the hood and open the water tap connected to the condenser;

• Select the correct reagents, aiming for the maximum yield of biodiesel.

• Heat the heating plate to 65 °C to boil the alcohol;

• Wait for the end of the reaction (at least one hour);

• Separate the biodiesel using the funnel.

User experience in VR

The steps of the synthesis procedure can be grouped in the three macro-phases:

• Choice of glassware and assembly of the structure where the reaction takes place;

• Selection and measurement of reagents;

• Insertion of reagents in the structure, reaction and separation of products.

First phase: glassware selection and structure assembly

During the first phase, the student can interact with two stations of the virtual laboratory (Fig. 1): the cabinet with glassware, and the fume hood. He or she needs to interact with a series of 3D objects that must be used to complete the phase:

• A lab coat;

• Nitrile gloves;

• Protective goggles;

• Support rod;

• Tongs;

• Flasks (400 ml, 500 ml, 600 ml);

• Separating funnel;

• Condenser;

• A container filled with water;

• Flat-bottomed flasks (1, 2, 3 necks);

• Round-bottomed flasks (1, 2, 3 necks);

• Thermocouple with cable;

• Tubes;

• Plugs;

• Perforated plugs;

• A heating plate.

Fig. 1

Virtual laboratory environment

As a first task, the student must equip himself or herself with the personal protective equipment (PPE) required for the reaction, just as he or she would do in a real laboratory. Should the student fail to choose the proper PPE, the experience would proceed anyway but the final score would be affected by the errors made.

Afterwards, the student must select the correct glassware required for the reaction, and move all the necessary objects into the hood where the structure has to be assembled (Fig. 2).

Fig. 2

First phase: Assembly of the structure in the fume hood

As the student selects the elements he or she considers necessary and adds them to the structure, he or she receives visual feedback on the correctness of the performed actions. The feedback provided by the application is quite generic and does not specifically indicate the error made, so that the student is prompted to reason, draw on his or her theoretical knowledge, and make corrections autonomously. When a predefined time limit is reached (set to 15 min), any errors made will be automatically corrected, so that the next phase can start with a correct configuration.

Second phase: reagents selection and measurement

During the second phase, the student can interact again with two stations: the hood, and the workbench. He or she needs to interact with a series of 3D objects that must be used to complete the phase:

• A set of reagents, from which the student can choose either correct, partially correct or incorrect ones for the considered reaction (Fig. 3);

• The safety data sheet of each reagent;

• A precision balance;

• Two graduated cylinders (100 ml, 250 ml);

• Two spatulas;

• Three containers.

Fig. 3

Second phase: Selection of reagents

Once the student has chosen the reagents he or she wishes to use, he or she should take the VR headset off and carry out the calculations needed to determine, based on knowledge of the reactions governing the process, the required quantities of the various reagents. To this aim, the application offers a desktop interface, which can be used on the PC. Once the quantities have been obtained and inputted in the application, the student can wear again the headset and use them to continue the experience. Afterwards, the students should use the other elements available on the workbench to measure the quantities of reagents required for the reaction. Specifically, liquid reagents should be measured with the graduated cylinders, and solid reagents should be weighted using the balance.

Once the measurements have been completed or the time limit has been reached (set to 30 min), whichever reagents have been chosen the experience continues to the third phase.

Third phase: insertion of reagents, reaction and separation

The third phase requires the student to operate in the hood. He or she needs to interact with a series of 3D objects that must be used to complete the phase:

• The previously assembled structure;

• A container;

• A heating plate;

• The cylinders and containers previously used to measure the reagents;

• Magnetic anchors.

As a first step, the student should place the reagents selected and measured in the second phase into the structure assembled in the first phase (Fig. 4), taking the right precautions so that the success of the reaction is not compromised (e.g., dissolving the solid components in the acid before starting the reaction).

Fig. 4

Third phase: Insertion of reagents into the structure

Afterwards, the student must set the reaction time, in accordance with the chosen reagents and his or her knowledge of the procedure. As stated earlier, the choices made in the second step have repercussions on this phase, particularly on the reaction yield. If correct or partially correct reagents have been selected, the student will be able to observe the products of the reaction and can move on to the separation step. The effects of using partially correct reagents or wrong quantities may lead to a decrease in the products obtained (which would impact the student’s evaluation). If incorrect reagents are used, though, the reaction does not take place, the experience will be completed, and the student will not be allowed to perform the separation on the obtained products.

Asymmetric mode

Besides the single-player mode described above, the devised application supports also a so called asymmetric mode, in which a group of students—two, in the specific case—can collaborate experiencing the above procedure using VR and a desktop interface on a PC. The student wearing the VR headset (actor) will be allowed to directly interact with the virtual objects. The student at the PC (assistant) will be allowed to see the virtual environment on a screen, and will have the possibility to highlight objects in the virtual environment via mouse and keyboard to help the other student. Since the interface used by the assistant is meant to be used on the same PC running the VR (the experience is co-located), the students are also allowed to talk; a voice communication channel could also be easily included in the application using Unity assets.

For the purpose of the user study, in the first half of the experience, the task of the assistant will be to help the other student to select the correct glassware and reagents. To this aim, he or she will have on the PC screen a view of the virtual environment from a fixed camera placed either in front of the glassware cabinet or the workbench, with the possibility to switch between the two. By clicking on one of objects with the mouse, an highlight is displayed onto them, visible also to the student wearing the headset.

Once the reagents have been selected, the VR student removes the headset, like in the single-user mode. Then, the students need to collaborate to calculate the quantities of reagents required for the reaction. In this case, they both use the desktop interface on the PC, through which they also have to insert the results of their calculations.

At this point, the two students will switch their roles: hence, in the second half of the experience, the student who was wearing the headset becomes the assistant, and viceversa. In particular, the student wearing the headset will perform the measurements and will carry out the operations required for the reaction, while the assistant will observe his or her operations on the PC screen, possibly highlighting objects and providing suggestions.

Although, in principle, the two students could decide when switching from one interface to the others, with the organization of the roles in the three phases described above it was possible to guarantee, for the user study, the same duration of the experience for the single-user mode and the asymmetric mode, as well as an almost comparable exposure to VR for the two team members in asymmetric mode.

Evaluation module

Based on the study by Lakshmana Naik et al. (2015) and on the indications received by the professors involved in the design of the application, an automatic module was implemented to evaluate the student’s performance. The module assigns an overall score by considering the student’s operations and the outcome of the simulated reaction.

A score is assigned for each phase, considering, e.g., errors made in the glassware selection, interaction with the fume hood, timers, reagents selection, etc. The score was designed considering the parameters described in Lakshmana Naik et al. (2015); moreover, aspects capable to mimic the evaluation that the professors would have performed during a real laboratory activity were also incorporated. For each phase, the maximum score that can be obtained is equal to 100.

Specifically, in the first phase, negative evaluations should be assigned if:

• The student fails to choose the correct PPE or fails to activate the fume hood; this was deemed necessary since, during real laboratory activities, it is forbidden to operate without the adequate protections or without following the laboratory safety guidelines. The experience in a virtual environment is inherently secure, and students may become overconfident and underestimate safety measures; for this reason, their simulation is essential and they must be enforced.

• The student selects the wrong glassware or assembles the glassware in the wrong way, since errors in the selection of the glassware can compromise the yield of the reaction and, consequently, the entire activity.

• The student fails to turn on the water tap, since a constant flow of water is necessary to avoid the complete evaporation of reagents during the reaction.

In particular, the score of this phase is initially set to 100 and decreased by 5 points (if the value becomes negative, it is set to zero):

• For each missing piece of PPE (gloves, coat, goggles);

• If the fume hood is not activated;

• If the water flow is not present;

• If the wrong piece of glassware is selected, or if a piece of glassware is missing, (the structure is composed of 14 elements, for a total of 70 points);

• If any element of the structure is placed incorrectly and overlaps other elements.

In the second phase, since the main goal of the procedure is the production of biodiesel, it is necessary to verify that the student is able to select the correct reagents, but also that he or she is able to determine their correct amount to avoid waste and enable the reaction. Specifically, if the student selects a partially correct combination, potentially leading to a sub optimal yield of the reaction, or a incorrect one, leading to no reaction, a penalty needs to be assigned accordingly. A penalty also needs to be assigned if the student, after choosing a correct or partially correct combination of reagents, fails to calculate the correct amount of each reagent, since errors in this phase could compromise the outcome of the procedure regardless of the selected reagents.

The score of the second phase is calculated once the student has confirmed the selected reagents and their quantities, and corresponds to the reaction yield computed using the amounts inserted in the application. This score was chosen since it takes into account both the type of reagents and the student’s calculated quantities, and produces a percentage value that can be expressed on a 0-to-100 scale. Additionally, 5 points are subtracted from the obtained score if the student fails to check the safety data sheets of the selected reagents (like for the first phase, it the score becomes negative, it is set to zero).

Finally, for what it concerns the third phase, if the chosen reagents are correct, then expected products are obtained. However, the yield of the reaction (i.e. the percentage of reagents that turn into biodiesel) is impacted by the quantities used, and may be lower than expected. If reagents are incorrect or are partially correct, and if quantities are not correct, the reaction does not take place or takes place only partially (due to saponification or dilution). In all these cases, the score has to be penalized.

Thus, the score for the third phase corresponds to the yield of the reaction, calculated considering the reagents that were inserted in the structure, normalized to the maximum value that could be obtained using the reagents at the student’s disposal (thus obtaining a score on a 0-to-100 scale). This assessment method was deemed necessary since the third phase of the procedure is strictly associated with the second one; evaluating the third phase without considering the second one could give an erroneous feedback to the students, leading to wrong behaviours during future, real experiences.

It is worth noticing that, since the evaluation module is able to produce a detailed report on each part of the experience, even in the asymmetric mode it is possible to use it for evaluating individual student’s performance.

Experimental evaluation

This section presents the user study that was designed building on the devised application to investigate the impact of asymmetric CL in VR-based learning scenarios. The VR application was deployed in an advanced Chemistry course taught at the authors’ university, and its single-user and asymmetric modes were leveraged to compare two different ways of using VR as a substitute of real-life laboratory activities.

Participants

The study, arranged following a between-subject design, involved 46 participants (17 males and 29 females) aged between 20 and 25 years ($M = 21.46, SD = 0.83$). The sample was randomly selected from students attending the mentioned course to ensure that all the study participants had the same, intermediate Chemistry knowledge. Considering their curriculum, the participants should have no previous knowledge on the biodiesel synthesis procedure. Before starting the experiment, the participants were illustrated the goal of the study and its organization, and were requested to sign an informed consent form.

Instruments

Two theoretical quizzes (TQ1, TQ2) were used to evaluate the participants’ knowledge before and after the VR experience. The use of theoretical quizzes to evaluate the participants’ knowledge gain, later used to validate a single VR-based training experience or compare different approaches, was inspired by other works in the literature, e.g., by Hu-Au and Okita (2020), Makransky et al. (2019), Albus et al. (2021), and De Lorenzis et al. (2023). The two quizzes were identical and consisted of 20 theoretical questions on the biodiesel synthesis. They included multiple choice, sorting, and open questions to evaluate the participants’ comprehension of the key steps and elements of the procedure. The questions were defined in collaboration with the professor and assistant professor of the course to ensure relevance and accuracy, and were inspired to the questions that are normally found in the exams. Additional validation was provided by other professors from the authors’ university. Each quiz session lasted 10 min.

A questionnaire (Q) was designed to investigate several aspects of the VR-based laboratory experience, such as the cognitive load, the motivation at learning, and the usability. All these subjective measures are not directly associated with the knowledge gain, but can influence the perception or even the effectiveness of a learning experience and must be taken into consideration to determine the validity of a devised approach. As seen in the literature, several outcomes are possible: some approaches may offer no advantages in terms of knowledge gain, but can still be preferred for their subjective qualities (Makransky et al., 2021); others can provide only objective advantages (De Lorenzis et al., 2023), or both objective and subjective (Calandra et al., 2022); finally, it is also possible that approaches that are potentially advantageous may be hindered by other factors such as the associated cognitive load (Makransky et al., 2019), resulting in motivating experiences that are unable to offer a significant knowledge gain. Based on these considerations, the questionnaire was deemed necessary to thoroughly investigate the impact of asymmetric CL in VR-based learning. Specifically, the devised questionnaire was based on several standard tools widely used in the literature, and consisted of the following sections.

• A section with custom questions on the participants’ previous experience, on perceived self-efficacy, and on aspects pertaining collaboration (or lack of it, for the SVR group). This section was mainly designed to profile the participants and recognize possible outliers (for instance, participants that had previous expertise with the considered procedure or with immersive technologies).

• A section corresponding to the NASA-TLX questionnaire (Hart and Staveland, 1988), assessing the cognitive load associate with the use of the VR application. This questionnaire evaluates several aspects of the perceived workload (Mental Demand, Physical Demand, Temporal Demand, Performance, Effort, and Frustration) on a 1-to-20 scale (with 1 corresponding to “Very low”, and 20 to “Very high”). This questionnaire is widely adopted to evaluate the cognitive load (Dolly et al., 2024; Alargkof et al., 2024), and has been specifically used in the assessment of VR training experiences (Wang et al., 2024; Solmaz et al., 2024).

• A section corresponding to the IMMS questionnaire (Keller, 2010), evaluating the participants’ motivation at learning the considered topics and consisting of 36 statements (to be scored on a 1-to-5 Likert scale from “Not true” to “Slightly true”, “Moderately true”, “Mostly true”, and “Very true”) organized in four categories (Attention, Relevance, Confidence, Satisfaction); this section was sent to the participants by email after the experiment. This questionnaire is widely adopted for the evaluation of learning experiences (Bond, 2020), particularly those involving immersive technologies (Low et al., 2022).

• A section including the questions of the System Usability Scale (SUS) (Brooke, 1996) and part of the questions of the VRUSE questionnaire (Kalawsky, 1999) (on fidelity, presence and satisfaction), estimating the overall usability of the devised VR application; like the previous one, this section was sent to the participants by email after the experiment. Both the questionnaires are widely used to assess the usability of various technological systems. On the one hand, the SUS is considered as a highly robust and versatile tool that can offer a quick and reliable usability rating (Aaron Bangor and Miller, 2008), and is often use to evaluate VR applications (Asadzadeh et al., 2024; Camara Machado et al., 2024; Ferris et al., 2021). On the other hand, the VRUSE can offer an in-depth analysis on the usability of a system and, although the authors do not report its validity properties, it is commonly used to obtain a comprehensive, heuristic evaluation of VR applications (Corelli et al., 2020; Julia Belger, 2023; Ferris et al., 2021).

• A final section collecting open feedback on the whole experience, with a focus on its positive and negative aspects.

Questions and items from the standard questionnaires used in this study (NASA-TLX, IMMS, SUS, and VRUSE) were not modified. Only an additional clarification was provided: whenever the word “experience” appeared in the original text, it was specified that it referred only to the VR-based experience that was the subject of the study.

It is worth highlighting that the participants of the CVR group (who were coupled during the VR part of the learning experience) were invited to take the theoretical quizzes and fill in the questionnaire individually.

Design

At the beginning of the experiment, the participants were asked to answer some preliminary questions regarding their experience with VR and with CL or teamwork, their knowledge of the laboratory procedure concerning the synthesis of biodiesel, and their attitude towards the experiment. Afterwards, all the participants went through a 30 min-long theoretical lecture on the biodiesel synthesis procedure, held by a chemistry professor from the authors’ university using multimedia material (slides and videos).

The participants were randomly assigned to two equally-sized groups (24 participants each) to avoid self-selection bias. The two groups were defined as follows:

• Single VR (SVR): the first group was composed of participants who experienced, in addition to the lecture, the application alone, i.e. with no external help (neither from the professor, nor from other students), using the headset.

• Collaborative VR (CVR): the second group was composed of participants who, after the lecture, were coupled and experienced the VR application in a collaborative, asymmetric way (i.e. switching roles after half of the experience, and using partly the VR interfaces and partly the desktop interface).

After the lecture, the first theoretical quiz (TQ1) was administered to all the participants to test their understanding up to that point. Afterwards, all the participants experienced the VR application according to their assigned group. Reports produced by the evaluation module of the application were collected, one report for each of the 24 participants in the SVR group and one report for each pair of participants in the SVR group.

Following the VR experience, the second theoretical quiz (TQ2) was administered to evaluate the knowledge gained from the experience. Also, the output of the evaluation module was collected for each participant of the SVR group and every pairs of the CVR group, in order to have additional information on the participants’ performance. Finally, the participants were invited to complete the questionnaire (Q) to assess various subjective aspects of the learning experience.

The complete design of the experiment is illustrated in Fig. 5. The lecture was the same for all the participants, and was designed making sure that all the contents relevant for the VR experience were provided.

Fig. 5

Design of the experiment

Ethical aspects

All the participants involved in the study were students regularly enrolled in the considered Chemistry course, in which both laboratory activities and the use experimental educational approaches are part of the curriculum. Informed content was collected for those who took part in the experimental evaluation.

Results

In this section, the results of the experimental evaluation will be analyzed, by considering first objective, then subjective results.

As said, the user study involved 46 participants. However, it is worth highlighting that the analysis for the sections of the questionnaire (Q) that were sent by email was based on a smaller sample, since only 31 participants (13 of the CVR, 18 of the SVR) returned their answers.

Data were analyzed using the Real-Statistics add-on of MS Excel. Comparisons were performed using the Mann–Whitney Test for Two Independent Samples to detect significant differences between the two groups (with a significant threshold $p \le 0.050$) where not stated otherwise.

Demographics

Based on preliminary questions, the participants had limited to no previous experience with VR ($M = 1.20$, $SD = 0.45$ on a 1-to-5 scale, with 1 corresponding to “Never used” and 5 to “I use it everyday”). The participants were also non-familiar with the biodiesel synthesis procedure at the beginning of the experiment ($M = 1.41$, $SD = 0.61$ on a 1-to-5 scale, with 1 corresponding to “Not familiar” and 5 to “Very familiar”), and were moderately used to CL and teamwork ($M = 3.26$, $SD = 0.74$ on a 1-to-5 scale, with 1 corresponding to “I never work in group” and 5 to “I always work in group”). Finally, all the participants were not anxious at the idea of taking part in the experiment ($M = 1.67$, $SD = 1.08$ on a 1-to-5 scale, with 1 corresponding to “Not anxious” and 5 to “Very anxious”). No significant differences between the SVR and CVR groups were found in these preliminary questions, suggesting that the compositions of the two groups were comparable under these perspectives.

Objective results

Objective results consisted of the answer provided to TQ1 and TQ2, shown in Fig. 6, and of the reports generated by the evaluation module, reported in Fig. 7.

Fig. 6

Knowledge gain of the theoretical quizzes; the $^{*}$ symbol indicates significant differences

Considering the results of TQ1 (that was administered after the lecture), no significant differences were found between the two groups ($M = 7.29$, $SD = 1.23$ for the CVR group, $M = 6.77$, $SD = 1.23$ for the SVR group, $p = 0.093$). Looking at the results of TQ2, however, a significant difference was found in favor of the CVR group ($M = 8.92$, $SD = 0.93$ for the CVR group, $M = 8.23$, $SD = 1.34$ for the SVR group, $p = 0.041$), with the participants who worked in pairs showing a larger knowledge gain with lower standard deviation compared to those who worked individually.

Fig. 7

Scores collected by the evaluation module; the $^{*}$ symbol indicates significant differences

By further analyzing the results of TQ2 and, specifically, by comparing the scores obtained by the participants of the CVR group who started the experience in VR (CVR-1) with those who started the experience using the desktop interface (CVR-2), no significant differences were found ($M = 8.83$, $SD = 0.90$ for the CVR-1 sub-group, $M = 8.91$, $SD = 0.95$ for the CVR-2 sub-group, $p = 0.932$). A TOST analysis performed on the results of TQ2 to constrast CVR-1 against CVR-2 showed that the 90% confidence interval was completely contained in the (– 1, 1) interval, indicating that the considered effects were equivalent (Fig. 8). The margin was set to – 10% and + 10% of the maximum score used to assess the quiz (a 0–10 integer scale).

Fig. 8

Representation of the 90% confidence interval of the comparison between CVR-1 and CVR-2 sub-groups, which is contained in the (– 1, 1) equivalence interval

By performing a Correlation Test using Spearman’s rho, it was also possible to highlight a significant positive correlation between the results of TQ2 obtained by the participants of CVR-1 and CVR-2 sub-groups (with rho $= 0.578$ and $p = 0.049$).

Moving to the results of the evaluation module, no significant differences were observed in the first phase evaluating the glassware selection and the assembly of the structure ($M = 91.15$, $SD = 7.11$ for the CVR group, $M = 91.75$, $SD = 5.54$ for the SVR group, $p = 0.483$). However, in the second phase, the participants of the CVR group performed significantly better than those of the SVR group ($M = 80.43$, $SD = 15.21$ for the CVR group, $M = 55.18$, $SD = 23.94$ for the SVR group, $p = 0.010$). Finally, in the third phase, the opposite trend was observed, with a significant difference in favor of the SVR group ($M = 66.62$, $SD = 21.91$ for the CVR group, $M = 84.20$, $SD = 15.32$ for the SVR group, $p = 0.049$).

Subjective results

For what it concerns the questionnaire delivered after the end of the experience (Q), considering the NASA-TLX results summarized in Fig. 9, no significant differences were found in the Overall scores ($M= 39.12$, $SD= 14.48$ for the CVR group, and $M= 46.97$, $SD= 13.05$ for the SVR group, $p=0.060$). However, looking at the single categories, it was possible to spot a significant difference for Mental Demand ($p = 0.030$), where the single-user interface used by the SVR group was rated significantly higher ($M = 49.09$ $SD = 21.35$) than the asymmetric interface used by the CVR group ($M = 39.16$, $SD = 16.81$).

Fig. 9

Results of the NASA TLX questionnaire on cognitive load using the application; the $^{*}$ symbol indicates significant differences

Considering the IMMS results (Fig. 10), no significant differences were observed over the four categories or overall between the CVR and the SVR groups.

Fig. 10

Results of the IMMS questionnaire on motivation at learning

Finally, to investigate the user experience with the application, the results of the SUS and of the VRUSE tools can be considered. Based on the overall SUS score ($M=78.54$, $SD=10.38$ for the CVR group, $M=75.00$, $SD=11.79$ for the SVR group, $p=0.209$), the application was considered as “good” by all the participants, without significant differences between the two groups. As for the VRUSE, no significant differences were found between the two groups for the three considered dimensions and, overall, scores were high for all the questions.

Discussion

The main goal of this study was to investigate the impact of asymmetric CL in VR-based learning scenarios, based both on objective measures (collected through theoretical quizzes and leveraging the evaluation module of the application) and subjective feedback (to evaluate the overall learning experience). The use of both CL and VR, in fact, is generally associated with a series of advantages, and their combination could be beneficial since the use of immersive technologies enables, and sometimes fosters, the creation of effective collaborative experiences (van der Meer et al., 2023).

The choice of implementing an asymmetric approach was made to overcome potential logistic limitations and cost issues that can be associated with the organization of large scale VR sessions involving numerous students, due to the space and number of headsets possibly required. Despite these potential advantages, the use of asymmetric approaches is still under-investigated, and can lead to surprising results. This approach puts the collaborating users on different levels, and this could undermine the inherent advantages of a collaborative scenario. Hence, the idea of comparing asymmetric CL against a more traditional approach, in order to evaluate if it can still be more effective than the use of VR in isolation.

Looking at the objective results and starting from the knowledge gain measured using the theoretical quizzes, it is possible to observe that the collaborative approach exhibited significant advantages over the individual one. Specifically, the CL approach was associated with a larger knowledge gain and a lower standard deviation, showing to be more effective in leveling the participants’ knowledge to higher levels. This outcome is probably due to the fact that VR technology, used in a traditional way, is usually advantageous for the acquisition of more hands-on, practical skills (Buttussi and Chittaro, 2021; Lovreglio et al., 2021), whereas it is less effective for learning theoretical knowledge (De Lorenzis et al., 2023; Calandra et al., 2022). In this study, collaboration between the participants seems to have compensated this limitation, and this outcome could be explained by the fact that, according to the literature, CL should be beneficial for the students, who can talk together and help each other (Zhao and Zhang, 2009), filling each other theoretical gaps and reaching higher learning performance (Laal and Ghodsi, 2012). It is worth noticing that the design of the study and, in particular, the organization of the collaborative VR experience, could potentially lead to unbalanced results, since the participants experienced the application in different ways (half of them started the experience in VR, the other half with the desktop interface). However, the results showed that the collaborative approach led to equivalent, good results for all the participants, also highlighting a significant positive correlation between the scores obtained in each pair. This outcome indicates that, regardless of the the role assumed by each of the two participants, the collaborative VR experience was able to fully capitalize on the benefits associated with CL. Moreover, it is also worth considering that, in many works such as the one by Makransky et al. (2019), it was shown that the advantages of VR can be undermined by the elevated cognitive load associated with an immersive experience; however, in this study, probably due to the use of an asymmetric approach, collaboration was associated with a significantly lower mental demand, meaning that the participants who worked in pairs may have been more favorable to the acquisition of theoretical notions.

Moving to the objective results produced by the evaluation module of the application and considering the first phase, it is possible to observe that no significant differences were found between the two approaches. As said, the evaluation module considered aspects like the correctness of the assembled structure and the number of errors made during the assembly. This phase was purely procedural, and required operations were not particularly complex; thus, the collaborative participants probably did not benefited from the help of the other team member (the assistant), and both the approaches were effective in the acquisition of the required practical skills.

Regarding the second phase, the evaluation module mainly assessed the correctness of the chosen reagents and of calculated amounts. This assessment was purely theoretical and, apparently, the collaborating participants were able to took advantage from the communication and sharing of knowledge allowed by the asymmetric mode; hence, the significantly better result. This finding confirms what was already indicated by the results of the theoretical quizzes, showing again that CL, even adopting an asymmetric approach, seems to be effective in overcoming the limitations of the traditional VR use, making it possible to acquire theoretical knowledge (probably thanks to the reduced mental demand experienced by the participants working in pairs).

However, in the third phase, where the evaluation module considered the yield of the reaction, the opposite trend was observed, with a significant difference in favor of the traditional, individual approach. This phase was basically practical, involving purely hands-on activities such as filling and emptying containers, or attaching and detaching 3D objects. Indeed, it has been shown already that VR is particularly effective when exploited in hands-on activities, where the procedural component is predominant. Based on these considerations, it can be hypothesized that the contribution offered by a collaborating approach and, specifically, the external help offered by the assistant, may not as be effective as in the previous phases, which were mostly theoretical.

A possible cause for this latter outcome could have been the handover (regarding both the interface and the tasks to be performed) between the two team members, since those who played the role of assistants in the third phase had just passed the headset to the other member, starting to use the desktop interface. Moreover, collaborating participants may have suffered from the fact that the team members who used the VR interface during this phase had just worn the headset and were not acclimatized to the virtual experience (Porter et al., 2020) (after having played the role of assistants for the majority of the experience); hence, they had less time to get accustomed to the virtual environment and its operation than the other team members (as said, this choice was made to guarantee that the duration of the experience for the two groups was the same, with the aim to investigate the possibility to serve twice the students in the same time with the same number of VR kits). In collaborative scenarios where the students cooperate and, especially, in asymmetric settings where only one student at a time is fully immersed in the virtual environment, benefits brought by VR appear to be reduced, since students have less control and mastery over the whole procedure. Students who experienced the VR application individually can instead experience all the steps of the procedure, getting more accustomed to it and, in particular to its repetitive, practical operations requesting a certain degree of dexterity (e.g., filling containers and mixing reagents).

Due to this consideration, it is possible that, in an asymmetric scenario with no handover, the results obtained by the collaborating participants, and specifically by the participant using immersive VR, could be comparable to those obtained by an individual participant. However, it is worth observing that, in a similar scenario, only one of the two collaborating participants would fully experience the immersive application, probably leading to a limited acquisition of practical skills since, as already shown in the literature, the use of desktop applications is less effective than immersive ones (Barrett et al., 2022; Freina and Canessa, 2015).

Finally, considering the subjective results on motivation at learning, no significant differences were found between the two approaches. Specifically, the results showed that the devised application was generally appreciated by all the participants and that, overall, it helped them to better understand the procedure. Since the two approaches were both VR-based and involved inexperienced students who were all enrolled in the considered Chemistry course, all of them probably perceived a comparable degree of motivation at learning. Moreover, considering the usability questionnaires, since all the participants used the same VR application, the similar results obtained by the two approaches were expected, and the desktop interface of the application was deemed on par with the immersive VR one. As previously stated, a significant difference was found in the results of the cognitive load questionnaire; specifically, the collaborating participants experienced significantly less mental load. This outcome, that had a positive effect on the acquisition of theoretical knowledge during the experience, was probably due to the fact that the collaborating participants were exposed to VR for half the time compared to the participants who worked individually.

Conclusions

This study investigates the effects of the use of CL in VR-based learning scenarios, focusing on how the combination of a team-based pedagogical approach and immersive technologies can affect the acquisition of theoretical concepts and practical abilities concerning a hands-on, laboratory procedure.

The investigation leveraged as a case study the procedure for the synthesis of biodiesel, a reaction which is part of the program of an advanced Chemistry course taught at the authors’ university. Since a real-life laboratory experience was lacking for this procedure due to inherent risks, a VR application was developed to virtually simulate the physical one. The students can use the application in two modalities: individually, or collaborating with another student. In the former case, the student trains on the whole procedure in immersive VR by wearing a headset and interacting with the hand controllers. In the latter case, an asymmetric interaction modality is used, with only one student at a time wearing the headset, while the other student can help him or her by observing the virtual environment on a PC screen and providing assistance using mouse and keyboard. The main findings of this work can be summarized as reported below.

• In terms of knowledge gain measured through theoretical quizzes, the asymmetric collaborative approach was found to be superior to the individual approach.

• Similarly, in the phase of the procedure where the students are asked to apply theoretical concepts, and no practical skills are involved (e.g, the selection of reagents), the collaborating pairs were able to take advantage of the communication and knowledge sharing possibilities enabled by the asymmetric CL mode, achieving better results.

• In the practical phase of the procedure (in which the selected reagents are used to perform the reaction), the individual approach yielded a significant advantage, probably due to the longer exposure to the immersive technology.

• The collaborative approach was associated with a significantly lower mental demand; consequently, the students who worked in pairs may have been more receptive to theoretical concepts.

In conclusion, combining VR with CL produces a pedagogical approach that is worth further exploration, as it can lead to good learning results when the delivered contents are theoretical; for more practical contents, however, it seems necessary to put all the students in the condition to directly experience more hands-on activities in VR to capitalize on the advantages of this technology. This is particularly true for asymmetric scenarios like the one considered in this work, where the objective is to involve a large number of students even in presence of logistical limitations (e.g., limited amount of VR kits and time). In other scenarios, in which the constraints to limit the number of VR kits used and the time of the experience can be removed, the impact of CL may be different. For instance, should it be possible to let both the team members fully immerse in the VR environment, level of communication between them could be higher than when using separate interfaces; nevertheless, none of the them would be able to directly perform the whole procedure, and the use of a VR interfaces for providing assistance may not be more effective that a desktop one.

Limitations and future works

For what it concerns limitations, this study, like many others in this domain, is based on a VR application that is very context-dependent. Specifically, the asymmetric approach and the handover were possible also due to the nature of the considered laboratory activity. Hence, further experiments are needed to confirm the applicability of the findings of this study to different use cases, since not every laboratory activity may be suitable for a collaborative approach or for adopting different interaction technologies (for instance, a desktop-based or a mobile-based interface could prove uneffective for a purely practical procedure).

Another limitation is related to the fact that, although the experience was designed to expose the collaborating participants to immersive VR for the same amount of time, the point where the handover is performed in the procedure, and therefore the phase where the actor becomes assistant and the assistant becomes actor, probably had an impact on performance. Specifically, the participant who started as an assistant and was not exposed to the application up to that point (as well as, based on preliminary questions, had generally limited experience with VR), took control during a phase that required elevated dexterity and ability with the devised virtual interactions. This fact influenced the final outcome of the more practical phase, leading to the lower performance of the collaborating participants. This effect could be smoothed, e.g., by making participants accustomed to the VR interface at the beginning of the experiment, leveraging a tutorial session that, however, would increase the total duration of the experience. In order not to alter the total duration of the laboratory activity, the same goal could be achieved, e.g., by using VR also for the introductory lecture. Moreover, it could be possible to find other solutions by moving the handover in a different point of the procedure, or by scheduling several handover points throughout the whole experience.

Regarding future works, other than addressing the aforementioned limitations, the application would probably benefit from the implementation of additional interaction means between the actor and the assistant, together with ways to support the handover. For instance, the assistant could be given the possibility to place visual hints (like arrows) or text messages in the virtual environment, making it probably easier for the actor to understand his or her suggestions. Speech recognition systems could also be used to replace mouse and keyboard inputs given by the assistant with voice commands, lowering the differences between using a 3D (VR-based) and a 2D (desktop-based) user interface. All these additions could improve the overall asymmetric CL experience, possibly leading to better objective and subjective results.

Other possible future directions of research could be to consider CL configurations with larger teams, potentially using additional technologies such as handheld devices (e.g., mobile phones) to further improve the availability of the educational application on affordable (and possibly student-owned) hardware. Moreover, a non-asymmetric CL mode where all the students have their own VR headset could be easily added to the application, and its effects compared with the other two modes. However, as said, this would introduce logistic limitations related to the number of VR kits required, and could possibly hinder the assistant experience, leading to possible problems when dealing laboratory procedures that are heavily based on practical, hands-on activities.