- Open Access
Toward a telepresence robot empowered smart lab
Smart Learning Environments volume 6, Article number: 5 (2019)
In the context of distance education, lab and field work are a considerable challenge for students, professors, and institutions. This paper presents a Telepresence Robot Empowered Smart Lab (TRESL) system as a new and emerging technology with a potential to overcome this challenge. The main focus of this paper is to propose a conceptual model, to define the ultimate goals of the proposed solution, and to present its system architecture and the modules to support the functions and features. A pilot project was conducted and is also presented in this paper. The pilot project includes the experimental implementation of a telepresence robot in a mock-up smart lab and the collection and analysis of survey data from users of the robot. The main findings of the pilot study are, 1) online users can easily operate the telepresence robot; and 2) operating the robot manually will increase the engagement of online users with the remote lab environment. The pilot study provides firm evidence that the proposed TRESL system is a promising and innovative approach for online students to conduct their lab work in a remote laboratory. The results also suggest avenues for further research and development of the system.
Online learning and distance education are playing an increasing role in today’s university education in this digital era. Through the Internet, most learning contents can easily be delivered online. However, science and engineering courses are more challenging since they usually require lab and field work components as well as hands-on experience for students to earn the full course credit. Lab and field work to enhance hands-on or first person experience remains a major challenge in today’s distance education. Currently, lab components in distance education institutions require students to travel to a university laboratory or to receive lab kits sent by the university. These solutions have their limitations because of high cost and time constraints. The performance and evaluation of lab work can be compromised if no supervision or tracking record is present. To address these challenges, computer generated virtual labs (Subramanian, & Marsic, 2001; Diwakar et al., 2014; Hallyburton et al., 2010), and Internet-accessible remote labs (Kist et al., 2013; Corter et al., 2007; Tatli & Ayas, 2013) have been introduced into online learning. Recently, innovative mobile delivery field work (mobile fieldtrip) for physical geography courses has been developed at Athabasca University (Pivot, 2015). Through a Telepresence Robot Empowered Smart Lab (TRESL) solution presented in this paper, students can use telepresence robots as their avatars in the remote laboratory, which will enable them to conduct lab or field work online as if they were present in the laboratory. It was expected that TRESL will increase online engagement with lab and field work and that it will enrich student learning experiences.
For the pilot study, we developed an affordable telepresence robot and a ubiquitous computing platform for remote labs. The prototype of the telepresence robot was built with a small budget, $400 CAD at Athabasca University (Denojean-Mairet, Tan, Pivot, & Ally, 2014). In addition, a two degrees of freedom arm permitted online users to do actual lab work remotely as if they were in the lab. The flexible design concept permits different configurations of the robot as needed. The successful development of the affordable telepresence robot enabled us to conduct several experiments, including driving the robot and manipulating the robot arm online to pick up simple objects (Denojean-Mairet, 2015).
In this paper, we will provide a literature review followed by the proposed solution of Telepresence Robot Empowered Smart Lab and its system architecture with a detailed description of each component of the system as well as the pedagogical issues. Furthermore, we will provide information about the experimental implementation of the TRESL system to demonstrate its feasibility for online learning. Finally, the results of our pilot study and directions for future research will be presented.
Related lab work solutions
A literature review revealed that research and development of lab solutions for online education fall into three interrelated categories: virtual, real and remote. An obvious challenge for students taking online courses in science and engineering is the requirement for lab work (Tan et al., 2016). Apart from requiring students to attend a designated lab often seen in traditional universities, other solutions include: demonstration labs; simulation labs; virtual labs; virtual reality labs (VR); and sending lab kits to students to do labs at home. As lab and field work are fundamental part of education in most scientific disciplines, the chapters on computer-based laboratory simulations, remote access laboratory, and enabling remote activity have been included in the book titled “Teaching science online: practical guidance for effective instruction and lab” (Kennepohl, 2016).
Among the aforementioned five types of lab solutions for online education, demonstration labs usually do not include learners’ involvement. Demonstration labs record operations in a physical lab to show how a lab work is done as well as all the details of lab outcomes, and send the recording to learners to watch, such as those at chemistry.about.com, where one may find a video demonstrates how a volcano erupts. In this category of lab delivery, not much research and development is involved other than the physical lab operation and a quality video recording. The advantage of demonstration labs is their easy delivery to learners. Students can easily access the demo videos or software through a web link. Demonstration labs may be accompanied by interactive communication between learners and instructors for questions or clarifications, while in the asynchronized learning model no instructor is included. In those cases, demonstration labs are less effective.
Simulation Labs require learners’ involvement, but not their direct handling of lab equipment and materials. Some examples of this type of simulation labs can be found at https://phet.colorado.edu/en/simulations/category/physics, and https://phet.colorado.edu/sims/html/balancing-act/latest/balancing-act_en.html, which provides a simulator to manipulate bricks to learn balancing acts in physics.
Virtual labs allow learners access to computer generated virtual labs through a computer interface. It is particularly useful for learners who lack access to a real lab, such as students in distance education. There are many virtual labs, such as genetic virtual labs at http://learn.genetics.utah.edu/content/labs/, the Physiology Virtual Laboratory at http://www.medicine.mcgill.ca/physio/vlab/, bio virtual labs http://www.mhhe.com/biosci/genbio/virtual_labs_2K8/, and Microsoft virtual labs athttps://www.microsoft.com/handsonlabs, and https://www.microsoft.com/en-us/evalcenter/my-virtual-labs. The Ministry of Human Resource Development (MHRD) of India has launched a large-scale initiative to develop and provide virtual labs to students and research scholars in many different disciplines, including Electronics & Communications, Computer Science & Engineering, Electrical Engineering, Mechanical Engineering, Chemical Engineering, Biotechnology and Biomedical Engineering, Civil Engineering, Physical Sciences, Chemical Sciences (India Vlabs, 2019). LabViIEW is a real-time software system and another example of using virtual labs as part of a distance learning strategy. It provides easy access to lab equipment anytime and anywhere over the Internet (Fridman & Mahajan, 2014).
Virtual labs can be seen as an early stage or simple form of virtual reality labs. While reality is not the goal of virtual lab implementation, virtual reality labs (VR) are an effort to give learners the impression of being in a real lab environment and doing the lab work. Examples of such VR lab include the virtual human interaction lab at https://vhil.stanford.edu/, the Virtual Reality Design Lab (VRDL) at https://vr.design.umn.edu/ by the University of Minnesota, the VR Labs for various training at http://www.buffalo.edu/ubnow/stories/2016/12/virtual-reality-lab.html by the University of Buffalo, the Virtual Reality and Perception Laboratory at http://percept.eecs.yorku.ca/, and many others. Although some of the VR labs are not designed for distance education learners, they can be used by learners who don’t have the convenience to be in the ‘real world’, such as the deep or far ocean, the inside of the human brain or other organs or body parts. The previously mentioned lab solutions can all be part of so-called online labs in today’s online education. In addition to the efforts already mentioned above, a large collection of online labs can be found at https://www.golabz.eu/labs.
For learners, one major disadvantage of the lab solutions mentioned above is the absence of real hands-on experience regarding the handling of lab equipment, devices and materials. This may be the reason why many courses have adopted a home lab kit solution, which asks students to do labs at home with the home lab kits specified by the course.
Courses using home lab kits for learners in distance education include BIOL230 offered at Athabasca University (Biol230, 2019), and System Dynamics and Controls Courses offered at the University of Minnesota (Abimbola, 2016). Home labs are the most frequently reported lab solutions in the literature (Atiq, Chenet al., 2015; Durfee, Li, & Waletzko, 2004; Jouaneh & Palm, 2011; Oliver & Haim, 2009). Home lab kits are widely adopted for courses in different subjects because there are many home-lab-kits vendors in the market, such as, Hands-On Labs (holscience,2019), eScience Labs (eScience, 2019), CISCO (CISCO, 2019), and those sold at Amazon and PricefindCanada.
Although doing lab experiments at home with a lab kit may give learners the opportunity to obtain real world experience of working with lab devices, materials, and practicing real lab processes, the types and scales of labs that can be done at home are limited due to their complexity or because of safety concerns. For many reasons, it is obvious that not all labs can be done at home. A different approach to providing learners with real lab experience is remote access to laboratories, i.e. remote labs (Cooper & Ferreira, 2009). So far, the biggest effort in this direction has led the development of The OpenScience Laboratory at the Open University of the UK (British Open, 2019). The essential idea of remote labs is to provide students with remote access to real lab facilities and services through the Internet, enabled through smart sensors. Another solution is the online lab, for example the one provided at https://www.golabz.eu/labs.
A recent paper presents a comprehensive review of contemporary virtual and remote laboratory implementations. It further confirmed that remote labs, virtual labs, and simulation labs provide great accessibility, flexibility, and reusability. The labs can contribute to higher learning outcomes and richer learning experience when they are incorporated with the appropriate pedagogical framework and learning support (Alkhaldi et al., 2016).
To provide a condensed overview of lab solutions that may be adopted for distance education, we summarize our review in Table 1. From the table, it is hard to tell if one or more are superior to the others. Ma and Nickerson (Ma & Nickerson, 2006) have carried out a comparative review among hands-on, simulated and remote labs without firm conclusions. In practice, the choice of the lab depends on the learning objectives, the lab work that is required for the course, and the availability of the lab.
The solution concept and system architecture
In this section, we introduce the solution concept and the system architecture of the Telepresence Robot Empowered Smart Lab (TRESL). In Method of solution development section, we will elaborate how and where the solution comes from. The solution concept and its ultimate goals section, presents the conceptual model of the solution and the ultimate goals. The TRESL system architecture section introduces the system architecture with a detailed description of each system component. The system service modules section focuses on the system service modules that are core part of this proposed smart lab system. Finally, in System pedagogical issues section, we will deal with the system pedagogical issues, and discuss how the TRESL system is going to be used by online students and how it can support the design of lab and field work.
Method of solution development
This research project is a response to the requirement of remote labs in distance education. For our science course teaching we have developed various methods to deal with the requirement of lab and field work, however there are several constraints, including high costs. The development of robotics, computing technologies, and the Internet provides options to introduce more effective and innovative methodologies for our online students. With video conference software applications, we can engage our students through synchronized audio and video interaction more efficiently than before. First, an affordable simple telepresence robot for studying the concept of driving robots online was built. This was followed by a mock-up lab to test how easily online users can drive a telepresence robot and complete simple actions, such as picking up objects and dropping them into a bowl. Based on the previous research and development, we propose the remote lab solution: The Telepresence Robot Empowered Smart Lab (TRESL) in this paper.
The solution concept and its ultimate goals
The presented remote lab solution in the form of the TRESL system is an example of the integration of Emerging Information Communication Technologies. Telepresence robots are avatars of the students, which provide them first-person experience at the remote laboratory. Online students are tightly integrated via the telepresence robots, the lab devices and equipment, and the lab environment through the Internet forming a Cyber-Physical System (CPS). In addition, through online driving, navigating, and manipulating the robots in the remote laboratory, students are included and engaged into the Supervisory Control and Data Acquisition (SCADA) system for controlling the telepresence robots in real time. Telepresence robots embedded with computation units, sensors, and actuators are also considered as the Uniquely Addressable Heterogeneous Electronics (UAHE) components. They will be interconnected with other UAHEs, such as the lab devices and equipment, and sensors in the remote laboratory through the Wireless Sensors Network (WSN) in order to create the mesh network of the Internet of Things (IoT).
The conceptual model of the solution, the Telepresence Robot Empowered Smart Lab (TRESL) is shown in Fig. 1 In the remote laboratory, computer servers serve for various features and functionalities of the smart lab system including system control, learning application, data analytics as well as security and privacy settings. On the client site, any computer with a web browser can access the TRESL system web portal or alternatively client software applications may support more sophisticated human-machine devices, such as a VR hand-mounted display, a joystick, or a nature language processing device. In the diagram, the arrows show the interaction with or without data flow among the system components. Furthermore, the TRESL system supports multiple telepresence robots acting or even interacting at the same time in the smart laboratory. The interaction among the robots will happen in many ways. Any robot in the TRESL system could play an instant and temporary role to other robots as an ad hoc wireless communication node, as a leader or a follower in the “follow mode”, as an avatar to interact with other avatar(s), as a moving object or obstacle to other robots in the traffic, etc. The multiple telepresence robots’ scenario is not included in the conceptual model diagram.
Within the TRESL system, a student can sign onto a telepresence robot in the remote laboratory and the robot becomes associated as the avatar of the student. The student will be able to remotely present her/himself inside the laboratory through her/his avatar (telepresence robot). Furthermore, she/he will be able to conduct lab work and interact with the lab environment. The student will be directly connected to the robot to extend her/his sensing, communicating, and mechanical capabilities to the remote laboratory or the field in the case of field work. The ultimate goals of the telepresence robot empowered smart lab solution is to enable online students to:
“Be There”, i.e. experience as if they were present in the remote lab, and “Act There”, i.e. extend their interactive capabilities, including sensing, communicating, and mechanical capabilities to do lab work and to interact with the lab environment.
The TRESL system architecture
The system architecture of the TRESL system is shown in Fig. 2 The architecture represents a typical cyber-physical system (CPS), which involves human controlling and monitoring a remote physical component, i.e. a telepresence robot through the Internet. The whole system includes a human operator, a communication link, and a telepresence robot, as well as computer servers, which are integrated in the cyberspace. The telepresence robot is composed of computing, control, sensing, and mechanical elements. The computer servers associated with databases are running different modules to support the functionalities and the features of the TRESL system.
From the top-down view of the system architecture diagram, the System Users, or human operators include online students, instructors, and a system administrator. An online student will be able to access the system through various possible human-computer interfaces. Through the interface, the student will log onto the TRESL system and also associate with a telepresence robot in the lab. In order to control the telepresence robot, the student will be engaged in the cyber-physical system’s manual closed loop control to form the Internet based SCADA control system as a human supervisor, while the associated robot plays the role of the remote terminal unit (RTU). The programmable logic controllers are running on the embedded computer of the robot. When a student drives the robot online, the system can be considered as a robot teleoperation system with the Internet as the communication link, in which the master is the human-machine interactive device on the client site and the slave is the associated telepresence robot at the remote laboratory. System control will be guided by the control module running on the control server shown in Fig. 2 In the smart lab system, the robot will turn into the student’s avatar to enable the student to walk in the laboratory, sense and interact with the lab environment, and conduct the assigned lab work or other actions. The instructors would have a similar capability and functionality, while they can also provide students with real-time supervision when they are on the TRESL system. As in any web-based application system, the system administrator is responsible for system maintenance and database administration. A web portal and a database interface will be provided for the system administrator to access the system with the highest access privilege.
On the client site, the Human Computing Interface includes both hardware and software. The hardware could be a computer with mouse and keyboard or a mobile device, a Virtual Reality (VR) hand-mounted display, a goggle, a joystick, a headset, a microphone, or any other human machine interactive device. Using virtual reality goggles for controlling a mobile robot can be a powerful virtualization and teleoperation tool for supervision (Bug, 2014). The software is the Human Computer Interface Module, which is running on the client site computer to support the human-computer interaction. It could be as simple as a web portal on a web browser, and/or a client site application that should be developed or installed and integrated as a part of the TRESL system. This way of human-machine interaction on a client site will greatly impact the system users, i.e. online students, on their experience of “being there” and their capability of “interacting there”. In terms of using the TRESL system, there is a trade-off between telepresence experience and convenience. The more sophisticated and immersive HCI devices and technologies are, the better the telepresence experience will be. The essential minimum requirement is that any online student will be able to access the TRESL system with a computer or even a mobile device. Nature language processing is a promising technology for HCI that could provide guidance for online students to send the command to control the telepresence robot, to access the instruction, and to create the report.
The Communication Link between the client site and the TRESL is the Internet. The system will provide two ways of access through the Internet. The first one is via direct access by using the URL of the access service running on the secure server computer at the remote lab, and another way is through indirect access with cloud services. Regardless of access type, the speed and bandwidth of the Internet will always be the critical factor for system performance. The system design, such as system settings, configuration, and specification, will depend on the available communication links, from 3-5G and Wi-Fi wireless networks to high-speed optical networks in order to reach optimal user experience.
The Data Repository is to store, process, and manipulate the data in the system. It should be properly chosen in order to meet the needs of the different system functionalities and features, especially for Big Data Analytics. The MYSQL database will be used to handle the structured data for user management and learning contents management, such as user information and course information. One NoSQL database will be used to deal with the semi-structured or non-structured data for Big Data Analytics, such as Unified Communication (UC) data and multimedia data. The databases will be collocated with their server applications in the laboratory. The databases will include the backup process to store the data in the co-location facility and the Cloud. A web portal will be provided for database administration.
The Smart Lab Server is the computer(s) running the service modules in support of the features and functions of the TRESL system at the remote laboratory. The computer service modules include the Security Module, the Control Module, the Application Module, and the Analytics Module. A detailed description of the computer service modules will be given in The system service modules section.
The Internet of Things (IoT) refers to the uniquely addressable heterogeneous electronics (UAHE) that can collect and exchange data based on the existing Internet infrastructure covering a variety of protocols, domains, and applications, which are interconnected. In the remote laboratory, telepresence robot(s) as the components in the system are considered as the most important part of the Internet of Things. Other UAHEs in the remote laboratory include sensors, actuators, and appliances, such as IP cameras, laser sensors, a heating ventilation and air conditioning system, as well as the lab devices and equipment, such as IP enabled microscopes, a DNA analysis device, and computers. They will be integrated and connected through the wireless sensors network to provide the platform of the TRESL system, and they contribute largely to build the controlled and smart environment and to enable online users to conduct lab work through the TRESL system.
On the remote laboratory site of the TRESL system, the main focus is on lab work, in which the laboratory is a controlled or smart environment. Multiple telepresence robots will be placed in the laboratory to allow online students to simultaneously sign onto their own robots, to remotely present themselves in the laboratory, and to conduct their lab work through their robotic avatars. The detailed description how to use the TRESL system will be given in System pedagogical issues subsection later. The laboratory will be fully equipped with various sensors to create the wireless sensor networked smart environment. The WSN provides sensing and communication links to support the control, navigation, and monitoring of the telepresence robots inside the laboratory, which creates the mesh network of the Internet of Things (IoT). The system service modules running on the computer servers reside in the laboratory to provide the TRESL system functionalities and features. The system data flow diagram is shown in Fig. 3 which gives an overview of how the data flow (represented by the thin arrow lines) among the components of the TRESL system. Furthermore, the TRESL system can also be designed to be used for conducting field work for science courses, such as physical geography courses. Online students could remotely participate in field trips or conduct field work, depending on the degree of sophistication of the telepresence robots.
The system service modules
On the system architecture side, the Smart Lab Server is composed of a security server, a control server, an application server, and an analytics server. Each server will have one service module running on it. The servers will also work collaboratively to provide services to the TRESL system.
The Security Module runs on the security server at the remote laboratory. Within the security module the access point is the first contact point of the local network from/to the Internet. User authentication and authorization will take place. The access point ensures that all traffic entering the TRESL system is traffic for authenticated users and that traffic for authenticated users will be redirected only to the right server or the system components. The security module should ensure the confidentiality and integrity of the communication and data exchanged during the application access. It will also monitor the traffic, ensure non-repudiation and log the actions for system analysis and audit. The system user management function will be residing in the security module, which will have information security, such as data encryption and user privacy protection features integrated in the design. The security server will be accessed via a reverse proxy or network address translation (NAT).
The Control Module plays the key role in the TRESL system to control the telepresence robot and the smart lab environment. In terms of controlling the robot, there are two main tasks, driving the robot in the laboratory and manipulating the robot’s hands to do the lab work. To remotely drive the telepresence robot, there are three control modes proposed in order for online users to control the robot in a more effective manner. To achieve the ultimate experience of telepresence through the robot, the TRESL system allows online users to drive their robots in autonomous mode, manual mode, and leading mode and to easily switch among the control modes through their user interface.
In the autonomous mode, the robot will automatically move from point A to point B. As the online user being a human supervisor, the robot control system could be considered as an Internet based SCADA control system in which the associated robot plays the role of remote terminal unit (RTU). The user only needs to set the destination point B, and then to monitor the robot automatically moving to the destination. In this case, the user has little effort and engagement with the robot motion other than monitoring and watching. The robot is controlled and navigated by its own control unit with support of the smart environment in the laboratory.
In the manual mode, online user remotely drive the robot through the Internet and the wireless network in the laboratory. The user will control the robot’s motion, speed and direction based on the navigation map of the laboratory. The control commands are sent through an HCI device, such as a keyboard, a mouse, a VR goggle, a joystick, or a NLP device that is supported by the human-computer interface module. The feedback from the robot and the TRESL environment is provided to the user in real time, and could be in the form of video, audio, other visual means, sound or touch feedback. In this mode, the online user plays the part of the closed loop real-time control system.
In the follow mode, the online user does not directly operate the robot. The robotic avatar of the online user follows a leading transmitter carried by an associated person or a collaborating robot in the remote laboratory. The transmitter should be adjustable for the distance and position between the leading person or robot on site and the following robotic avatar.
The control module provides the core services to the TRESL system, which includes robot localization, path planning, navigation guidance, and lab traffic control, as well as integrating the unique addressable lab devices and equipment to form the local mesh network of the Internet of Things.
The Application Module runs on the application server to provide application functionalities and features to the TRESL system. The TRESL system as an immersive learning system is essential to support learning. Thus, a learning management system (LMS) provides web portals to instructors and students. Through the instructor’s portal, instructors can create, store, and manage the lab and field work instructions or guides, grade students’ works, answer students’ questions, and discuss the labs or field work. A web portal allows students to obtain their lab or field work instructions or guides, submit the lab or field work reports, ask questions to their instructors, receive feedback from their instructors, and participate in discussions. The LMS will be implemented with the 5R adaptation framework (Tan et al. 2016) to provide the students the 5R adaptive contents for their lab or field work. The application module also provides unified communication features to support the communication for the online users who are working within the TRESL system. In order to enable the immersive lab or fieldwork contents, the application module offers a platform to run simulation and visualization software, provides server services to VR, AR, and MR applications, play videos, and run multimedia instructions or demos. When the TRESL system provides users as a cloud service, LaaS (Lab as a Service), the application module will also include the functionalities and feature as a Cloud Service Provider. In terms of architecture, the application module is open and flexible to easily add additional applications, functions, and features to the TRESL system.
The Analytics Module is dedicated to the Big Data Analytics (BDA) running on the analytics server. BDA is used to extract useful information that can be used to enhance online students’ learning performance and to improve the TRESL system through learning analytics and system function analytics based on the data collected from the system implementation, user interaction with the system, and the communications among the users. Through BDA, the TRESL system is able to analyze these data and gain insights into the users’ behaviors and trends on the system and to eventually provide effective smart lab services to the users and to enlarge the TRESL system capability. The module consists of selected BDA software engines implementing various BDA methods and algorithms. Hadoop is a good candidate because it is an open source framework with MapReduce. It is effective in the analysis of vast datasets in distributed environments by converting data into HDFS to create clusters of data for analysis. Apache Spark uses similar analytics as Hadoop but employs RDDs data sharing abstraction in the analysis of data, which has allowed the engine to perform more tasks including graph processing, machine learning, and streaming. Apache Storm is designed for the analysis of streaming data that are gathered from social media. Its engine is the most preferred in the analysis of unbounded data streams. Apache Flink is capable of analyzing both streaming and historical data (Abimbola, 2016).
The Human Computer Interface Module is installed as the client site application software of the TRESL system and runs on the client site computer to enable online students or users to interact with the TRESL system with the immersive interaction devices or equipment, such as Head-Mounted Display, VR goggle, joystick, and natural language processing devices. The human interaction module is to enhance the online users’ telepresence experiences, which is not a mandatory component of the TRESL system but by using the module supported it will largely engage online users into the TRESL system to improve their experiences of conducting their lab or field work online. For immersive HCI, this module includes necessary server applications for supporting the state-of-art HCI features.
System pedagogical issues
The purpose of the TRESL system is to provide a remote lab solution for students to conduct their lab work or field work online. The pedagogical support is important for the students to effectively use the system to achieve their educational goals. Pedagogical support for virtual and remote labs has been addressed in a recently published paper (Nedungadi et al., 2018).
Using the TRESL system requires proper design of the lab or field work based on the capacity of the robot avatar.
Telepresence robots are the key component of in the TRESL system. The system’s capability will largely depend on how dexterous the robots are and how much the robots can be manipulated by the online users. Thus, the TRESL system will be more like a framework or platform that can easily integrate various types of robots to work as avatars in order to meet the lab or field work needs in terms of dexterity or manipulability. Furthermore, the inclusion of educational and pedagogical considerations when designing lab or field work in the TRESL system should also greatly enhance the system’s usability. Educational and pedagogical improvements and refining of the TRESL system for lab or field work is a crucial aspect of the research and development for the TRESL system application.
The TRESL system is designed to be used in the following three ways: Independent, Collaborative, and Blended. The Independent way of using the TRESL system applies when an individual student uses the TRESL system to conduct her/his own lab work alone. In this case, the student can access the system 24 X 7 online. When multiple students, regardless where their geographical locations are, sign on the TRESL system at the same time and collaborate through their robotic avatars to complete lab work together, the Collaborative option of using the TRESL system applies. If at least one student is physically present in the laboratory to work together with at least one robotic avatar towards the same lab or field work, we consider this option as the Blended way of using the TRESL system. The word “blended” here means human and robotic avatars are blended together as a group. This case is useful in situations when the lab or field work is required as a group activity and some of group members can cannot be on site. In such a case, the robotic avatars’ control mode usually should be set as the follow mode to enable online members participate in the lab or field work in a synchronous manner as if they were working shoulder to shoulder in the laboratory or field by just simply following their peers on site. This blended way of using the TRESL system would be helpful for many applications of group field work.
Furthermore, in consideration of control mode, in most cases of lab work, students have to come back to the remote lab multiple times, which makes them familiar with the laboratory environment even if they only access the smart lab online. Manually driving (manual mode) the robot could become inefficient and boring to the students. Therefore, autonomous driving (autonomous mode) should be the most effective and efficient mode for students to focus on the lab work rather than on driving the robot. In addition, the TRESL will also implement personalized services because the robot avatar will automatically move to the correct lab table or spot when it is associated with a student as the TRESL system knows the progress of the student’s lab or field works.
Experimental implementation and pilot study of the TRESL system
In order to test the TRESL solution, a mock-up smart laboratory and a simple experiment were designed and implemented. A telepresence robot was used for the mock-up remote laboratory. The robot was developed as a ubiquitous computing platform (Denojean-Mairet et al., 2014) to demonstrate that an affordable telepresence robot can be applied in a remote lab setting. This experimental implementation was conducted to study the telepresence robot’s usability and interactivity with a web-based user interface. This experiment also helped us to optimize the design of the robot and to discover barriers of using the robot in the smart lab, enabling us to further study the telepresence robot empowered smart lab for online education. The focus of the experiment was on the use of the telepresence robot by geographically dispersed participants that were connected to the telepresence robot via the Internet. In this section, the setup of the experimental implementation and the results of the pilot study are presented.
Experimental implementation setup
The main components of this experimental implementation were the telepresence robot, a lab camera and a STUN/TURN server. The lab camera system is composed of a HD webcam and a Raspberry Pi computer running a video steaming server to support smart lab environments. The Raspberry Pi can also be used to acquire data from sensors in the laboratory and stream the data to the user. With a Bluetooth dongle, the Raspberry Pi can be connected to any Bluetooth sensors. Figure 4 shows the experimental implementation system architecture.
A human operator, i.e. a pilot of the online telepresence robot guides a closed-loop control system via the Internet communication link. The pilot controls the robot by sending commands through the control and communication process and by receiving feedback from the cameras. The feedback originates from three cameras, a lab-mounted camera and two onboard robot cameras. With the stream video feedback, the pilot can adjust the telepresence robot’s movements in real-time. Figure 5 shows the closed-loop control system.
The telepresence robot uses WebRTC (Web Real-Time Communication) technology for video conferencing. WebRTC is implemented in browsers such as Chrome and Firefox. Online users connect to the robot via a URL. One main advantage of using WebRTC is that it can handle multi-party calls. As a result, users could share the experience of driving the telepresence robot by through different actions, e.g. one user could control the arm and gripper and the other user could control the movement of the robot. Each user would see the same video feeds from the robot.
The pilot interface is composed of three main views. The head camera view is the central view on the interface. Some simple graphics are used to provide an augmented reality of the scene. The graphics are a ruler and a line scale which help the pilot in judging the size of the objects and the distance from these objects. More information could be displayed on this view such as temperature, humidity, lighting conditions, etc. The other two views are secondary views and are used to provide additional spatial references to the pilot. The secondary view on the top right of the interface is the lab-mounted camera view. A wide-angle lens is used to provide over 180 degrees of view. This gives a “bird-view” of the mock-up laboratory and it is very easy for the pilot to see where the robot is in relation to the lab table, for instance. The other secondary view is the down facing camera view. The field of view of this camera is closer to the body of the robot and can help in judging distances between the robot body and an obstacle. These two secondary views can be turned off by the pilot if they are too distractive. The telepresence robot and the mock-up laboratory setting is shown in Fig. 6.
The robot’s motions are controlled via the keyboard. The arrow keys are used to move the telepresence robot forward, backward, left and right. Since the telepresence robot uses differential steering, pressing the right or left arrow key will make the telepresence robot turn clockwise or counter clockwise. The W, A, S, and Z keys are used to pan and tilt the head camera. The N and M keys are used to close and open the gripper. The < and > keys are used to move the arm up and down. The telepresence robot has two speeds. The normal speed is used to drive the telepresence robot around the environment and it matches the average slow walking speed of a person. The slow speed is used to move the telepresence very precisely and it is best suited when grabbing and moving objects on the lab table. To toggle the speeds, the pilot can just press the 1 or 2 keys at any time.
The pilot study and its findings
To validate this approach, a test drive of the telepresence robot was followed by an exploratory survey. The test drive entailed operating the telepresence robot in a remote mock-up laboratory. We tried not to focus on any age group and educational background in order to emulate the diversity of the online learners. This pilot study is to get a general idea about driving the telepresence robot and manipulate the robotic arm online. 40 participants from different nationalities including Canada, China, Egypt, France and Taiwan were instructed to drive the robot to a lab table and use the robot’s arm and gripper to move two small objects into a container. The two small objects were located on the table close to the container. The participants’ dexterity task consisted in picking up each object and deposit them into the container within twenty minutes. After the test, the participants were asked to answer a survey with 21 questions (See Additional file 1). Among the forty participants, 36 completed the tasks successfully, and 33 answered the survey questionnaires and 14 of them provided additional comments. 9 females and 25 males from aged 18 to 64 answered the survey. The methodology of the survey is described (Denojean-Mairet, 2015).
After analysing the survey and observing the driving test, we found that our simple affordable telepresence robot can be easily driven and used to manipulate objects by online users who did not receive prior training. Most of the participants of the driving test were able to successfully complete the test by moving the two objects into the container. Overall, the survey revealed that most of participants had a positive experience when driving the telepresence robot and performing the dexterity task. Very few participants rated their experience negatively.
Nominal and ordinal scales were used in the questionnaire. A nominal scale was used to collect descriptive information about the participants such as gender and age group. An ordinal scale was used to learn about the attitudes and opinions of the participants toward the use of telepresence robots. In the ordinal scale, the ranking of the value is important but the difference between values is not known. As a result, no means are calculated on these ordinal scale questions. Instead, the Median and Mode is used to describe the central tendencies. The scale for the ordinal questions is from 1 to 5 with 1 being the worst and 5 being the best. In this five-point scale, 3 is the neutral variable. In addition, for analysis categories were aggregated. For instance, “agree” and “strongly agree” were collapsed under “agree”. The same is done with “strongly disagree” and “disagree” changed to “disagree”. The aggregation was done to provide a simpler picture of the participants’ attitude toward a specific question.
The answers of the two questions are considered as the most important and interesting findings related to the proposed remote lab solutions:
Question 1 (Question 6 in the survey): How easy was it to navigate the robot within the remote lab?
Question 2 (Question 16 in the survey): Do you think that the robot should move more autonomously?
Most respondents agree that it was easy to navigate the robot within the lab (Median = 4, Mode = 4, N = 21, 63.63%). Only 18.18% (N = 6) of the respondents found it difficult to navigate the robot within the lab. Furthermore, only 6 respondents selected 3 (neutral) for their answers (18.18%). From the comments, we see that a lag from the main video feed was a major problem. The lag was probably due to the fact that three HD cameras were streaming real-time videos on a 2.5 Mbps upload bandwidth. In addition, when considering another answer from one of the questions, among the 33 persons who successfully completed the test, only 2 persons had driven the telepresence robot before. This finding supports the idea that telepresence robots are easy to drive online (Figs. 7 and 8).
Regarding Question 2, many respondents (N = 12, 36.36%) disagreed but a roughly equal number (N = 9, 27.27%) agreed. Moreover, 36.36% of the participants chose neutral as their answers (Median = 3, Mode = 3, N = 12). These results are reflected in the comment section where a few respondents indicated that some automation could be desirable, while others stated that automation would negatively impact user experience. This result contradicts the assumption that a fully autonomous robot which moves from point A to Point B would be more user friendly for online operators. It suggests that manually driving the robot can make users more engaged with the remote environment, enjoy the whole experience, and it supports the “manual mode” design in the control module. Obviously, this finding does not lead a conclusion that manually driving a telepresence robot from its base station to its destination position, such as a lab table, is an effective and efficient way in a real educational scenario.
In this paper, our focus is on proposing the remote lab idea and solution: Telepresence Robot Empowered Smart Lab and presenting TRESL system. Therefore, we only present two important findings here. The finding from Question 1 convinces that it is easy to navigate the robot in the remote laboratory online, which is the fundamental concern of the proposed TRESL solution; and the finding from Question 2 provides an evidence that “manual mode” can help users to engage with the remote laboratory, which enhances their experience as if they were present in the remote lab. The detailed descriptive analysis about all the 21 questions can be found in the thesis (Denojean-Mairet, 2015).
With the growing popularity of online education, the challenges of conducting lab or field work for online students have to be addressed. Based on our experience with existing lab solutions and combined with our research, technological knowledge, and teaching experiences in related technological fields, we propose the remote lab solution: Telepresence Robot Empowered Smart Lab (TRESL) in this paper, which is to tackle the challenges of lab and field work in distance education using robotics and emerging computing and information technologies. The proposed solution is expected to provide online students with a much higher level of engagement regarding their lab or field work, to enrich their online learning experience, and to enhance their learning performance. The conceptual model of the proposed solution illuminates how the telepresence robot and the emerging technologies can empower the smart lab and eventually enable the online students to conduct their lab or field work as if they were present in the remote laboratory. The ultimate goals of the remote lab solution have been described as: “Be There”, i.e. experience as if they were present in the remote lab, and “Act There”, i.e. extend their interactive capabilities, including sensing, communicating, and mechanical capabilities to do lab work and to interact with the lab environment.
A system architecture has been proposed in this paper. The architecture has included various emerging technologies to support the cutting-edge remote lab solution. The detailed description of the architecture and the system modules provides the information of how the solution works and how the functionalities and features are to support online students to use the TRESL system. The three modes of driving the telepresence robot and three ways of using TRESL system are presented in this paper to address the key technological aspects of the TRESL system and the pedagogical issues of the remote lab solution.
A pilot study was conducted through the experimental implementation of a mock-up system. Forty voluntary participants from several countries were invited to drive the telepresence robot online and to answer the survey questionnaire. From the collected, processed, and analyzed survey data, we present the two most important and interesting findings of the pilot study in this paper. The results of the pilot study were encouraging and supported the idea that the TRESL solution is feasible for online students because telepresence robots are easy to drive and manipulate online.
However, the pilot study has its limitation and it doesn’t address many technological and pedagogical issues of the proposed remote lab solution. Future research will include many technological issues, such as using Machine Learning and Artificial Intelligence to provide solutions for autonomous robots’ path planning, navigation, as well as traffic control in smart laboratory. The future efforts of research and development of the TRESL system will involve the creation of the TRESL framework to integrate various telepresence robots and IoT sensors to meet the different needs of lab or field work in terms of educational and pedagogical requirement and advancement, as well as the fully functional and featured prototype system. Through using the fully developed TRESL system at Athabasca University for its online courses, the TRESL solution related educational and pedagogical issues can be further studied, for example, how embracing IoT with telepresence robot can leverage smart lab services and effect the end-to-end interactions between users and remote lab services; how integrating the pedagogy with the distribution of the cyber physical lab operations can impact on students’ intrinsic motivation of doing lab or field works.
B. Abimbola, The relationship between unified communications (UC) and big data analytics, MScIS essay, Athabasca University, December 2016 (2016)
T. Alkhaldi, I. Pranata, R.I.J. Athauda, A review of contemporary virtual and remote laboratory implementations: observations and findings. J Comput Educ 3, 329 (2016). https://doi.org/10.1007/s40692-016-0068-z
S. Z. Atiq, X. Chen, D. D. Cox, J. DeBoer. (2015). International STEM classrooms: the experiences of students around the world using physical remote laboratory kits paper presented at 2015 ASEE international forum, Seattle, Washington. https://peer.asee.org/17146
D. Bug. (2014). Oculus rift control of a mobile robot: providing a 3D virtual reality visualization for teleoperation or how to enter a robots mind, Master’s Thesis at School of Computer Science and Communication, KTH Royal Institute of Technology, Sweden, 2014, Retrieved from http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A754499&dswid=-3354
M. Cooper, M.M.J. Ferreira, Remote laboratories extending access to science and engineering curricular. IEEE Trans Learn Technol 2(4), 2009 (2009)
J.E. Corter, J.V. Nickerson, S.K. Esche, C. Chassapis, S. Im, J. Ma, Constructing reality: A study of Remote, Hands-On, and Simulated Laboratories. ACM Tran Comput-Hum Interaction 14(2), 7–es (2007). https://doi.org/10.1145/1275511.1275513
M. Denojean-Mairet, Q. Tan, F.C. Pivot, M. Ally, A ubiquitous computing platform affordable telepresence robot design and applications. Published on the (13th IEEE international Conference on Ubiquitous Computing and Communications, Dec. 19-21, 2014, Chengdu, 2014)
M. Denojean-Mairet. (2015). Telepresence Robot Enable Remote Lab in Distance Education, MScIS thesis, Athabasca University, December 2015. https://dt.athabascau.ca/jspui/bitstream/10791/189/3/Thesis_final.pdf
S. Diwakar, H. Parasuram, C. Medini, R. Raman, P. Nedungadi, E. Wiertelak, B. Nair, Complementing neurophysiology education for developing countries via cost-effective virtual labs: Case studies and classroom scenarios. J Undergrad Neurosci Educ 12(2), A130–A139 (2014)
W. Durfee, P. Li, D. Waletzko, in The Proceedings of the 2004 IEEE American Control Conference. Take-home lab kits for system dynamics and controls courses, vol 2 (2004), pp. 1319–1322
E. Fridman, S.H. Mahajan, Heat Transfer Virtual Lab for Students and Engineers: Theory a Guide for Setting up (Momentum Press, New York, 2014), p. 2014
"C. L. Hallyburton, E. Lunsford, S. C. College, C. Drive, S. Nc. (2010). Challenges and opportunities for learning biology in distance-based settings. Bioscene: J Coll Biol Teach, v39 n1 p27-33
M.K. Jouaneh, W.J. Palm, in 2011IEEE Frontiers in Education Conference (FIE). Control system experiments at home (2011) Pp. T2G-1
K.D. Kennepohl, Teaching science online:parctical guidance for effective instruction and lab work, 1st edn. (Stylus Publishing, Sterling, 2016)
A. Kist, P. Gibbings, A.D. Maxwell, H. Jolly, Supporting remote laboratory activities at an institutional level. Int J Online Eng (iJOE) 9(S5), 38 (2013). https://doi.org/10.3991/ijoe.v9iS5.2772
J. Ma, V. J. Nickerson. (2006). Hands-on, Simulated, and Remote Laboratories: A Comparative Literature Review, ACM Computing Surveys, Vol. 3, No. 3, Article 7, 2006
P. Nedungadi, M.V. Ramesh, P. Pradeep, R. Raman, in Cyber-Physical Laboratories in Engineering and Science Education, ed. by M. Auer, A. Azad, A. Edwards, T. de Jong. Pedagogical support for collaborative development of virtual and remote labs: Amrita VLCAP (Springer, Cham, 2018), pp. 219–240
P. Oliver, F. Haim, Lab at home: Hardware kits for a digital design lab. IEEE Trans Educ. 52(1), 46–51 (2009)
F. C. Pivot. (2015). Mobile -Technology Guided Fieldtrips for Distance Education in Geography, project proposal for Campus Alberta’s Open Education Resource Initiative. April 2015
R. Subramanian, I. Marsic. (2001). ViBE : Virtual Biology Experiments, 316–325. 2001
Q. Tan, X.K. Zhang, F.C. Pivot, J. Evans, Kinshuk, R. McGreal, The 5R Adaptation Framework: Concepts, Systems, and Learning Scenarios. J Internet Technol 17(5), 971–980 (2016)
Z. Tatli, A. Ayas. (2013). Effect of a Virtual Chemistry Laboratory on Students Achievement, Educational Technology & Society, 16(1), 159–170
We like to thank all the participants who took part in the experimental implementation and answered the questionnaire during the pilot study. We also like to thank Dr. Lisa Carter, Dean of Faculty of Science and Technology, Athabasca University for her support to this research.
Availability of data and materials
Ethics approval and consent to participate
Athabasca University Research Ethics Board reviewed the research and approved on ethical grounds on February 17, 2015.
The authors declare that they have no competing interests
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Tan, Q., Denojean-Mairet, M., Wang, H. et al. Toward a telepresence robot empowered smart lab. Smart Learn. Environ. 6, 5 (2019). https://doi.org/10.1186/s40561-019-0084-3
- Telepresence robot
- Smart lab
- Remote lab
- Cyber-physical system
- Online learning