Evaluation of enhanced educational experiences using interactive videos and web technologies: pedagogical and architectural considerations
Smart Learning Environments volume 3, Article number: 6 (2016)
Nowadays, teachers and students utilize different ICT devices for conducting innovative and educational activities from anywhere at any time. The enactment of these activities relies on robust communication and computational infrastructures used for supporting technological devices enabling better accessibility to educational resources and pedagogical scaffolds, wherever and whenever necessary. In this paper, we present EDU.Tube: an interactive environment that relies on web and mobile solutions offered to teachers and students for authoring and incorporating educational interactions at specific moments along the time line of occasional YouTube video-clips. The teachers and students could later experience these authored artefacts while interacting from their stationary or mobile devices. We describe our efforts related to the design, deployment and evaluation of an educational activity supported by the EDU.Tube environment. Furthermore, we illustrate the specific teachers’ and students’ efforts practiced along the different phases of this educational activity. The evaluation of this activity and results are presented, followed by a discussion of these findings, as well as some recommendations for future research efforts further elaborating on EDU.Tube’s aspects in relation to learning analytics.
Over the past decade, mobile technologies have been used for supporting social interactions in different domains of our daily lives (Goggin 2012). Nowadays, mobiles are used to support different types of social activities conducted across different settings and for various purposes including education (Kravcik et al. 2004; Thompson 2013). Specifically, mobile technologies offer support for educational interactions practiced along pedagogical strategies conducted anywhere and anytime (Parsons 2014; Sarrab et al. 2012). These technologies can be conveniently used by teachers and students to access educational content in forms of rich and interactive media available in the web (Frantzis et al. 2012; Wang and Dey 2013).
Recently, teachers have started to discover and to exploit these affordances and offer them to their students requiring better and more appealing forms of educational experiences (Kurkovsky 2012). Furthermore, they encourage students to use their own devices while aiming to enrich their educational experiences with, deeper and more interactive media (Cochrane and Bateman 2011). For example, teachers use rich media to scaffold their students’ educational process while being on field-trips (Giemza et al. 2011). In other occasions, students use their mobiles to interact with rich media at their free time when being in places located beyond the boundaries of school (Jacob and Issac 2014). These examples rely on complex but open technological infrastructures enabling vast utilization while using diverse types of end user devices owned by teachers and students (Godhwani 2013; Mohammad and Tomberg 2013).
The deployment of educational scenarios supported by rich media and web & mobile technologies rely on efforts practiced by various types of stakeholders including educational designers and technological developers (Zbick et al. 2014). From the technological point of view, developers should consider mobile infrastructures as well as compatibility and adaptability of a wide-range of devices (Heikkinen and Still 2008; Kukulska-Hulme 2007). From the pedagogical point of view, educational designers aim to create activities that are adapted and later experienced on mobile devices. The design of these activities aim to create experiences that could be practiced anywhere and anytime by individuals or groups of students (Kearney et al 2012).
Accordingly, the development and deployment of educational scenarios relying on these type of technologies, require to cope with challenges related to coordination and cooperation among stakeholders addressing pedagogical and technical aspects (Saifudin et al. 2012). Technological wise, these challenges could be addressed and alleviated while practicing programming techniques that incorporate libraries and modules used to support mobile development. These implementation efforts should address challenges related to the wide variety of hardware and software settings relevant for mobile development. Furthermore, these efforts should also consider computational and network abilities that may impact performance and usability aspects. In addition these aspects are related to mobile applications aimed for educational purposes possibly used across contexts (Sotsenko et al. 2013). Last but not least, development efforts should also address the wide variety of technical specifications concerning display resolutions and sizes (Marcotte 2010; Giemza and Jansen 2011).
As already implied, such type of tasks exercised by stakeholders practicing technological deployments should be considered and coordinated with other efforts dealing with pedagogical design (Petrova and Li 2009). This coordination of tasks is suggested as a key aspect while aiming to offer teachers and students with a more facilitated use of mobile technologies (Boticki et al. 2013).
In this paper, we present our efforts related to the development of EDU.Tube as well as its exploitation for designing and deploying educational activities containing rich and interactive media usable also on mobile devices (Kohen-Vacs et al. 2014). In the next section we offer a description of the EDU.Tube environment. This environment is proposed for teachers and students interested in authoring educational interactions aimed to be incorporated along the timeline of occasional video-clips, e.g., found in YouTube. The outcomes of such efforts could later be used as educational opportunities offered to be incorporated in different types of pedagogical strategies. Next, we describe our efforts related to the design of a pedagogical scenario supported by EDU.Tube and conducted with 75 students attending courses at the bachelor level at two different academic institutions in two countries. The description of the scenario is followed by another section dealing with our evaluation efforts aimed to reveal pedagogical and technological challenges identified by teachers and students. We conclude the paper by presenting our future efforts discussing how to improve EDU.Tube and its educational affordances. In addition, we also present challenges and drawbacks encountered in the methodology used to evaluate the mentioned activity. We share this information as part of our report aiming on refinement of the methodologies we used. In addition, we considered that the reported experience could be valuable for other researchers that aim to conduct future experiments in similar settings.
Accordingly, we expect to further refine the tool, the activities used with it and the methodologies to evaluate them both. These future efforts are aimed in order to offer more meaningful and appealing educational experiences enabled with additional functionalities including new aspects related to learning analytics.
Description of the environment and development efforts
As mentioned earlier, the EDU.Tube environment aims to offer support for the authoring of educational interactions to be incorporated into occasional videos found in YouTube. One of the main aims of our development efforts was to enable teachers and student to author, use and share interactive and educational opportunities combined with rich media.
EDU.Tube enables an authoring process that could be initiated by teachers or students seeking to use occasional video-clip with potentials to be used as educational opportunities. As soon as authors find a suitable multimedia resource, they may incorporate educational interactions along its timeline. An author may choose to incorporate various types of interactions including open questions as well as questions dealing with single or multiple selections. In addition, an author may choose to incorporate a static announcement used for providing information along the playback of a video-clip. Figure 1 illustrates the portal to the authoring environment as well as an examples dealing with authoring of single selection question equipped with customized feedback.
As implied, EDU.Tube’s artefacts can be used and shared on stationary computers, laptops, smartphones or tablets. These artefacts can be utilized while using different communication channels like regular mail, social networks or a Learning Management Systems (LMS). The authoring process was designed to be short, effective and user-friendly while aiming to alleviate technical challenges possibly encountered along the authoring process.
In the following sub-sections, we will describe some of the most salient concerns and aspects addressed during the deployment of EDU.Tube. First, we identify the stakeholders (teachers and students) and their goals used to catalyze this development and deployment efforts. Next, we present our developmental approach. Specifically, we illustrate aspects of the environment’s data structure and control used to support the mentioned type of activities. This illustration is followed by another section presenting EDU.Tube’s control. Last but not least, we deal also with interface related aspects enabling teachers and students to experience EDU.Tube on different devices.
EDU.Tube’s stakeholders and goals
The identification of stakeholders and the discovery of their common or different goals is considered as a primary requirement addressed in any development effort (Alexander and Beus-Dukic 2009). In this subsection, we deal with the identification of stakeholders involved in the deployment of EDU.Tube as well as with their goals.
In these efforts, we involved two types of stakeholders expressing some joined but also different set of goals. Educational stakeholders expressed their goals reflecting their aspirations to enrich their common pedagogical practices. They also aimed to design innovative activities that are technologically supported and could be experienced always. In addition, they intended to seek on an innovative approach that could be implemented across domains. Developers’ efforts were catalyzed from technical related goals that aimed to best realize the previous mentioned goals. Specifically, these goals concerned data modeling, programming and graphical design practiced while developing different aspects of EDU.Tube.
Stakeholders’ efforts were consolidated into a coherent set of specifications used to catalyze EDU.Tube’s development. In addition, it should be mentioned that the involved stakeholders dealt with different challenges according to their experience related to technological development used later to support pedagogical deployment. Educational stakeholders were involved during the deployment efforts of EDU.Tube. For example, Teachers were involved in the process while expressing their requirements concerning pedagogical approaches that could be enriched by interactive video-clips accessible from anywhere at any time. Developers working at different institutions (at HIT and LNU) were required to cope with different aspects of the development, including the design of EDU.Tube’s data structure, business layer and user interfaces. Some of these developers were involved in providing technical support for teachers and students while using EDU.Tube.
Design pattern supported development
In the previous section, we described EDU.Tube’s goals as expressed by different stakeholders. In this sub-section we present our development approach supported by a design pattern identified by us as suitable for our project settings. Design patterns are a development approach aiming to offer an effective mean to cope with given challenges that were described in an abstract way and could be reused across different situations and similar cases (Schmidt et al. 2013). We used the Model View Controller (MVC) design pattern to support our developmental efforts. MVC is a classical pattern enabling separation of concerns (SoC) between the different aspects of projects. For this case, SoC was used to facilitate development and deployment efforts practiced by different stakeholders including teachers and programmers addressing different aspects and concerns in our project. Specifically, SoC consists of data model, business logic supported by controllers and visual aspects provided by user-interfaces (Reenskaug 1979).
SoC could offer convenient support for interactive environments used for educational purposes and developed by heterogeneous type of stakeholders (Jansen et al. 2004). Figure 2 illustrates the pre-developmental phase followed to the actual development supported by MVC. In addition it shows the later deployment used by teachers and students in real educational settings.
Figure 2, describes a developmental phase that is iteratively assessed by stakeholders considering how the environment meets their set of goals. The deployment efforts for this environment were practiced in different versions of EDU.Tube deployed for use in real educational settings while considering that development complies with the mentioned goals. Nowadays, more than 1200 users dealing with a variety of subject matters use EDU.Tube for authoring activities in different domains including Math, Physics and History. They constantly contribute with new insights and recommendations on how to improve the environment.
EDU.Tube’s data structure and control
One of the main aspects of EDU.Tube’s development concerns its data structure aiming to provide support for services like authoring, experiencing and sharing of educational and interactive video-clips. EDU.Tubes’ services are incorporated and exploited for various types of educational activities. During our efforts, we aimed to integrate tasks authored in EDU.Tube with other learning activities supported by other technological environments. For example, interactive video-clips authored in EDU.Tube could be experienced on regular or mobile playback system available in the environment. In other cases we conducted peer assessments combining EDU.Tube with another environments called CeLS, an authoring tool enabling to design of collaborative learning activities (Ronen and Kohen-Vacs 2010). In this case, the authoring efforts from EDU.Tube were shared and incorporated for later educational interactions conducted in CeLS.
Our development efforts included the consideration of requirements dealing on how to share EDU.Tube’s data with other environments. We accordingly put a special emphasize on the modeling of EDU.Tube’s data structure. The modeling aims to reflect the nature of the videos and their corresponding interactions focusing on sharing and reuse while using other technological environments. EDU.Tube’s Data model contains a main class, including various global types of properties. In addition, it also consists of specific information related to the actual interactive video activity stored in an XML based data format as illustrated in Table 1.
The mentioned information includes a unique ID key used in order to distinctively identify an EDU.Tube activity. In addition, it contains a publish property used to indicate if the activity is ready to be used. We also annotate other information including the data in which the activity was created, its title, a description and its set of instructions for use. The following information contains a link to the video-clip followed by a description of the educational interactions incorporated to the video. Next, we describe the unique author’s identity of the mentioned interactions followed by a description of the language in which this interactions are communicated.
Another feature of EDU.Tube concerns its control implemented in its business logic. Users requesting its services for experiencing video-clips activate the environments’ control. This request is delivered with data that may influence the information that is returned. During the process, users send their requests with data reflecting the language that they prefer to use and the type of the device from which they are going to experience the video. Specifically, users may consume videos on two main type of technologies: stationary computers and laptop computers (later referred as regular computers), smartphones and tablets (later referred as mobile devices). In addition, the sent data also includes a unique identity of the interactive video that they request. As a result, EDU.Tube sends the information adapted to the language characteristics and to the technical specifications for the device that is going to be used for video playback.
EDU.Tube’s user interfaces
EDU.Tube’s interactions were designed with the central idea of being responsive in order to optimize the user experience both with stationary and mobile devices. Figure 3 illustrates the EDU.Tube interaction, as it would be experienced in regular computers.
This kind of interactions could be also required from EDU.Tube’s control for experiencing videos on mobile devices. In such cases the environment is required to cope with a series of challenges related to the nature of mobile devices. First, we aimed to cope with challenges related to the presentation of media on various types of displays possibly characterized by different sizes and resolutions. Another challenge is related to the experience itself. In such cases, when experiencing videos on smartphones and tablets, the mobile player will stop the video and present the corresponding interaction. However, the video will not start automatically following to the users’ response to the interaction. This behavior has its point of departure from security restrictions set by the operating systems on mobile devices. One additional challenge concerns the amount of information that is relevant and required to be displayed for the users at the same time.
We coped with this challenge while splitting the presentation on mobile devices into two separate states as illustrated in Fig. 4 for the case of an Android based device.
The first state is displayed as long as the video is playing. The second state hides the video and displays the interaction while such point is arrived. As mentioned the video on mobile devices will not restart automatically. Accordingly, continuation of videos requires users’ confirmation. In the next section, we present a pedagogical scenario implemented with CeLS environment.
In this section we present an actual implementation of EDU.Tube used along a learning activity implemented in 3 of university courses. The learning activity described in this section was designed for undergraduate students learning essential terms in the field of computer science. Specifically, this activity was practiced by 75 students attending bachelor degree courses at two institutions located in different countries. The presented activity was practiced along three phases conducted during two weeks. The activity was technologically supported by the EDU.Tube authoring environment, enabling students to incorporate occasional video-clips found in YouTube with educational interactions. In addition, the activity included other orchestrated interactions related to the ones from EDU.Tube that were supported by another environment called Collaborative e-Learning Structures (CeLS) already discussed in the previous section. The activity, its sequenced phases and tasks are illustrated in Fig. 5.
The activity starts with a phase in which students are required to seek for video-clips with lengths between three and five minutes. Students were instructed to seek for videos with educational potentials that may assist with teaching of concepts related to computer science according to the subjects as dealt in different courses (Gilroy 2010).
Students from each course were required to seek for different kind of videos. Those attending the software engineering course were instructed to seek for videos related to modeling. Students attending the basic programming course were instructed to seek for videos dealing with different aspects of sorting algorithms. Finally, students attending the advanced course were required to seek for videos dealing with client-server model. In addition, students were required to author educational interactions and to incorporate them at moments that would transform these YouTube videos into interactive and educational opportunities. The outcomes of the authoring efforts are stored in the database of the EDU.Tube environment. In the following phase, students assessed seven interactive video-clips authored by their peers studying with them in the same group. Throughout this phase, we aimed to evaluate an increased number of interactive video-clips. These video-clips are included in repositories and are offered for teachers and students as more appealing educational scaffold to be used in future lessons. Students conducted their assessments while using regular or mobile instances of EDU.Tube and using various types of devices owned by them.
In the next phase of the activity, students are required to select the three best videos while pointing to their preferred selection (without further ranking among the three best). In addition they are also required to encompass their selection with textual justification. The results of this voting and the justifications are also stored in the database of the CeLS environment.
The final phase of this activity is taking place during a debriefing session. In this session, teachers use CeLS and EDU.Tube to present the selected (mostly voted) videos to the students. In addition, teachers also present the students’ insights expressed by their fellow students during their peer-assessments. The most selected artefacts are used in the debriefing session as educational and appealing opportunities that were recognized as pedagogically contributions by both the teachers and the students.
The design and deployment efforts addressing this scenario were exploited by and evaluated by teachers and students practicing it in real settings (Kohen-Vacs et al 2013). The results of these efforts are detailed in the next section.
In this section, we present the results of the evaluation of the learning activity that was carried out during the spring semester of 2013. This evaluation was conducted with 75 students participating in an collaborative learning activity supported by EDU.Tube and CeLS. The data we collected was gathered using a questionnaire accessible via a web site that addressed aspects related to the technology used by students. The questionnaire consisted of 12 questions divided into 5 categories addressing different aspects including availability of computing devices. The next category addressed the purpose and locations in which students use their own devices. In the following category, we addressed the pattern in which devices are being used. The next category directly deals with aspects related to regular and mobile instances of EDU.Tube. Last but not least, students could openly expressed their insights and suggestions based on their experiences while using EDU.Tube.
The evaluation was conducted with an adapted form of the Technology Acceptance Model (TAM) questionnaire pointing out different technological and pedagogical aspects addressed in the activity in which EDU.Tube instances were used (Davis et al. 1989). It should be mention that originally, TAM questionnaires were validated with various traditional technologies including e-mail, voice mail, word processing, and spreadsheets (Lederer et al. 2000). Possible Challenges and drawbacks concerning adaptations of TAM for web and mobile devices were acknowledged by us as well as by other group of researchers (Huang et al. 2007). The questionnaire used in this evaluation represents our attempt for adapting TAM questionnaires for mobile learning. This attempt is further discussed later in the section dealing with conclusions and future work.
We calculated the Cronbach’s Alphas for this questionnaire in terms of the two items addressing the contribution of the used technology to the learning process (α =0.65). In addition, Cronbach’s Alphas this was checked with items addressing the contribution of regular and mobile technologies (α =0.69). The questionnaire included a section enabling students to express the ICT technologies they preferred to use including regular computers (stationary or laptops) as well as mobile devices (smartphones and tablets). In addition, the students could select that they do not have a specific preference (3 categories in total). The next items of the questionnaire enabled the students to express their perceived experiences through five categories addressing different levels of user friendliness.
In the following sub-section, we offer a more profound analysis of the data aimed to identify trends and patterns of use related to experiencing interactive video-clips on mobile devices.
Presentation of students and their used devices
The activity described in the previous section was conducted with regular and mobile technologies owned by the students. Table 2 described the used rate of owned technologies across participants of the 3 courses.
The table addresses the topics dealt in various courses, their academic level and their affiliation. In addition, the table also addresses the kind of devices available to the students and their use for experiencing EDU.Tube. The details described in the table were collected from students answering the first section of the questionnaire addressing the variance of technological devices, particularly owned by the students.
We checked the availability of stationary, laptops, tablets and smartphones among the students. Accordingly, we found that 42 students have a stationary computer, while almost all of the students reported to have a laptop computer. In addition, 62 students reported to have a smartphone device while a minority of them (17) reported having a tablet device. In the majority of the cases, the students have both one regular computer (stationary or laptop) and a mobile one (smartphone or tablet). In this respect, we found that 47 students reported to have both a laptop and a smartphone, resulting in a number of students potentially using both devices.
In the next section of the questionnaire we aimed to examine the contexts in which the students used EDU.Tube instances with their own devices. Specifically, the students were required to report about places and situations in which they used EDU.Tube. This included locations such as their homes, university, and work places or while using public transportation. In addition, we also examined the variety of technologies that was used in each context. Table 3 summarizes the nature and diversity of technologies used to experience EDU.Tube across locations.
In addition it should also be considered that the quoted numbers in the table represent the percentage of ownership among all the students across the 3 groups, explaining that the different percentages presented in the table do not sum to 100 %.
The questionnaire includes sections addressing those issues exploring the patterns in which the students used the technology across locations. Almost all the students reported to use their laptop while studying. A closer look at the data revealed that the students attending the software engineering course used almost all their available technologies. In the case of the introductory course, we found that only 22 % of the class attendees used smartphones. We also checked the use of technology among the students attending the server side programming course and found that 50 % of the attendees used smartphones. This data may indicate a more extensive exploitation of available technologies among participants of the software engineering course compared to the other two.
The next questions specifically addressed the EDU.Tube environment and its perceived added value related to the activity in which it was used. We started by posing questions that addressed EDU.Tube in general without particularly mentioning its regular or mobile instances. We found that 70 % of the students reported that the EDU.Tube’s activities presented by the teacher following each lecture helped them to some extent, or more to deepen their understanding of the learning materials. A closer look at this data reveals that only a minority of 29 % of the students felt that the activity was much or even very much helpful.
We then asked the students to specify their preferred instance of EDU.Tube. We found that 69 % of the students reported that they would prefer using EDU.Tube on their laptop devices while only 28 % of them reported to have no specific preference. Almost none of the students preferred to use the environment exclusively on his/her mobile device.
In the next questions, we requested the students to specify the reasons why they preferred the regular or the mobile instance of EDU.Tube. Most of the students reported that the regular version of EDU.Tube is user friendly while almost all of them reported that the mobile instance and its interface is difficult to use. We then closely examined the specific reasons for which they indicated their preferences and impressions. About 50 % of the students attending the courses dealing with procedural programing and server side programming pointed out that there is prominent challenge while using small-scale devices for interaction combined with video-clips. About 20 % of the participants attending the course dealing with software engineering mentioned the same. Some of the students expressed that there is a need to re-arrange and improve the adaptation of EDU.Tube content for small-scale displays. In addition, there were also some students that raised comments referring to the nature of the interactive video mechanism by themselves. In this respect, they mentioned that for some situations they prefer a continuous video experience without the disruption typically involved in interactive videos. Furthermore, in some of the cases the students even mentioned that they require watching the entire video-clip in one sequence in order to better understand the ideas it aims to communicate. Therefore, the students recommended to watch the video-clips in 2 rounds while the 1st one is performed in a complete sequence and the second one includes the incorporated interactions.
Another concern of the students involved the requirement for meaningful feedback and accreditation of the activity. They wondered if the results from the interactions were accumulated to be used for recommending adapted learning pathways according to the interactions previously practiced. Finally, there was a small minority of students that reported operational issues with mobile phones like problems connecting to the network, lack of compatibility to some of the smartphone operating systems, and some operational bugs in the environment’s mechanism.
The last part of the questionnaire required the students to openly state their comments including aspects of the environment that they thought required further improvement, and other aspects they already found satisfactory. In this section, the students refined all points for improvements which they had previously provided. They reaffirmed their desire to keep using an improved version of EDU.Tube. Specifically, they reported that they liked the pedagogical approaches encompassed with interactive videos provided by a relatively user-friendly environment.
Analysis for patterns of use
In the following steps of our analysis, we proceeded our examination with some deeper statistical elaboration of the results. For this analysis, we used questions addressing various aspects including patterns in which the students use their available devices, their perceived added values from EDU.Tube and their preference of technologies while using it. Before analyzing these questions, we formed groups of users according to the type of used devices. This grouping was performed in order to enable an analysis of the students’ opinions and insights according to the technology they preferred to use. Here, the grouping was performed according students preferring to use their regular devices as well as with the ones preferring to use their mobile devices. The answers to this question revealed that 27 students used regular computers as well as mobiles for studying purposes. In addition, 32 students reported to use only regular computers for studying purposes. About one percent solely used the mobile instance of the environment. In a further examination of the results, we examined answers provided by the students while filling their insights as open text. In these results, students expressed the reasons for such selection is not necessarily related to EDU.Tube itself but to other aspects associated with content selection and adaptation.
In other words, some of the students stated that it would be helpful if the authors of the video-clips would carefully select the YouTube video while bearing in mind the possibilities to consume them across different type of devices. Specifically, they mentioned that the rich media displayed in these video-clips could be consumed in a small size display, typically existing in smartphones and tablet devices. Some of the students even mentioned the possibility to use YouTube features in order to deliver the media in various qualities adapted to the technological capabilities of the used network, and to the unique technical specifications of the devices in which the interactive video-clips are experienced. In the next steps of the analysis we kept using the groups of students according to their technological preferences. This analysis was conducted on answers provided by students to various questions addressing different aspects of EDU.Tube’s including recommendation for its future use. In addition, the questions also addressed its perceived contributions for the courses as well as its perceived affordances to pedagogically assist the students. Finally, we also check aspects related to the interface of the environment. The analysis of these questions was performed while using independent t-tests.
Would you recommend to other teachers to use similar video based EDU.Tube activities?
This question was followed by options from which students had to select one: yes (1), depends (2), and no (3). The options related to this and to further questions were presented in a form displaying a list of options, encapsulating values as mentioned in the parentheses below. We found that the average value was 1.38 and standard deviation of 0.65, expressing a clear recommendation of this approach by the students. Furthermore, there was a significant difference between the stationary group (M = 1.52, SD = 0.83) and the mobile group (M = 1.33, SD = 0.71); t(51) = 2.02, p = 0.049. This shows that the number of students that would recommend teachers to use similar EDU.Tube activities is significantly different in terms of the mobile group being more in favor for the suggested technological solution. Furthermore, the use of EDU.Tube on a mobile device is perceived by the students as an added value for the educational scenario and not as something exclusively for such devices. This evaluation may emphasize the need for an additional and improved mobile version of EDU.Tube enabling teachers and students to better exploit the mobile instance of this environment.
Video based activities like the ones performed with Edu.Tube can contribute to learning in university courses
This statement addressed EDU.Tube and its pedagogical potentials perceived by the students. For this statement, students had to select one of the options from the following: very much (4), much (3), to some extent (2), and not at all (1). This question revealed a coherent view towards the benefits provided by interactive and video based activities with respect to learning in university courses. Among the complete group, the average value for this question calculates to 2.72. A detailed analysis of the two groups, stationary (M = 2.73, SD = 0.71) and mobile (M = 2.68, SD = 0.87) did not show significant differences; t(51) = 0.51, p = 0.61. Analysis of the data has shown that the design of the available options could be improved towards a less biased formulation while using refined Likert scale.
EDU.Tube activities, presented by the teacher after each lecture helped me repeat and deepen my understanding of the learning content?
This question was followed by the optional answers that stated the following: very much (5), much (4), to some extent (3), a little (2) and not at all (1). The results from this question in which the average calculates to 3.07, shows that the students believe that interactive and video based content helps them improving and deepening the understanding of the learning content. However, a significant difference was revealed in the mobile group (M = 2.73, SD = 1.01) and the stationary group (M = 3.22, SD = 0.98); t(50) = 3.52, p = 0.001. This leads to the question whether the presented approach might be better suited for introducing new learning materials instead of a repetition of already discussed learning content. This dilemma will be discussed in the concluding section, as it is an issue for our future work. As already mentioned in the previous subsection, the analysis of the data has also shown here that the design of the options could be improved towards a less biased formulation.
The Regular/Mobile EDU.Tube interface is…
This statement was presented in order to conduct an analysis aiming to compare two questions addressing interface related aspects existing in the regular as well as in the mobile instances of EDU.Tube. The students were asked to rank each of the environments while having to select one of the options: simple and user friendly (1), not so easy to use (2), quite difficult to use (3) and difficult to use (4). The options posed to the students were similar for both types of instances. Interestingly, the outcome showed that the stationary user interface was rated simple and user friendly (M = 1.1, SD = 0.3), while for the mobile user interface, the students testified that they were not able to evaluate it (M = 1.88, SD = 0.86). Only two students from the group preferring to use the mobile UI evaluated the mobile user interface as slightly more difficult to use in comparison to the stationary one. This may imply a need for a different kind of evaluation of the usability of the mobile user interface. However, the difference in the evaluation capacities of the students is significant; t(72) = 20.927, p = 0.01.
The analyses presented in this section indicates that the students are aware of challenges related to authoring, adaptation and use of EDU.Tube’s artefacts on the mobile instance, while still recognizing the added values and potentials.
In addition to the statistical analysis, we also conducted open interviews with four teachers involved in the enactment of the previously discussed activities. The teachers provided their impressions and suggestions related to educational and logistical aspects of the activities. They were generally satisfied with the potential benefits of EDU.Tube. They mentioned that this facilitated authoring environment enabled them to easily create, share, incorporate and use interactive video-clips in their common educational programs. Furthermore, they mentioned the great potentials of incorporating their students’ artefact authoring in EDU.Tube in their educational practice. However, they also expressed some concerns related to different aspects addressing the use of EDU.Tube. Firstly, they were concerned about the enactment of complex pedagogical activities depending on the support of several interrelated technologies. Secondly, they were apprehensive about competing with the interactive videos for their students’ attention during the lessons.
The teachers also shared their impressions with regard to the two versions of EDU.Tube (stationary and mobile). They recognized the added value of both the regular and the mobile versions, but pointed out the friendliness of the regular version. Furthermore, they mentioned that the regular environment enables an easy access to the interactive and rich media resources. They reported that video-clips experienced on EDU.Tube’s stationary version are more convenient for them to use. In addition, they also addressed the mobile instance and pointed out its affordances to enable their students to experience such interactions anywhere and anytime. Furthermore, they addressed some challenges to be considered for future development of the mobile version, e.g., issues related to the considerable diversity of mobile technologies owned by their students, which is much greater than with laptops or regular computers. They mentioned this variety in terms of the complexity which can cause the need for a dedicated operational guidance for each unique type of device. In addition, they expressed similar feedbacks as commented by the students expressing difficulties to use the mobile version while using small scale screens. The teachers also claimed that the way in which the mobile version presents the interactions is too challenging, as it does not simultaneously display the video and its corresponding interaction. Despite the mentioned challenges concerning the mobile version, the teachers still believed that these challenges could be coped with and be resolved.
Another topic treated in the interviews concerned the various technologies used to support the different phases of the activity. Teachers referred to interoperability issues among the technological environments enabling sharing, and experiencing EDU.Tube’s artefacts along with their pedagogical strategies. They perceived and mentioned the added values gained from the use of these various interoperable technologies for supporting different educational interactions practiced along the proceedings of this activity.
In the next section, we sum-up our efforts and suggest some salient points and recommendations dealing with different aspects and conditions related to the design, development and implementation of educational activities supported by EDU.Tube. We will also examine the possible impact of these recommendations in relation to future versions of the tool that may include learning analytics features and additional interactive elements.
Conclusions and future work
In the previous sections, we presented EDU.Tube and its affordances offered to teachers and students to support their authoring and use of interactive videos. The environment was introduced in terms of its functionalities including its authoring abilities. We described the process and outcomes of an activity supported by EDU.Tube. We mentioned, that during the activity, teachers and students interacted with the environment while using regular and mobile technologies. In addition, we introduced EDU.Tube’s architecture including its regular and mobile instances. We presented our exploration efforts concerning possible exploitation of our technological approach for supporting better and more appealing collaborative learning activities. We proposed to use our technological approach to empower such processes with richer and more interactive forms of media that could be experienced anywhere by individual or groups of students. We used EDU.Tube and combined it with the CeLS environment in order to enable support for the orchestration and enactment of collaborative learning activities (Ronen and Kohen-Vacs 2010). The integrated approach used during these research efforts is intended to be further developed and refined. These coming efforts are aligned with similar activities recently exercised and addressed advantages and possible drawbacks for using online videos for educational purposes. Some of these efforts were addressed in other researches also considering potentials of experiencing educational video-clips on mobile devices (Diwanji et al. 2014; Kukulska-Hulme and Traxler 2013; Revelle et al. 2015; Reychav and Wu 2015). Specifically, these other efforts addressed topics like adaptation of existing content for educational purpose that could be experienced on mobiles. Few of these efforts addressed aspects related to the development of technological architectures used for supporting elicitation and deployment of such educational activities relying on interactive video-clips as educational content.
Our pedagogical design aimed to provide students with the opportunity to learn by teaching while preparing educational materials, as well as assessing educational materials created by their peers (Nagel and Kotzé 2010; Ross 2012). Specifically, students had to author interactions incorporated to video-clips that were later offered as new forms of educational opportunities. The outcomes of this activity included the creation of new educational material that could be used by teachers in future lessons encompassed by assessments indicating on their qualities and drawbacks. In addition these materials could be considered to be shared and used by other teachers and students accessing Open Educational Recourses (OER) for seeking better and more appealing educational materials to be used for their teaching or learning practices.
The mentioned activity was followed by our evaluation and analysis aiming to assess different aspects of the used technology and its instances including its ease of use and acceptance among students. Using an adapted form of the TAM questionnaire supported this analysis. The outcomes of this analysis addressed various aspects including students’ preferences in terms of their selection of technologies and therefore instances used across contexts. In addition, students also reported about processes and actions they prefer to take before interacting with the video-clips. Finally, they suggested adapting their future learning pathways according to their previous performance practiced with these interactive-videos.
As mentioned, we acknowledge the challenges and drawbacks concerning various aspects of our use of the TAM questionnaire. Specifically, we realized that the structure of groups using various technologies might be addressed differently while enabling a better examination and comparisons of users’ experiences using different devices. Accordingly, the used questionnaire requires additional refinements in terms of the Likert scale for enabling a more efficient comparison among the various and addressed categories.
These aspects concerning evaluations using TAM in the context of mobile learning were also addressed by other researchers (Huang et al. 2007). However, we share this experience towards its further refinements for future deployment and research efforts. Nevertheless, we considered that the information reported in this paper concerning our deployment efforts, includes novel ideas and new knowledge. Specifically, we offer new perspectives concerning challenges related to how to exploit rich and interactive media technologies to better support educational processes. We bring here this information as we consider its possible benefits to the community of researchers and teachers involved in similar and future design and deployment efforts. In addition, we emphasize the concepts and architecture implemented in EDU.Tube for future deployments possibly practiced across levels and domains.
The insights expressed by students while using the questionnaire are also considered here towards our future research efforts aiming to design new features dealing with learning analytics. As implied from the results of the questionnaire, students’ insights and suggestions are related to locations and situations in which they experienced the environment (Zimmerman et al. 2007). Specifically, these suggestions address different aspects of context concerning individual aspects, temporal dimensions, location of use, characteristics of the educational activity and finally relations between participants in the mentioned activity. Specifically, in our future efforts we will use these insights and outcomes in order to provide recommendations for more appropriate, rich and interactive videos used as educational opportunities across contexts (Ferguson and Shum 2012). We aim to provide such type of recommendations while relying on contextual information including the time, place and situation in which the video clips were experienced (Sotsenko et al. 2013). This information will also rely on data automatically provided by devices in which videos were experienced. In addition, our recommendations will address learning materials while considering the frequency in which these artefacts were used. Finally, we will focus our future efforts in exploring possible evolutions of artefacts occurring along authoring processes.
IF Alexander, L Beus-Dukic, Discovering requirements: how to specify products and services (John Wiley & Sons, Chichester England, 2009)
N Baloian, J Frez, M Jansen, G Zurita, The Future Role of HTML5 in Mobile Learning Scenarios. In proceedings of Mobile Learning Conference, 2011, pp. 249–257
I. Boticki, L.H. Wong, C.K. Looi, Designing technology for content-independent collaborative mobile learning. IEEE Trans. Learn. Technol. 6(1), 14–24 (2013)
T Cochrane, R Bateman, Strategies for mlearning integration: Evaluating a case study of staging and scaffolding mlearning integration across a three-year bachelor’s degree. Waikato J Educ. 16(1), 107-122 (2011)
FD Davis, Perceived usefulness, perceived ease of use, and user acceptance of information technology, 1989, MIS quarterly, pp. 319-340
P Diwanji, BP Simon, M Marki, S Korkut, R Dornberger, Success factors of online learning videos. In Interactive Mobile Communication Technologies and Learning (IMCL), 2014 International Conference on IEE, 2014, pp. 125-132
R Ferguson, SB Shum, Social learning analytics: five approaches. In Proceedings of the 2nd international conference on learning analytics and knowledge, Vancouver, Canada, 2012, pp. 23-33
M Frantzis, V Zsombori, M Rusu, RL Guimaraes, I Kegel, R Craigie, Interactive video stories from user generated content: a school concert use case. In Interactive Storytelling, 2012, Springer Berlin Heidelberg, pp. 183-195
A. Giemza, L. Bollen, H.U. Hoppe, LEMONADE: field-trip authoring and classroom reporting for integrated mobile learning scenarios with intelligent agent support. Int. J Mobile Learn. Organ. 5(1), 96–114 (2011a)
A Giemza, M Jansen, An architectural approach to support multi-device learning environments. In Proc. of the IADIS International Conference on Mobile Learning, 2011, pp. 319-321
M. Gilroy, M. Higher education migrates to YouTube and social networks. Educ. Digest 75(7), 18–22 (2010)
P.B. Godhwani, A research paper on SEO aid in mobile website optimization. Int. J Manag. IT Eng. 3(6), 282–288 (2013)
Goggin, Cell phone culture: Mobile technology in everyday life (Routledge, Abingdon UK, 2012)
SM Jacob, B Issac, The mobile devices and its mobile learning usage analysis. arXiv preprint arXiv:1410.4375 (2014)
J.H. Huang, Y.R. Lin, S.T. Chuang, Elucidating user behavior of mobile learning: A perspective of the extended technology acceptance model. Electron. Libr. 25(5), 585–598 (2007)
M Jansen, J Gottdenker, P Rossmanith, M Milrad, Exploring the use of web services for the design and implementation of innovative collaborative technologies. In Proceedings of the 3rd Nordic Conference on Web Services, Växjö, Sweden, November 2004
M.T. Heikkinen, J. Still, Benefits and challenges of new mobile service development in R&D network. Pers. Ubiquit. Comput. 12(1), 85–94 (2008)
M Kearney, S Schuck, K Burden, P Aubusson, Viewing mobile learning from a pedagogical perspective. Research in learning technology, 2012 p. 20
D Kohen-Vacs, M Jansen, M Milrad, M Ronen, Designing and Developing Interactive Video Experiences for Supporting M-Learning. In the 22nd International Conference on Computers in Education ICCE 2014, Nara, Japan, pp. 413-422 (Asia-Pacific Society for Computers in Education, 2014)
D Kohen-Vacs, M Jansen, M Milrad, Integrating Interactive Videos in Mobile Learning Scenarios. QScience Proceedings, (12th World Conference on Mobile and Contextual Learning (mLearn 2013)
M. Kravcik, A. Kaibel, M. Specht, L. Terrenghi, Mobile collector for field trips. J Educ. Technol. Soc. 7(2), 25–33 (2004)
A Kukulska-Hulme, J Traxler, Design Principles for Mobile Learning. Rethinking Pedagogy for a Digital Age: Designing for 21st Century Learning, 2013, p. 244
A Kukulska-Hulme, Mobile usability in educational contexts: what have we learnt?, The International Review of Research in Open and Distributed Learning, 2007, 8(2), pp. 1-16
S Kurkovsky, Integrating mobile culture into computing education. In Integrated STEM Education Conference (ISEC), IEEE, 2012, pp. 1-4
A. Lederer, D.J. Maupin, M.P. Sena, Y. Zhuang, The technology acceptance model and the World Wide Web. Decis. Support. Syst. 29(3), 269–282 (2000)
E Marcotte, Responsive web design. A list apart, 2010, p. 306
AS Mohammad, V Tomberg, Harnessing the Potential of Accessibility Standards and Responsive Web Design Practices to Achieve Learning Interoperability on the Level of the User Interface. In Advances in Web-Based Learning–ICWL 2013 (Springer Berlin Heidelberg, Heidelberg Germany, 2013), pp. 294-305
L. Nagel, T.G. Kotzé, Supersizing e-learning: What a CoI survey reveals about teaching presence in a large online class. Internet High. Educ. 13(1), 45–51 (2010)
D Parsons, The Future of Mobile Learning and Implications for Education and Training. Increasing Access, 2014, p. 217
K. Petrova, C. Li, Evaluating mobile learning artefacts, 2009
I. Reychav, D. Wu, Mobile collaborative learning: The role of individual learning in groups through text and video content delivery in tablets. Comput. Hum. Behav. 50, 520–534 (2015)
T Reenskaug, Mvc xerox parc 1978-79, Trygve/MVC, 1979
G Revelle, E Reardon, K Cook, L Takeuchi, R Ballagas, K Mori, KH Horii, H Raffle, M Sandberg, M Spasojevic, Agents: Combining Collaborative Mobile Augmented Reality and Web-Based Video to Reinvent Interactive Television. Computers in Entertainment (CIE), 2015, 12(3), 1-23.
M Ronen, D Kohen-Vacs, Modeling, enacting, sharing and reusing online collaborative pedagogy with CeLS. Techniques for Fostering Collaboration in Online Learning Communities: Theoretical and Practical Perspectives, IGI Global, 2010, pp. 319-339
M.T. Ross, Teachers who study and students who teach: Are we really so different? Medical Teacher 34(5), 351–353 (2012)
W.S.N.S. Saifudin, S. Salam, M.H.L. Abdullah, Multimedia mobile content development framework and methodology for developing M-Learning applications. J Technical Educ Training 4(1), 15–21 (2012)
M. Sarrab, L. Elgamel, H. Aldabbas, Mobile learning (m-learning) and educational environments. Int. J Distributed Parallel Syst 3(4), 31–38 (2012)
DC Schmidt, M Stal, H Rohnert, F Buschmann, Pattern-Oriented Software Architecture, Patterns for Concurrent and Networked Objects, Vol. 2 (John Wiley & Sons, Chichester England, 2013)
A Sotsenko, M Jansen, M Milrad, About the Contextualization of Learning Objects in Mobile Learning Settings. QScience Proceedings, (12th World Conference on Mobile and Contextual Learning (mLearn 2013)
P. Thompson, The digital natives as learners: Technology use patterns and approaches to learning. Comput. Educ. 65, 12–33 (2013)
S. Wang, S. Dey, Adaptive mobile cloud computing to enable rich mobile multimedia applications. IEEE Trans. Multimedia 15(4), 870–883 (2013)
J Zbick, M Jansen, M Milrad, Towards a web-based framework to support end-user programming of mobile learning activities. Learning Technologies (ICALT), 2014 IEEE 14th International Conference on Advanced Learning Technologies, 2014, pp. 204-208.
A Zimmermann, A Lorenz, R Oppermann, An operational definition of context. In Modeling and using context (Springer Berlin Heidelberg, Heidelberg Germany, 2007), pp. 558-571.
About this article
Cite this article
Kohen-Vacs, D., Milrad, M., Ronen, M. et al. Evaluation of enhanced educational experiences using interactive videos and web technologies: pedagogical and architectural considerations. Smart Learn. Environ. 3, 6 (2016). https://doi.org/10.1186/s40561-016-0029-z