Introduction to smart learning analytics: foundations and developments in video-based learning

Smart learning has become a new term to describe technological and social developments (e.g., Big and Open Data, Internet of Things, RFID, and NFC) enable effective, efficient, engaging and personalized learning. Collecting and combining learning analytics coming from different channels can clearly provide valuable information in designing and developing smart learning. Although, the potential of learning analytics to enable smart learning is very promising area, it remains non-investigated and even ill-defined concept. The paper defines the subset of learning analytics that focuses on supporting the features and the processes of smart learning, under the term Smart Learning Analytics. This is followed by a brief discussion on the prospects and drawbacks of Smart Learning Analytics and their recent foundations and developments in the area of Video-Based Learning. Drawing from our experience with the recent international workshops in Smart Environments and Analytics in Video-Based Learning, we present the state-of-the-art developments as well as the four selected contributions. The paper further draws attention to the great potential and need for research in the area of Smart Learning Analytics.

“Smart” is a new notion coined to describe technological, economic and social developments enabled by technologies that rely on sensors, big data, open data, new ways of connectivity and exchange of information (e.g., Internet of Things, RFIDs. Wearable Technologies). In most of the times, it is not so much the individual technological developments but rather the interconnection, synchronization and concerted use of different technologies that constitutes smart behavior (Höjer and Wangel, 2015).

Gathering evidence of learning laid the foundation of educational technology and learning science, in general. Today’s role of analytics on helping individuals to make sense of the learning procedures, and smart environments on providing feedback and diverse “smart” functionalities, have drawn the interest of many scholars and practitioners. Latest developments in these two areas posit their intersection as a very promising research area. From the synergy of learning analytics and smart learning, we define Smart Learning Analytics as a subset of learning analytics that focuses on supporting the features and the processes of smart learning.

In practice smart is often used to drive specific political agendas and to promote technological solutions (e.g., promoting mobile, VR, wearable technologies). While these technologies and new approaches to data-collection, data-sharing and sense-making are important for smart learning analytics, they provide only part of the overall picture smart learning analytics encompass. Contemporary research in learning technologies is largely focused on describing the phenomenon in the form of case studies or on discussing isolated technological developments rather than leveraging learning analytics capabilities to enable effective, efficient, engaging and personalized learning (smart learning). There is therefore a need to provide insights of what smart learning analytics is, and how can help us to advance the current learning technologies research.

Video and multimedia technologies are becoming more prominent in the world of education. Most of today’s learning environments have video affordances. In addition, advanced video repository systems have seen enormous growth (e.g., Khan Academy, Lynda.com, PBS Teachers) through social software tools and the possibilities to enhance videos on them. During the last years we have seen several research studies in interactive and innovative features, such as slide-video separation, annotation, social categorization and navigation, and advanced search, that have become standards for any state of-the-art video-based learning system (Kim et al., 2014; Kleftodimos & Evangelidis, 2016a; Wachtler & Ebner, 2015). Thus, video-based learning is an emerging field with its elements available in most of the contemporary learning systems. In this section, we will see aspects related with video-based learning, the analytics around that as well as the potentialities for integrating “smartness”.

Video-based learning

The advances of technology-supported open access to education indicate an increased use of video technology; video technology has tremendous potential when pedagogically appropriate and designed purposely to facilitate teaching and learning. From current research, it is difficult to tell what aspects of the video-lectures and video-based learning systems can have a positive impact. In order to employ videos that serve as powerful pedagogical tools, care should be taken to examine their impact on the overall learner experience. As such, exploring how smart learning analytics can help us to improve video-systems learning potential is of great importance.

Existing empirical research (e.g., Giannakos, 2013) has begun to identify the educational advantages and disadvantages of video-based learning. However, there still remain many essential unexplored aspects of video-based learning and the related challenges and opportunities; such as, how to use all the data obtained from the learner, how to combine data from different sources, how to make sense heterogeneous learning analytics, how to synchronize and take the full advantage of learning analytics coming from different sources, how to use analytics to inform and tune smart learning and so on. Videos have long been used for learning, for instance more than forty years ago Spivack (1973) used VHS video simulations to help train counselors to be more effective. Throughout the years the video formats, quality and delivery (e.g., CD, DVD, VHS, web) have changed dramatically. Today what makes again video-based learning ripe for exploration is the possibility to incorporate interactive elements and smart behavior and enable effective, efficient, engaging and personalized learning. In this turning point, smart learning analytics is expected to have critical role.

Video learning analytics

Millions of learners enjoy video streaming from different platforms (e.g., YouTube, Coursera, Khan Academy, EdX, Udacity, Iversity) on a diverse number of devices (desktop, smart phone, tablets) and create large volume of interactions. This amount of learning activity might be converted via analytics into useful information (Giannakos et al., 2015) for the benefit of all video learners. As the number of learners' watching videos on Web-based systems increases, more and more interactions have the potential to be gathered. Capturing, sharing and analyzing these interactions (big datasets) can clearly provide scholars and educators with valuable information (Giannakos et al., 2015). We also expect that the combination of various learning analytics (e.g., content metadata, learners’ profile) as well as the state-of-the-art statistical analysis techniques (e.g., Kidziński et al. 2015; Pappas et al., 2016) will allow us to better understand complex learning phenomena by making sense of heterogeneous big learning analytics; this is of particular interest to video-based learning due to the large and complex datasets.

Smart video-based learning

The International Association for Smart Learning Environments (IASLE: http://www.iasle.net/) provides a broad interpretation of what a smart learning environment is. In particular, IASLE states that a learning environment can be considered smart when various innovative features and attributes like adaptation, flexibility, thoughtfulness and so on are associated with the system (Spector, 2014). In a general sense, a smart learning can be described as a learning process characterized from effectiveness, efficient and engaging for a wide variety of learners with different levels of prior knowledge (adaptation affordances). Smart learning is enabled by technologies that rely on sensors, big data, open data, new ways of connectivity and exchange of information (e.g., Internet of Things, RFIDs); those integrated environments belong on the broad sense of smart learning environments. Like any other type of learning environments, video-based learning environments need to follow the same principles, and while video-based learning environments are becoming more flexible, thoughtful and adaptive (e.g., Khan Academy, Udacity) as well as several new such environments that incorporate “smart behavior” are created (e.g., Adaptemy, Dreambox, SmartSparrow); there is a lack of empirical analytics-based research on what ingredients of smart behavior can indeed increase effectiveness, efficiency and engagement. There is therefore a need to conduct empirically-driven research in the area of Smart Learning Analytics.

While it is a phenomenon of growing significance, scholarly work on smart learning analytics is lacking, both conceptually and empirically. In response to this emerging need, the journal of Smart Learning Environments presents a special issue that disseminates the latest research findings related to smart learning analytics in video-based learning (i.e. Smart Video Learning Analytics).

Smart video learning analytics for open educational resources

Smart interactive features in videos are constantly evolving and a recent trend is the integration of smart behavior, interactive elements and web content into learning videos. Kleftodimos and Evangelidis (2016b) provide a roadmap for using open source tools and open internet resources to develop a learning environment where open video content is aggregated with interactive elements. In addition, they describe how open source tools are used for capturing and storing smart learning analytics. Kleftodimos and Evangelidis (2016b) also present findings obtained from smart learning analytics gathered in an educational setting during an academic year. In summary, the authors provide innovative ways of integrating smart interactive features in a video based environment using open educational resources and open source technologies.

Collaborative video annotation resources for smart video learning analytics

Despite the number of potentialities, video technologies for learning provide limited video-learner interaction (VLI). In order to increase VLI, Chatti et al., (2016) present the design, implementation, and evaluation of collaborative video annotation platform (called CourseMapper) that enables learners to collaborate and interact with a video lecture. CourseMapper as a mind map-based collaborative video annotation and analytics platform that enables learners’ collaboration and interaction around a video lecture. CourseMapper focuses on the application of learning analytics mainly from a learner perspective to support “smart” features like, self-organized and networked learning through personalization of the learning environment, monitoring of the learning process, awareness, self-reflection, motivation, and feedback.

Smart video-based learning ecosystem to support active learning

Current learning management systems and environments are only a subset of different kinds of learning materials and learning tools that an educator should take into consideration; and most importantly they do not offer an overview of the different learning dynamics. In this paper, Giannakos et al., (2016) provide a video-based learning ecosystem framework, which will allow us to describe “the complex of learning organisms” as well as their interrelationships; the proposed framework is expected to help better understand and further develop the current teaching approaches. In their paper, Giannakos et al., (2016) present the video-based learning ecosystem framework and the first captured results of its application in an introductory computer science course. The framework incorporates basic e-learning tools and active learning practices, making it accessible to anyone wanting to implement a video-assisted active-learning experience in his/her course. The case study was implemented based on open and easy-to-use tools, allowing for the incorporation of any additional functionalities. The framework can be used for those interested in incorporating active learning practices like, project-based or flipped classroom approaches; further empirical validation, extension of the framework and incorporation smart learning elements is expected to increase the effectiveness, efficiency and robustness of the framework.

Design considerations for interactive question-enhanced video lectures

The technical aspects of video-based systems have changed dramatically over the last years. The potential to incorporate smart and interactive state-of-the-art features posit video learning in the forefront of the learning technologies arena. Towards this direction, Wachtler et al., (2016) focus on interactive questions within learning videos. During the last years, more and more universities and educational technology providers (e.g., Udacity, edX) have incorporated interactive questions in their video-based learning systems; however, there the research in the area is limited. Wachtler et al., (2016) investigate the success rate of interactive questions, in relation to their position (e.g., early appearance) and tightness (e.g., time between the questions). After gathering data from a four-week case study, Wachtler et al., (2016) provided several design considerations for interactive question-enhanced videos.

Towards smart-interactive affordances in video-based learning environments

Considering all the four contribution of the special issue and the recent developments in the area of video-based learning; it is clear that research efforts focusing towards the development of interactivity affordances and smart features to support video-based learning. In general, whenever new interactivity affordances and smart features appear, users tend to ignore them and keep using these media just like how they did before. For instance, many early digital textbooks were simply used as traditional books, without taking into consideration the various offered affordances (e.g., search functionalities, annotations, collaborative reading and writing). As time progresses, people eventually develop creative ways to take full advantage of the new affordances and mediums. Thus, it is expected that findings from the current state-of-the-art research in video-based learning environments will need time and great effort from the various users (e.g., learners, teachers), before being fully accepted and exploited in their daily learning activities.

Role of data science methods in the development of learning technologies changed greatly over the last few decades. Before Digital Age, when experiments included only a handful of students, data science provided tools for designing experiments, for confirming or rejecting hypothesis and for exploring intermediate or latent patterns. Now, since datasets at hand often concern millions of subjects and include gradual information about their behavior, data science is used also to form hypotheses (e.g. Guo et al., 2014; Li et al. 2015). Efficient analysis not only requires execution of right data science methods but also requires deep understanding of the domain.

The development of data-driven Smart Learning Environments requires experts from educational science as well as from data science. This situation contributed to the quick rise of communities of Learning Analytics and Educational Data Mining (Siemens & d Baker 2012). Techniques employed to make sense of data gathered in Smart Learning Environments range throughout data scientists’ toolkit. For regression researchers reach for classical linear models, as well as complex machine learning models such as neural networks (Harp et al., 1995) or Markov Chains for modeling in-video behavior (Branch et al., 1999). For clustering they employ classical k-means methods as well as state-of-the-art improvements such as Neural Gas (Li et al. 2015).

This technical expertise of researchers in Smart Learning Environments makes us confident that future research will not only bring to educational research great techniques borrowed from data science (like Fuzzy-Set Qualitative Comparative Analysis; Pappas et al. 2016), but also will give raise to new important methodological developments, designed in the context of education such as Bayesian Knowledge Tracing (Corbett & Anderson 1994).

In order to employ videos and integrated video-based systems that can serve as powerful learning tools, care should be taken to examine the impact on the overall learner experience. The overreaching objective of this special issue is to explore how smart learning analytics can improve video-systems potential to enhance learning experience. Drawing from our experience during the 2015 and 2016 Smart Environments and Analytics on Video-Based Learning (SE@VBL) workshops, the research in the area is can be portrayed from the following seven objectives:

1. Q1.

   What kind of learning analytics can help us to better understand and empower “smart” learning experience?

2. Q2.

   What affordances are vital for smart video learning environments?

3. Q3.

   Is any type of content more appropriate for smart video learning environments?

4. Q4.

   What practices and pedagogies can be (in particular) supported from smart video learning environments?

5. Q5.

   What assessment functionalities can be integrated in smart video learning environments?

6. Q6.

   What are the main design challenges in smart video learning environments?

7. Q7.

   What are the expected results/outcomes from smart video learning environments?

The main objective of SE@VBL workshops was to develop a critical discussion about the next generation of environments and analytics employed in video learning tools, the form of these environments and analytics, and the way they can be utilized in order to help us to better understand and improve the value and “smartness” of video-based learning. The seven aforementioned objectives were used to guide the discussion.

In order to collect all the different opinions and categorize them together, we decided to employ affinity diagram technique within the different focus groups we had in the workshop. Using a focus group enables a wide variety of collective views and often leads to results based on a consensus among participants (Maguire and Bevan, 2002). The affinity diagram is a tool used to organize ideas and data from a large amount of data (Maguire and Bevan, 2002). This tool is commonly used within project management and ethnographic studies as it allows large numbers of ideas stemming from brainstorming and other qualitative data to be sorted into groups, based on their relationships, for review and analysis. The main steps of the technique are: (1) record each idea on a note, (2) look for ideas that seem to be related, and (3) sort notes into groups.

In the 2016 SE@VBL workshop jointly organized with the sixth international conference on Learning Analytics & Knowledge (LAK ‘16); the eighteen participants were divided into focus groups. The focus groups that worked with the seven aforementioned objectives and used post-it notes to construct affinity diagrams and provide information for each of the seven aspects. This was an iterative process that consisted of adding or removing post-it notes until a consensus was reached. The affinity diagrams resulted were then presented and discussed with the other focus groups. Table 1 summarized the outcome of this process.

Capturing and mapping ideas coming from experts is a difficult task, and although, the followed method has certain limitations. We believe that the categorized information (Table 1) can serve as useful guidance in future discussions and research of Smart Learning Analytics in video-based learning. According to the extensive discussions and idea collection/categorization during the workshop, we think that these seven dimensions can serve as a reference to assist future research on how Smart Learning Analytics can help us to advance the current learning technologies research, especially the video-based environments.

The roles of 1) analytics on helping individuals to make sense of the learning procedures and the 2) smart environments on providing feedback and diverse “smart” functionalities have drawn the interest of many scholars and practitioners in the last years. In this editorial we introduce the intersection of these two promising areas under the term of Smart Learning Analytics. This focus of the special issue is in video-based learning, video-based learning has seen enormous growth during the last years and comparing with research on text and discourse analytics, the research on video analytics is still on an early stage. Video analytics have an enormous potential, especially given what is currently happening around MOOCs and adaptive video-based learning systems. Most of the current learning systems incorporate video elements; during the last years most of the learning environments adopted video technology as the primary content delivery mechanism. Hence, research on video-based learning environments has great potential and will heavily influence the future of learning in both formal and informal settings.

This potential will grow as learning platforms, like Coursera and Edx make their data publicly available to the research community as well as integrate more “smart” qualities like, system’s ability to: achieve recognized goals and objectives, adjust to different situations and provide the appropriate feedback to the learner. Towards this direction research on Smart Learning Analytics is expected to play critical role and shape the future of environments and analytics for learning.

This work was partly supported by the Research Council of Norway under the projects FUTURE LEARNING (number: 255129/H20) and SE@VBL (number: 248523/H20), and by the GEMINI Center in Technology Enhanced Lifelong Learning (http://www.tell-gemini.org/). We would also like to thank SE@VBL workshop Program Committee members and participants who provided us with valuable input and useful discussions. We also want to thank the reviewers of this special issue for their contribution. Last but not least we would like to thank the Editors-in-Chief of Smart Learning Environments Journal for their support and guidance throughout the special issue processes.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

All authors read and approved the final manuscript.

Affiliations

1. Norwegian University of Science and Technology, Trondheim, Norway

   • Michail N. Giannakos

2. Curtin University, Perth, Australia

   • Demetrios G. Sampson

3. Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

   • Łukasz Kidziński

Corresponding author

Correspondence to Michail N. Giannakos.

Reprints and Permissions