The design requirements collected above have built the basis for the implementation of CourseMapper 1. To note that in this paper, we only focus on the realization of the the first five requirements as these are related to video content. In the ensuing sections, we present the technologies used in the implementation of CourseMapper followed by a detailed description of the implemented video annotation and visual analytics modules and their underlying functionalities.
Technologies
In the server side backbone of CourseMapper lays Node.JS and Express Framework. Node.JS provides great event-driven, non-blocking I/O mode, which enables fast and scalable applications to be written in plain JavaScript (JS). Node.JS has a very steep learning curve and its default callback based programming style makes it harder for developers to write any blocking code. Express is a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications.
In order to provide real-time annotation updates and editing, CourseMapper has integrated Socket.IO engine. It bases the communication over WebSockets, however it does not assumes that they are enabled and will work by default. At first it establishes a connection with XHR or JSONP and then attempts to upgrade the connection. This means that users with browser, which does not support WebSocket-based connections will not have any degraded experience. Persistent login sessions are established via Passport.JS middleware, supporting multiple authentication schemas, including OAuth. Upon their choice users can select to login with their Facebook account and do not maintain one within the system.
Application data is stored inside MongoDB a cross-platform NoSQL document-oriented database. It substitutes the traditional table-based relational structure with JSON-like documents, which allows data easier and faster data integration. In order to simplify client side development and testing CourseMapper uses Angular, a framework providing modelviewcontroller (MVC) and modelviewviewmodel (MVVM) architectures, along with commonly used components.
For content playback, CourseMapper uses Videogular. It is an HTML5 video player for AngularJS. The player comes with default controls and multiple plugins, such as several scrub-bars, cue points (a way to trigger functions related to time) and many more. Videogular also significantly simplifies the way new plugins and controls can be developed, styled and integrated into it.
Components
The video annotation section workspace of CourseMapper can be seen in Fig. 1. It consists of a video player and several components that are listed below. A general note to take is that there are many other features of CourseMapper, which we will not describe in this paper in order to focus mainly on the video annotation and analytics parts of the platform.
Annotation viewer
The annotation viewer is a system component that loads existing annotation from the server via WebSockets and reflects any changes in real-time. Each annotation is displayed in its own container and further comments can be made when the comment section is expanded, as shown in Fig. 2.
Annotation editor
The CourseMapper annotation editor allows users to create or update existing annotations. It is a user control placed within the layout of the annotation viewer and hosts editors for each field of the annotation model, such as text, start time, end time, annotation type. It is important to note that everyone can create annotation, however only moderators which are listed for the current course or annotation owners can edit and update the content of an existing annotation. A snapshot of the control can be seen in Fig. 3.
Embedded note vs note
CourseMapper enables users to distinguish between two different types of annotations, namely notes and embedded notes. However, they can be mutually exchanged for a single annotation, or to be more precise an embedded note can be easily converted to a note or vice-versa.
Note
is an annotations that is bound to a specific timeframe within the video content, however it is only displayed inside the main annotation viewer control. A note inside the annotation viewer is activated and highlighted when the current player position crosses and stays in between the start/end time of the annotation. Once the player position exits this window the annotation is therefore marked as completed, it gets deactivated and visually grayed out in order to avoid disturbing the viewer’s attention further on. As an addition this behavior can be seen as two-way binding, due to the fact that if an annotation from the annotation viewer is clicked, it will transition the video player to the start time of the annotation, allowing easy navigation between important parts of the media.
Embedded note
is an annotation that possesses all features of a regular note with an addition of pointing a specific "hotzone" - an opaque rectangular which is overlaid on top of the video content. The rectangular zone’s position and size can be edited and stored as a supplement to the annotation model. Both dimensions are relative and restricted to the maximum of those of the video player’s container. This way a user can specify an important part of the content and focus views attention to it. Whenever the embedded zone gets hovered over inside the player it will display the annotation’s text (see Fig. 4). This features is of a significant use in full screen mode, when the annotations viewer and the rest of the application is not visible.
Find and order annotations
Because users can generate long lists of annotations in a MOOC context, the system provides functionality to sort annotations by alphabetical order, by author name, by time of beginning of the annotation and several others which have been planned in a near release. There is also an easy to use single search control, which performs a lookup on all possible fields of the annotation model, e.g. text, author name, start/end time, creation date. Moreover, it also finds comments to the annotations, that contain the search term in their body or their author, if this is the given search term.
AnnotationMap scrub bar
AnnotationMap is a visual learning analytics component of CourseMapper that extends the regular scrub bar, as shown in Fig. 5. It overlays stacks of annotation windows within the given timeline. It is placed in the controls panel of the video player. In order to keep the user confusion minimal and simplify the visual seeking for annotations the cue points here are displayed in opaque yellow color. The stack zones of overlapping annotation times will sharpen and brighten in a yellow nuance, notifying the viewer that this portion of the video timeline has a larger congregation of annotations and most likely contains interesting information.
Heatmap scrub bar
Heatmap is another visual learning analytics component of CourseMapper. Whenever a student navigates back and forward and interacts with the player he leaves his "footprint", which contributes to the overall heatmap. The Heatmap control extends normal scrub bar with a heatmap based color scheme, where the most viewed parts of video are marked with warm colors such as orange and red, neutral are shades of the yellow spectrum and less viewed parts are usually displayed with cold purple and blue colors, as depicted in Fig. 6. Based on this picture students can visually scan and easily find the most interesting areas of the video. Moreover, the Heatmap shows how many times the video has been watched.
The Heatmap module consists of five parts, two on a server side and three on a client side. The server side provides common API for all clients. All received data is processed and stored on the server side, NodeJS and MongoDB work together in order to process requests as fast as possible and to support large numbers of users online. The server side provides two routes:
-
GET/get - returns data of the particular page based on request headers. It is not possible to specify page URL, this decision will be made in automatic manner on the server side.
-
POST/save - saves or update data of the particular page based on request headers.
The main task of the client side is to avoid all interaction with the structure of a host system or web site. It consists of three parts: Observer, Heatmap and Viewer. Each part has its own task, for instance, the Observer has to handle all important events in order to track user behavior. It also handles special types of events about a state of a user, like "idle" or "active". The Heatmap uses HTML5 canvas in order to represent input data using predefined colors. And finally the Viewer is a part which mostly interacts with the host system. It fetches data and embeds heatmap in content viewer. In the next sections, we discuss the implementation of these parts in more details.
Observer
The Observer class is used to collect information about how users view a content and then send the data to the server side using POST/save AJAX call. HTML5 Video provides API to get such events like play, pause, stop, seeking, etc. The Observer class subscribes to those events and listens for all actions that user makes while watching a video. Each time when a user is watching some part of a video the Observer stores start point as a value from 0 to 1. For example, if a user starts watching from the middle of a video the Observer will save new start point - 0.5. In the same way Observer stores endpoint of a watched video.
Heatmap
The Heatmap component is a basic implementation of 2D heatmaps called "simpleheat". However, instead of 2D, FootPrint implementation works in 1D space. As an input, LinearHeatmap accepts an array of values and maximum possible value. LinearHeatmap is a light implementation of linear heatmap that allows precise heatmaps configurations. The colorization algorithm works as follows:
-
1.
At first LinearHeatmap generates color palette which will be used to set correct colors in draw function. This step passes only once.
-
2.
LinearHeatmap builds grayscale gradient using standard canvas API. The result of the first step will be black linear gradient with different values of alpha.
-
3.
Based on alpha value in each pixel LinearHeatmap applies correct color that is stored in color palette
Viewer
The main task of the Viewer class is to extend regular controls with generated heatmap. Video Viewer uses standard HTML5 player and adds an additional slider on the top part of a video. This slider based on custom HTML and CSS with canvas element inside, that is used by LinearHeatmap class to draw a heatmap. Additional slider shows "hottest" or most viewed parts of the video. At the same time, Observer class gathers data about viewed parts of the current user and each viewing of some part of a video is a contribution to the entire heatmap.