- Open Access
Video-based learning ecosystem to support active learning: application to an introductory computer science course
Smart Learning Environmentsvolume 3, Article number: 11 (2016)
The systematic use of technologies in order to orchestrate learning has become widely used in the past years. Diverse technologies have been applied in a variety of teaching practices; for instance learning tools which allow you to flip the classroom or monitor other active learning practices. However, the developed systems are only a subset of different kinds of learning materials and learning tools that an educator should take into consideration; and most importantly they do not offer an overview of the different learning dynamics. The development of a learning ecosystem framework, which will allow us to describe “the complex of living organisms” as well as their interrelationships, will help us to better understand and further develop our teaching approaches. In this paper, we present a video-based learning ecosystem framework and the first captured results of its application in an introductory computer science course. The framework incorporates basic e-learning tools and traditional learning practices, making it accessible to anyone wanting to implement a video-assisted project-based experience in his/her course. Its application is based on open and easy-to-use tools, allowing for the incorporation of any additional functionalities. This work aims to provide insights for other scholars and practitioners to further validate, examine, and extend the proposed framework. This approach can be used for those interested in incorporating project-based or flipped classroom approaches in their teaching, since it is a flexible procedure that may be adapted to meet their needs.
Traditional lecture style is a common teaching approach in higher education classes; however, the traditional lecture style of teaching can often place students in a passive role, which typically involves students retaining isolated facts that can later be forgotten. Following Bligh (2000) definition, traditional lecture style is a “continuous exposition by the teacher”; in a traditional lecture instruction students’ activity is limited to taking notes and asking questions to the instructor. Over the last few years, instructors have been moving away from the traditional lecture style by implementing more active learning practices, like project-based learning and flipped classroom, and increasing the technology use as a way to extend and enhance students’ understanding.
Active and problem-based learning activities are founded upon a constructivist theory and traditional lecturing derived from direct instruction methods is founded upon behaviorist ideology. Active learning is a model of instruction that focuses the responsibility of learning on learners. It was popularized in the 1990s by its appearance on the Association for the Study of Higher Education (ASHE) report (Bonwell & Eison 1991). During the last years, active learning practices like project-based learning and flipped classroom have gained prominence worldwide, however sometimes we face a lack of consensus on what exactly active learning is. A working definition of active learning, derived from collecting opinions from 338 experts is the following: “Active learning engages students in the process of learning through activities and/or discussion in class, as opposed to passively listening to an expert. It emphasizes higher-order thinking and often involves group work.” (Freeman et al., 2014). Although this definition is quite generic, it perfectly portraits the rationale of active learning, without being restrictive. Active learning has been deployed in a number of education studies, however a framework describing the learning dynamics is typically not described; the development of a learning ecosystem framework to support active learning, will allow us to better understand and further develop teaching approaches enhancing students’ dynamics and needs.
A learning ecosystem has been described as a means for orchestrating a variety of learning approaches given by the varied characteristics of learning processes (Siemens, 2003). Learning ecosystems have seen as environments which are “consistent with (not antagonistic to) how learners learn.” (Giannakos et al., 2016), focusing on the learning process and take into account learners’ characteristics, needs and the potential dynamics and interactions with different actors (students, educators), as well as the learning environment and resources. Thus, the concept of learning ecosystem provides an ideal ground to orchestrate multiple tools and practices in the best possible way (Dillenbourg & Jermann, 2010).
Today, diverse technologies have been applied in a variety of active learning practices. However, isolated use of different technologies does not offer an overview of the different learning dynamics. Developing a learning ecosystem framework, which will allow us to describe “the complex of living organisms” as well as their interrelationships, will help us to better understand and further develop our teaching approaches. In this work we work towards this direction, by the following twofold contribution, first we present a conceptual framework of a learning ecosystem which can host active learning instruction and second we provide some first analytics-based evidence regarding its effectiveness and acceptability.
Building upon existing technologies and practices like video-assignments, clickers and micro-project approach, in the next section we propose a learning ecosystem for active learning. In the third section we present an empirical study following the proposed approach, were by collecting diverse-sourced data, we portray students’ experience throughout the course of the semester. The last section of the paper draws conclusions and discusses ideas for further research in the area. This study aims to provide a springboard for other scholars and practitioners to further examine the efficacy of this specific blended learning approach. Our conceptual framework is a flexible procedure that can be utilized and adapted to meet different needs.
Learning ecosystem for active learning
A traditional ecosystem has been described as “the complex of living organisms, their physical environment, and all their interrelationships in a particular unit of space” (Encyclopedia (2011)). By applying this simple and good working definition to learning; we can describe a learning ecosystem “as the complex of living organisms in a learning environment (e.g. students, educators, resources), and all their interrelationships in a particular unit of space (can be digital or physical)” (Giannakos et al., 2016). In a learning ecosystem it is important to consider the interrelationships of the main actors (students and educators) but also the role of the learning space (both digital and physical). The learning space is by analogy the physical environment in a traditional ecosystem, includes (organisms) information and digital resources like slides, lecture recordings, blog entries and forum discussions; but also physical materials like books, notes and handicrafts, to mention some. The space is where teaching or learning is happening and where such processes and interrelationships are conducted. The interrelationships exist (Shum & Ferguson, 2012; Bernard et al., 2009; Chang & Guetl, 2007) between the main actors (students and educators), the main actors with the resources (content), and the resources themselves (e.g. recommender systems) (Anderson, 2003), see Fig. 1. Those interrelationships shape the quality and value of students’ learning experience and our ability to develop teaching approaches enhancing students’ dynamics and needs is critical towards the emerging ubiquitous learning era.
Active learning as defined by Freeman et al. (2014) is broad and fits many classroom activities. Active learning acts as a superset for both peer-assisted, problem-based as well as flipped learning approaches. Prince (2004) clarifies the relationship between peer-assisted and problem-based learning, indicating that problem-based learning is, “always active and usually (but not necessarily) collaborative or cooperative.” The flipped classroom is also always active and usually (but also not necessarily) collaborative or cooperative, and might be supported with a project work. Hence by adapting and expanding an initial diagram from Bishop & Verleger (2013) we can portrait the complex relationship between these groups of theories, under the term active learning, with the following Venn-diagram (Fig. 2).
Our approach emphasizes relationships and interactions related to the information flow as well as knowledge transfer and transformation. Similarly with a biological ecosystem, in a learning ecosystem, individuals can form groups spontaneously and can interact with each other or with learning utilities at the individual or group level. They can also perform specific behaviors in order to contribute to or perturb to the success of the learning ecosystem (Chang & Guetl, 2007). Changes in the learning ecosystem conditions (external influences) shape the “behavior” of the system and its components. To be successful and to be valuable for the system, each individual and group must adapt to the environmental conditions to find their niches. In order to fit them all together, proper learning infrastructures must also be available (Fig. 3).
The development of the framework emphasizes in the generic view of the learning ecosystem, hence it is possible for an individual to apply it to any active learning situation, such as project based learning, or peer instruction. Another important aspect is to assert that the interrelationships and interactions with all the organisms, external influences as well as the infrastructures of the learning ecosystem are in principle dynamic (Fig. 3). This generic view helps to get a better picture about a specific learning situation, and allows educators and practitioners to achieve a more holistic approach for the development of more effective learning. A graphic representation of this definition is shown in Fig. 3.
An initial empirical validation of the proposed framework took place in a public university. The goal of this empirical validation is to provide the first analytics-based evidence regarding the effectiveness and acceptability of the proposed framework. The early results should not be seen as a rigorous evaluation of the proposed framework, but as reflections rising from a particular case study as well as empirical evidence for further development of the framework.
Following the proposed framework (Fig. 3), we implemented video-assignments to scaffold the know/believe part, this helped us to assure that students have attained the fundamental knowledge. This is typical in many active learning approaches (e.g., flipped classroom), where students are involved with the learning materials in order to obtain the initial fundamental knowledge. This basic knowledge was made available using video lectures, in addition we employed an integrated assessment to the video lecture (see Fig. 4) and weekly exercises. These self-regulated, but also well-defined concepts allowed students to better understand the video and reflect about their learning (Giannakos et al., 2015; Kleftodimos & Evangelidis, 2014). Upon students’ completion of the video lecture and the respective assessments, instructors can access all the collected data, visualize students’ activity and progress, and ultimately identify students’ interaction with the materials. Such information allowed instructor to be well-prepared for the active engagement session by addressing all the misunderstandings and misconceptions of the students (Ruipérez-Valiente et al., 2013; Zhang et al., 2010).
Based again on the proposed framework, the active engagement activities are developed with a main focus on recalling the basic knowledge and then engaging students with active learning and critical thinking processes. The recall part is conducted using a mobile game/quiz at the beginning of the class. The instructor prepared a session with questions related to the basic knowledge, supported with different forms of audio visual materials (e.g., videos) (Wang 2015). The class was equipped with a projector, which was used to display the main screen of the game (see Fig. 5a), and each student used his/her own mobile phone to give the answer to the respective question (see Fig. 5b). At the end of the course, the instructor could download all the collected analytics of the game and explore students’ progress and understanding.
In order to engage students deeply in the process of learning, they worked with a group project throughout the semester and were asked to apply the obtained knowledge as well as to make progress presentation and get feedback. This gave them the opportunity to be involved in an active learning process. The aim was to engage students more deeply in the process of learning course material by encouraging critical thinking and fostering the development of self-directed learning. Active learning affords the opportunity for application and practice, and the asking of questions. During the team micro-projects, students had to find a client and mimic the professional software development process. In particular professional software development projects had the following sequence of phases: requirements, design, implementation and testing.
The empirical evaluation was conducted in two identical classes (in terms of learning goals, teacher, teaching method and so on), those classes had 510 computer science students (20–29 years old, 105 females and 405 males) enrolled in the web technology course. The course lasted 12 weeks, and we applied the proposed framework (as described in section 2.2). In addition to students’ analytics obtained from the aforementioned systems, we employed a post survey. A total of 73 students’ (14.31 %) volunteered to participate on the survey (14 females, 59 males, with mean age 22.38 S.D. 2.50).
In order to be able to portray students’ experience during the course, we employed three different types of measures.
Students’ video navigation (collected via a video learning analytics system (Giannakos et al., 2015), log-files),
Students’ learning performance/score (collected via the quizzes (Wang, 2015), log-files) and,
Students’ attitudes toward the course (collected via the post survey, see Table 1)
In particular, video navigation was captured based on students’ interaction with the video player. Students learning performance was collected in pre-middle-post measures throughout the semester. The survey included multi questions factors of 1) ease, 2) control over the course, 3) intention to participate in similarly made course, and 4) usefulness of the overall teaching approach as well as the single question factors of 1) enjoyment, 2) excitement and 3) boredom. Table 1 lists the operational definitions and the number of items (questions) of each of the constructs (measures), as well as the source from which the multi question measures were adopted. We employed a 7-point Likert scale anchored from 1 (“completely disagree”) to 7 (“completely agree”).
As aforementioned, the collected data consists of three different types; therefore, an appropriate data analysis was used for each different set of data. Students’ video navigation was analyzed with aggregated time series visualizations, in order to identify students’ navigation throughout the video lecture and the importance of having a concrete assignment alongside with the video lecture. To do so we employed the socialskip.org video analytics system (Giannakos et al., 2015).
As for students’ learning performance, we captured students’ pre-mid-post assessments scores mapped them in a diagram and employed an Analysis of Variance (ANOVA), this will allow us not only to capture students’ performance toward the course, but also to identify any potential shift during the course. Last but not least we used descriptive statistics on students’ attitudes towards the course.
By visualizing students’ video usage throughout the semester, we concluded that most of the viewings are happening in three instances. First during the first days of the video release, second during the assignment period where students might need to use some information/knowledge from the video and third before the exam. As we can see in Fig. 6 from a typical video usage visualization, the three main peeks have been identified in the release week, the assignment week and during the last week before the exam.
With the visualization of the students’ activity using graphs, we reach the conclusion that the use associated with the video assessments provides an important motivation for students to watch the videos and be prepared for the class active learning activities. As we can see from Fig. 7, most of the students’ stopped the video after reaching into the segment where the information for the last question was found. On the other hand, the “attractive” (many views) video segments were identified at the video segments where the presenter was giving the solution of the respective problem.
Another important question is related with the intensity of the questions (number of questions per video) in the video assignment. In order to investigate any potential relation between questions’ intensity and students’ watching behavior; we used activity graphs from video assignments with different intensity. We qualitatively concluded that low questions intensity (very few questions, i.e., 2 or 3 in the video) decreases the number of students’ views and increases the early dropouts from the video. Very high intensity (too many questions, i.e., more than 10 in the video) also decreases the number of students’ views and increases the early dropouts from the video. Medium questions’ intensity (i.e. 3–10 questions) is found to keep students’ interested in the video and also not to frustrate them. Figure 8 gives an indicative example of how questions’ intensity is related with students’ watching behavior. We didn’t investigate questions’ quality, but it is very likely that questions quality and demands (questions need students’ to be able to assemble and synthesize instead of merely reproduce the content from video) will also play an important role in students viewing behavior.
To examine any potential shift in students’ performance during the course, we used Analysis of Variances (ANOVA) between the pre, mid and the post-scores. As we can see from Table 2 students’ scores were at very high levels in the pre, mid and the post assessment. Performing an ANOVA, the results showed no significant difference (Table 2). As a consequence, there was no shift in students’ score throughout the course.
Regarding students’ attitudes toward the approach, we used an attitudinal survey to assess them (Table 1). We proceed to test the reliability of each measure using Cronbach α coefficient. The result of the test revealed acceptable indices of reliability in all the factors (> 0.7). We also computed descriptive statistics for the factors under investigation. Students expressed high control (mean: 6.36/7; S.D.: 0.88) over the course, lower (though still very high) level on the intention to participate (mean: 5.91/7; S.D.: 1.06) in similarly developed course in the future. In addition, students’ rated very high the easiness for them to follow the processes (mean: 5.84/7; S.D.: 0.91) of the video course and the usefulness of this teaching approach (mean: 5.63/7; S.D.: 1.09). High levels of these constructs indicate positive views concerning usability, control, and usefulness of the proposed approach. In addition to the multi-questions attitudinal factors, we employed three single-question factors related with students’ feelings regarding the teaching approach. Students expressed high scores on the positive feelings of enjoyment (mean: 5.68/7; S.D.: 1.29) and lower (though still very high) level on the excitement (mean: 4.90/7; S.D.: 1.5) and low scores of boredom (mean: 2.63/7; S.D.: 1.37). Figure 9 depicts the mean scores of the attitudinal survey.
Although all the factors employed in the post-test exhibit students’ intentions and beliefs, they possibly have interrelationships. Noticeable, participants’ intention to participate in the course is quite high, however it would be very important to explore factors that may be strongly related to participants’ intention to participate, we used the Pearson’s rank-correlation coefficient, statistically testing the null hypothesis which is equal to 0. Pearson’s test suggests that some of the factors are related, in some cases relatively strongly. More precisely, the most important positive relationships involve intention to participate on the one side and all the other positive attitudes on the other side. In addition, intention to participate is negatively related to boredom. We present the results of Pearson’s test in Table 3.
To visualize the effects of other attitudes in students’ intention to participate in the course, we divided Easy, Control, USE, Enjoyment, Excitement and Boredom on high and low categories performing median split. We then created a diagram in order to provide a better understanding of the effect. In Fig. 10, we can observe that Boredom is the only one with a negative effect. Overall, Fig. 10 clearly exhibits the positive and significant (based on the Pearson’s correlation) influence of students’ attitudes on their intention to participate in similarly developed courses in the future.
Discussion and conclusions
Practical principles and heuristic models that enable actors within the learning ecosystem to understand and shape their learning future, is considered as one of the cornerstones of smart learning environments (Kwok, 2015). In this research, we presented a learning ecosystem framework and the first captured results of its application. The framework can be put into practice using basic e-learning tools and active learning practices; hence the framework can be used for those interested in incorporating project-based principles as well as flipped classroom or any other active learning approach in their teaching, since it is a flexible procedure that may be adapted to meet different needs.
By exploring the notion of learning ecosystem, we shown that is applicable to describe and model the main actors, diverse resources and sociotechnical dynamics within learning. Hence, the concept of ecosystem is an interesting approach which can be applied in learning and give an overview of the different roles, processed as well as learning dynamics.
The proposed framework was applied in an introductory course in web technologies. In the empirical study, we investigated students’ content navigation, learning performance and attitudes. We also indicated that the main quality of the “attractive” information video-segments is the rich and useful amount of transferred information and knowledge, and of course its association with students’ assessment. Last but not least, we presented students’ progress throughout the course using pre-mid-post assessment, and examined their attitudes regarding easiness, usability, usefulness, and acceptance of the course as well as the two positive and one negative emotions.
We want to emphasize that our findings are clearly preliminary with inevitable limitations. As for the internal validity of the empirical study, data are based on a self-reported method, log-files and assessments. Other in-depth methods such as semi-structured interviews could provide a complementary picture of the findings through data triangulation. As for external validity, the subjects were computer science majors, which may somewhat limit the generalizability of our results. Nevertheless, the insights drawn are not connected with the subjects’ background and can be applied on any population.
Another direction of research is to apply the framework proposed in this paper to the investigation of existing highly engaging learning practices, such as peer instruction and inquiry-based pedagogy. Our future research will concentrate on further refinement of the proposed framework by applying and evaluating it on classes of larger scale and different topics. Further research will also inform the development and evaluation of e-learning environments to better support active learning and livable and sociable learning spaces (Kwok, 2015). This study can provide a springboard for other scholars and practitioners to further examine the efficacy of this specific approach to learning, since it is an established flexible procedure that can be used and adapted to meet the needs of those interested in using the flipped classroom approach.
T. Anderson, Modes of interaction in distance education: Recent developments and research questions. Handbook of distance education, 129–144 (2003)
R.M. Bernard, P.C. Abrami, E. Borokhovski, C.A. Wade, R.M. Tamim, M.A. Surkes, E.C. Bethel, A meta-analysis of three types of interaction treatments in distance education. Rev Educ Res 79(3), 1243–1289 (2009)
J.L. Bishop, M.A. Verleger, The flipped classroom: A survey of the research. In ASEE National Conference Proceedings, (Atlanta, GA, 2013)
D.A. Bligh, What’s the Use of Lectures? (Jossey-Bass, San Francisco, 2000)
C.C. Bonwell, J.A. Eison, Active Learning: Creating Excitement in the Classroom. 1991 ASHE-ERIC Higher Education Reports. ERIC Clearinghouse on Higher Education, The George Washington University, One Dupont Circle, Suite 630, Washington, DC 20036–1183 (1991)
V. Chang, C. Guetl, E-learning ecosystem (eles)-a holistic approach for the development of more effective learning environment for small-and-medium sized enterprises (smes), in Digital Ecosystems and Technologies Conference (IEEE Press, New York, 2007), pp. 420–425
Encyclopedia Britannica, Ecosystem, Encyclopedia Britannica, Inc. last retrieved Jan. 21st, 2015. (2011), http://www.britannica.com/EBchecked/topic/178597/ecosystem. Accessed 2 July 2016
S. Freeman, S.L. Eddy, M. McDonough, M.K. Smith, N. Okoroafor, H. Jordt, M.P. Wenderoth, Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci 111(23), 8410–8415 (2014)
M.N. Giannakos, K. Chorianopoulos, N. Chrisochoides, Making sense of video analytics: Lessons learned from clickstream interactions, attitudes, and learning outcome in a video-assisted course. Int Rev Res Open Distributed Learn 16(1), 260–283 (2015)
M.N. Giannakos, J. Krogstie, T. Aalberg, Toward a Learning Ecosystem to Support Flipped Classroom: A Conceptual Framework and Early Results. State-of-the-Art and Future Directions of Smart Learning (pp. 105–114) (Springer, Singapore, 2016)
P. Jermann, Technology for classroom orchestration. New science of learning (pp. 525–552) (Springer, New York, 2010)
A. Kleftodimos, G. Evangelidis, Exploring Student Viewing Behaviors in Online Educational Videos, 14th International Conference on Advanced Learning Technologies (ICALT), 367–369. (2014) doi 10.1109/ICALT.2014.109
L.F. Kwok, A vision for the development of i-campus. Smart Learn Environ 2, 1–12 (2015)
B.-C. Lee, J.-O. Yoon, I. Lee, Learners’ acceptance of e-learning in South Korea: theories and results. Comput Educ 53(4), 1–44 (2009)
E.W.T. Ngai, J.K.L. Poon, Y.H. Chan, Empirical examination of the adoption of WebCT using TAM. Comput Educ 48(2), 250–267 (2007)
M. Prince, Does active learning work? A review of the research. J Eng Educ 93, 223–232 (2004)
J. A. Ruipérez-Valiente, P. J. Muñoz-Merino, C. D. Kloos, An architecture for extending the learning analytics support in the Khan Academy framework. In: Proceedings of the First International Conference on Technological Ecosystem for Enhancing Multiculturality (TEEM '13), Francisco José García-Peñalvo (Ed.) (ACM Press, New York, USA, 2013), pp. 277–284. http://dx.doi.org/10.1145/2536536.2536578
S.B. Shum, R. Ferguson, Social Learning Analytics. Educ Technol Soc 15(3), 3–26 (2012)
G. Siemens, Learning Ecology, Communities, and Networks - Extending the classroom, elearnspace, last retrieved Jan. 21st, 2015 (2003), http://www.elearnspace.org/Articles/learning_communities.htm. Accessed 2 July 2016
A.I. Wang, The wear out effect of a game-based student response system. Comput Educ 82, 217–227 (2015)
H. Zhang, K. Almeroth, A. Knight, M. Bulger, R. Mayer, Moodog: Tracking Students’ Online Learning Activities. J Interact Learn Res 21(3), 407–429 (2010)
The authors wish to thank the students of the study who kindly contributed their time and effort. We also thank the reviewers for their helpful comments and recommendations. The paper is an extended version of the paper “Toward a Learning Ecosystem to Support Flipped Classroom: A Conceptual Framework and Early Results” presented in the 2nd International Conference on Smart Learning Environments (ICSLE’15), Sinaia, Romania. This work was partly supported from The Research Council of Norway (RCN)) under the project FUTURE LEARNING (number: 255129/H20).
All authors read and approved the final manuscript.
The authors declare that they have no competing interests.