Supporting the smart teacher: an agenda for the use of embedded sensing in novel learning spaces

Abstract

We converge the disciplines of context-aware computing, human-computer interaction and pedagogic research practice to propose an agenda for the use of embedded sensing for novel learning spaces. In this case, embedded sensing is the identification and analysis of in-the moment individual, group and class level behavioural data from students engaged in physical learning activities. Our work is motivated by the challenges and opportunities for teachers inherent in the rise of the design, development and evaluation of novel learning spaces augmented with multidevice technology. We present a framework for the use of embedded sensing, its relationship related and emerging work in the fields of social learning analytics and smart learning, and a practical illustration of SOLE (Sugata Mitra’s Self Organised Learning Environments). Our agenda addresses the conceptualization, data collection, and analysis of learning; zooming in on hard-to-identify individual-within-group learning processes. For the educational researcher, we propose a context-sensitive, dynamic and situated approach that can inform analytic frameworks and development of tools for sense-making. For the teacher-inquirer, the smart teacher, we propose that this approach directly addresses issues linked to the complexity of the ‘what and how’ of education-based evaluation and assessment of students in unstructured and multidevice learning spaces more broadly.

Learning in novel learning spaces

Recent years have seen a rise in the design, implementation and evaluation of novel learning spaces which incorporate a range of digital technologies from mobile devices (smart phones and tablets) to laptop, desktop and classroom scale computers and displays. These spaces offer new possibilities for students to experience the benefits of collaborative and individualised learning simultaneously. They are designed to foster learning opportunities which more closely mirror, prepare and equip students with lifelong skills. The broader formal recognition of this is reflected in PISA’s newly introduced Assessment and Analytical Framework (2015), specifically designed to assess “communicating, managing conflict, organising a team, building consensus and managing progress” (OECD, 2013, p.4). Collaborative problem solving skills form part of a growing agenda which seeks to embed twenty-first century skills as indicators of students’ proficiency in preparation for workplace and civic life, as well as “the ability to perform that collaboration using appropriate technology” (OECD, 2013, p.4).

From a research and a practitioner relevant perspective, this represents a move away from heavily instructional technology design approaches. Most prevalent in the 1980’s and 90’s as ‘expert’ adaptive and intelligent tutoring systems, technology is used with associated pre-specified learning analytic categories or assessment criteria based on “system of rules” from student modelling and uWiseser interaction studies (Wise, 2013, p. 1). In novel learning spaces, the notion that one can directly design for, and as a result, measure for and/or assess collaboration is not straightforward.

Rather than identifying and directing the pathways that students take, technology should facilitate the self-organising character of learning as students decide what is to be learned, when, how and where (Scardamalia & Bereiter, 2014). Such a move leads to a change in thinking about the organization of learning for educational researcher and practitioner. Rather than being a transmitter of knowledge, a teacher is viewed as a facilitator or coach. Although frameworks and general descriptions of learning goals and curriculum standards exist, there is little guidance on what specific skills are targeted (Darling-Hammond, 2011). This also puts a premise on the reliability and validity of assessment methods of skills and their impact (OECD, 2013). Moreover, Funke et al. (2007) state that any framework of assessment must take into consideration the technology, task and assessment context in which it is applied.

There is still a need for conceptual and methodological development on how learning takes place. This includes a research approach which makes the situated dimensions of learning (collaboratively created in the moment-to-moment dynamics of often unstructured learning) more visible to teachers and students. In this paper we converge the disciplines of context aware computing, human-computer interaction and pedagogic research practice to propose an agenda for the use of embedded sensing for learning analytics. Our focus draws on the expanding discipline of learning analytics concerned with the “measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the spaces in which it occurs” (Ferguson, 2012, p.305). For learning analytics, this offers a more holistic perspective than has been previously offered and can specifically address in educationally-relevant concerns related to the complexity of physical novel learning spaces.

What and why embedded sensing?

Embedded sensing offers a way of conceptualising, collecting, identifying and processing, through sensing, physical learning behaviours or traces at a micro level with a deep level of sensitively to context. Sensing is carried out in such a way that tools providing access to behavioural data are seamlessly embedded in the environment itself. Embedded sensing provides and filters information about learning behaviour as temporal, multimodal and created in the moment-to-moment dynamics of learning events. The notion of sensitivity to context is rooted in ethnomethodology, a term coined by Harold Garfinkel in the early 1970’s (Garfinkel, 1974). Roger and Bull (1989), describe it as a study of the: “ways in which everyday common-sense activities are analysed by participants, and [of] the ways in which these analyses are incorporated into courses of action” (p.3). Ethnomethodology emphasises two defining elements: a) an acknowledgment of social actors’ implicit skills and b) a focus on studying experiences in everyday life from their point of view. Underlying these features is the central assumption that people are continuously engaged in building and demonstrating their knowledge of the social world and their conduct within it. This led Sacks et al. (1974), for example, to make the distinction between “context-free” and “context-sensitive” (p.699) features of talk-in-interaction. Features which are general and open to a range of uses for a range of purposes relevant to the speaker are context-free; features which are produced in relation to the demands and relevancies of the local situation are context-sensitive.

The context-sensitive nature of learning processes and their relationship to embedded sensing is further supported by situated action accounts of system design in the Human Computer Interation field. Building on the seminal and widely cited work of Suchman (1987), a situated action approach studies the design of technological systems by explicating “the relationship between structures of action and the resources and constraints afforded by physical and social circumstances” (p.179). Button and Dourish (1996) advocate for a “technomethodological” approach to design (drawing on part context-sensitivity, part context generality) through the development of specific understandings and applications of both generalisation and abstraction in the study of human-computer interaction. They point out that the advantage of such as approach allows designers to work “with a set of sensibilities rather than with the details of specific activity” and to “align system design not so much with the details of specific working practices, as with the details of the means by which such working practices arise and are constituted” (p.19).

There are other ways in which a concept of embedded sensing has been applied to develop an understanding of learning processes. In the field of learning analytics for example, “embedded analytics” associate ‘embedded’ with automatic capture of pre-specified behaviour based on existing models of learning activity (i.e. the analytics are embedded in the design of educational technology systems) (Wise et al., 2013). In that case, sensor data is used more specifically as a trigger for feedback or ‘in the moment’ support.

The agenda for the use of embedded sensing of learning spaces proposed here brings a closer focus on enabling teacher rather than system-led local decision making and action taking about learning events. Our work is particularly relevant to unstructured learning spaces, that is say, ones which where there is a range of possibilities for different interactions with social (peers and the teacher) and material (technology) resources and affordances for information access, collaboration and output.

Background

Our work can be viewed as an application of learning analytics which builds on relevant and emerging work in the fields of social learning analytics and context-aware computing.

Social learning analytics

Social Learning Analytics (SLA) is an existing and growing subset of Learning Analytics. SLA acknowledges the central role that interaction and collaboration play in the quality of learning experiences and outcomes, so far studied in web-based settings. Theories of how learning takes place draw on Sociocultural, Social Constructivist and Cognitivist approaches, used to identify the tools which can both form the basis for the identification of analytics and the analytical frameworks. Our work draws on the Sociocultural strand of SLA research which focuses on the tools and contexts in the joint construction of knowledge at group and individual levels (Buckingham-Shum & Ferguson, 2012).

Social context analytics is one of five SLA categories formulated by Ferguson and Buckingham Shum (2012) relevant to sense-making around unstructured learner activity. It enables a closer focus on learning beyond the classroom to encompass the range of situated, informal, mobile, synchronous and asynchronous settings where learning can take place (Ferguson and Buckingham Shum (2012). Social context in SLA brings analytic tools which work towards making visible an understanding of these settings as learning contexts. For example, early work in context-aware computing developed learning analytic approaches based on context-specific objects and services in learners’ spaces to provide ‘right place right time’ feedback to learners (Buckingham-Shum & Ferguson, 2012). Previous systems such as MOBIlearn developed this to support learning activities through devices on the basis of tracking social as well as physical interactions (Syvänen et al., 2005). Social context analytics provide a way of operationalizing how “personal analytics” rooted in individual and beyond the classroom behaviours can be ‘socialised’ in collaborative learning contexts as indexes of objective data sources for learning (Buckingham-Shum & Ferguson, 2012).

Anastopoulou et al.’s (2011) work tracked individual student’s interactions with interactive technologies and demonstrates the potential for learning analytics in the context of formal science learning spaces augmented with technology. In Prieto et al’s (2016) recent work, individual level ‘teaching analytics’ in face to face classroom interaction is brought into the fore. That study focussed on the reliability of yielding usable analytics from the automatic extraction multiple data streams linked to teaching practice data via eye-trackers, portable electroencephalograms (EEG) sensing (a monitoring method to record electrical activity of the brain), accelerometers and audio and video data. The work demonstrates the potential for how context analytics have now begun to be applied to natural classroom settings and the building of statistical models to feed into the study of orchestration. The agenda set out in this paper complements this growing area in learning analytics but from the perspective of ‘student analytics’ in situations where it is harder to track non-teacher-led learning processes or teacher interventions. As a consequence, this moves learning analytics towards a more reliable and useful view of teaching and learning processes (as well as their orchestration) which are less the outcome of planned and structured classroom behaviours but of student-led, self-directed and collaborative learning.

Embedded sensing for sense-making

Identifying the ‘what and how’ in understanding unstructured learning processes, and from this, enabling assessment for learning is non-trivial. Our proposition is that the use and application of embedded sensing can play a role in supporting reflective practice and development of pedagogy around what knowledge and skills should and could be learned from the perspective of teacher theory-building. It can also support new forms of analysis beyond those achievable with existing (social) learning analytics and multimodal analysis.

Our previous work in this area has led to educationally relevant insights through context-aware systems which track situational and physical activities. Previous cross-disciplinary research in the design and development of task-based language learning pedagogy combined an evaluation of the contribution of embedded sensing and context-aware interaction with learning theory (Preston et al., 2015; Hooper et al., 2012), support for classroom-level multi-touch table technology integration and orchestration using interaction logging and visualisations (Kharrufa et al., 2013, Kharrufa et al., 2018) and the development of tangible technologies such as the Subtle Stone (Balaam, 2013). This moved the discourse around the role of learning analytics towards a more context-sensitive consideration of how we measure, collect and consider the application of embedded sensing insights, specifically by teachers and students.

Central to practice-based sense-making around embedded sensing is: a) the physical ‘instrumentation’ of activities (such as computer use) which allows us to explore learning processes and their associated assessment and b) the analytical questions they facilitate and answer when we consider the sensor data yielded as context-in-use (and context-sensitive). Figure 1 is a model which represents the relationship between the different levels of data processing and associated analytical queries our embedded sensing approach enables.

Fig. 1

Analytical processing and query model for embedded sensing

To begin, we have a range of data streams linked to embedded sensing which provide a digital representation of the environment. This provides first level preliminary processing for the data (analytical processing level-1). This enables us to answer basic queries that are, to a great extent, independent from the “learning design” and more reliant on the nature of the sensors used. For instance, who is doing what, who are working together, the time spent on each activity, etc.

The analytical processing (level-2) focuses on devising algorithms that answer analytical questions related to a particular learning design (put differently, this level focuses on meaning-making of the combined data streams with respect to the learning design goals).

Devising the algorithms for the level 1 progresses as we have more accurate technology, and more sensing or data collection affordances. To advance in the second level, however, we need to characterize and understand the behaviour in the unstructured environments. As Ferguson points out, this where we “build the strong connection with learning sciences” 2012, p. 13) (using qualitative reflective methods with the teachers or participatory analysis methods to work on the data from level 1 as visualization and reflection tools). So, teachers and educational researchers is essential to this operationalization of the embedded sensing approach.

The conceptual two-level separation is important because each level has its own research inquiries and methods, more quantitative in nature for level 1, and more qualitative in level 2.

Advancing in devising level-2 analytics should at some point feedback to learning design, pedagogical intent and assessment since it is grounded in the understanding for learning as it actually emerges in unstructured environments.

Embedded sensing for learning analytics in context: The case of a self organised learning environment (SOLE)

Educational context to SOLE

To demonstrate the application potential of our embedded sensing agenda, we outline its practical use in the context of a novel learning environment called a Self Organised Learning Environment (SOLE). A SOLE is a novel learning environment applied across many geographical and educational contexts. It is a space which sparks curiosity to learn through the exploration of Big Questions using the Internet and students’ organic drive to work together. In this space, as the teacher is a facilitator rather than transmitter of knowledge, SOLE can also be described as a minimal intervention teaching approach.

SOLE is both a pedagogical approach and a physical space where “educators encourage students to work as a community to answer their own vibrant questions using the Internet” (Mitra, 2014, p.7) (Fig. 2). Its universal methodological principles are to stimulate curiosity and engagement in learning content, a social and collaborative atmosphere and peer-interest.

Fig. 2

SOLE as a physical and conceptual space for learning

SOLE is not unique in its focus on developing the physical and conceptual space for learning with the inclusion of technology. Novel physical learning spaces across educational contexts are being increasingly augmented with mobile devices and other types of visualization displays by teachers, which are seen to offer new possibilities for collaborative and autonomous learning seen to promote motivation and confidence. As well as autonomy, such environments have led to a change in thinking about the organization of learning by teachers and students (Spector, 2014).

SOLE grows out of more than 15 years of research based on a series of experiments (the ‘Hole in the Wall’ experiments) with children in rural India by Prof Sugata Mitra (1999 to 2006). This research demonstrated children’s ability to learn independently using computers with internet connection (through assessment practices before and after different kinds of SOLE situations) and which involved various forms of mediation to support student learning (Mitra, 2012).

In SOLE based (2007–9) research project (Mitra & Dangwal, 2010) focused on designing, constructing and investigating 12 unique SOLEs in disadvantaged areas of Hyderabad and Sindhudurg, Maharashtra, India. The low quality of education in rural areas in India provided a strong case for the use of technology to enhance student access to learning. This research showed Tamil speaking children in a remote Indian village were able to learn basic molecular biology in English, initially on their own and later with a mediator without subject knowledge, and found evidence that these children could reach similar levels of learning as children in formal ‘advantaged’ schools. SOLE experiments conducted in Uruguay (Mitra & Quiroga, 2012) where almost every child had an internet connected laptop, research looked at children’s attempts to answer questions in groups, read beyond their expected levels in Spanish and English, and to see if children would read better in groups than individually. The questions answered were on curricular subjects the children had not yet been taught and were regarded by the teacher as ‘hard’ questions. Results found that the children involved performed ‘hard’ problems better in groups than they did individually and they were also shown to be capable of researching effectively using the Internet.

SOLE is not an isolated solution to access to education in the developing world but builds on research into children’s innate curiosity and ability to learn independently using computers and the vital role that technology plays in improving learning outcomes and quality of education. Today, this innovative pedagogy translates into models of enquiry-based learning in which students self-organise in groups and learn around a computer with minimal teacher intervention. The notion of minimal teacher intervention links to the enacting of the learner led processes in the SOLE but does not limited the role, expertise and craft of the teacher in the design for learning as well as its assessment and evaluation of the learning process.

Outside the rural India setting, the adoption of SOLE approaches worldwide (Mitra's TED prize, Mitra, 2013; School in the Cloud, 2018) by educators and children demonstrates that the SOLE concept has uptake as a viable approach for sustainable education across many contexts.

Mapping educational aims to learning gain

SOLE shares commonalities with collaborative, personalised and student-led learning (for example, in the seminal work of Rogers (1993); Dewey (1933)) but is distinguishable through its focus on the creation of the social, intellectual and academic space for learning to take place rather than prescribing specific teaching methods. In facilitating SOLE conceptual and physical spaces, these principles are all fuelled by adult encouragement and admiration but not by direct intervention. SOLE is therefore based around the teacher as facilitator of knowledge and learning content rather than its transmitter.

With the teacher as a facilitator, SOLE supports the learner-driven identification and reflection on what is, could and should be learned and how. The notion of ‘amplification’ is used to describe how the learning process takes place in self-organised groups (Mitra & Crawley, 2014). Learning to learn through SOLE and metacognitive thinking skills are an essential component in how it can specifically promote learner autonomy, engagement and a disposition towards long-term motivation for lifelong and lifewide learning.

In setting up a SOLE, there are some key elements which combine with these principles contribute to enabling self-organisation to take place:

• 1) Students are given one big question or choose from a selection

• 2) Students choose their own groups and can change groups at any time

• 3) Students can move around freely speak to each other and share ideas

• 4) Students can explore in any direction that they choose: there may not be one single right answer

• 5) Groups present what they have found towards the end of the session through a debrief

Connecting embedded sensing and SOLE

Theorising, designing, and developing learning analytics approaches for novel physical learning spaces with technology like SOLEs is an example of how participatory processes can be used with teachers on evaluating and assessing the learning experience. This includes the development of classroom-friendly platforms to support teacher design and orchestration of heterogeneous digital technologies in and across physical learning spaces. A significant challenge is that of identifying and assessing individuals in groups.

Mitra has spoken on numerous occasions on the science behind SOLE as a self-organizing system, “a concept that comes out of maths and physics which is that if you allow a system to be chaotic then, under certain circumstances, you get spontaneous order” (Mitra, 2015). The challenge of self-organizing system is that it doesn’t actually exist in a state that can be ‘empirically’ probed. This becomes clearer by comparing the self-organizing system of a SOLE to the study of systems in the field quantum mechanics, where an unresolved problem in the nature of measurement is that it is impossible to measure a system without disturbing it, and even when disturbed, it is hard to locate the point at which this occurs (Mitra, 2014).

From the perspective of research and practice, SOLE represents a challenge in how learning processes are conceptualised and studied in educational technology research, how learning is assessed and assess-able as well as how teachers can reflect on their growing appropriation of learning design and “pedagogical intent” (Lockyer et al., 2013, p. 1439).

Qualitative observations (what they refer to as “learning effect”) reported in Mitra and Crawley (2014, p. 87) small-scale project on the Gateshead Experiments based on SOLE provide a useful starting point for taking practitioner-relevant observations and an initial framework for identifying the added value of embedding sensing. At the level of individual-in-group learning processes in SOLE which embedded sensing can make visible, and subsequently available for the teacher-enquirer, the most relevant learning behaviours concern group processes: role taking, movement, flocking and sound. Together, these effects are characterised in terms of “intellectual ‘amplification’” and explanatory factors of “how the learning process in a SOLE works” (p.87). Importantly, Mitra and Crawley point out that these observational contexts are attributed to the construction of whole-group processes which are inextricably self-organized and as a consequence, unstructured. For the purposes of this demonstration, we focus on the interchangeable role adoption and adaptation in individual-in-groups:

“In a group one child operates the computer, another takes notes, a third directs the other two, while a fourth entertains, and often disturbs, the other three. They take turns at each of these roles” (p.87). Whole-group processes include the interchangeable role adoption and adaptation in a group concerning a number of forms, namely, computer user, note taker, director, entertainer and disturber. The notion of an individual in a group is highlighted but not as a static construct.

In this demonstration, embedded sensing is used to address the challenges of documenting and assessing SOLE-based learning using single wrist-worn embedded sensors designed to develop specific activity recognition of writing, typing and touching (i.e. scrolling and tapping). In this way, we have a direct inroad into the possible roles learners adopt and adapt when using different tools and technologies available to them.

Activity recognition is “the ability of the intelligent system to deduce temporally contextualized knowledge regarding the state of the user on the basis of a set of heterogeneous sensor readings” (Pecora & Cirillo, 2009). Our work on activity recognition forms part of larger body of research around wearable computing to build computational systems which can capture behavioural data ‘in the wild’ to enable more objective and accurate pictures of practices in education and beyond (Ploetz, 2010; Khan et al., 2015; Khan et al., 2016; Miu et al., 2015; Wilkinson et al., 2016).

Using the data from sensors we are working towards being able to reliably and unobtrusively identify both actions and actors within unstructured learning activities. In the first phase of this work reported here, we are using sensor data from SOLE-based learning to later build complex algorithms for activity recognition in the three areas: hand-writing, typing and touching (i.e. scrolling and tapping).

We use accelerometer as a wrist worn device or embedded into a wrist worn device (Fig. 3). An accelerometer measures the acceleration in a three-dimensional space of a moving body. Acceleration is the rate of change of speed. The data collected by these accelerometers are then used to train and test an algorithm to recognize specific learning activities. The IMU (inertial measurement unit) uses the accelerometer and the gyroscope (and sometimes a magnetometer) in these devices to measure the linear acceleration and rotation of the wrist. Such sensors are now commonly worn by people (with varying degrees of activity-recognition) as they are present in smart-watches (e.g., apple watch) and activity trackers (e.g. FitBit) making this approach practical to everyday classroom life. The IMU (inertial measurement unit) sensors in these devices enable the measurement of linear acceleration and rotation of the wrist. The wrist provides many trackable properties that can be tracked and used as input parameters to recognise handwriting, typing and touching (i.e. scrolling and tapping). We use the Open Movement WAX9 IMU platform (Open Movement WAX9, 2018) which includes a 3-axis accelerometer, gyroscope and magnetometer as well as a Bluetooth-compatible radio. Together, these sensors allow for an accurate representation of the state and movement of a user’s wrist to be calculated. For this SOLE-based based illustration, we only used the data from the 3-axis accelerometers.

Fig. 3

Wrist-worn sensors

Architecting amplification

From the point of view of the smart teacher of SOLE-based learning, there are a number of challenges in conceptualizing and making visible how learning happens as it focusses at the level of group/s rather than individual participation. Table 1 shows the analytics opportunities in building a picture of the architecture of SOLE from the perspective of data capture method and analytic application for sense-making for individual-in-group participation. The notion of an architecture acknowledges the complex nature of unstructured learning spaces like this whilst at the same time moving towards a characterization of a context of it which is socialisable in terms of analytic, coherence and ultimately for the teacher, orchestration. Moreover, whilst from the perspective of the learning process, the group is seen the unit of analysis, context-aware technologies are designed to recognize activity in terms of the users (i.e. the wearer of the wristband) and its application (i.e. the wrist worn sensor) thus enabling a view of the individual in context.

Table 1 Context-aware data capture and its relationship to learning analytic application

Using data collected from a wrist-worn accelerometers, we can calculate the amount of time spent performing activities that involve a specific hand (Fig. 3). In turn, we can determine the chronological order of activities. By combining this physical activity data with activity data inferred from other sensors and other sensing modalities, we can make education-specific observations that relate spoken and physical behaviour with digital learning content (i.e screen recordings and application logs such as web searches) to enable video indexing. Within reason, wrist-worn sensors could allow for the identification of almost any physical activity (including those using tangible objects such as Lego), provided sufficient data is available.

Table 1 outlines the context-aware data capture from wristworn sensors for hand-writing, typing and touching activities and a summary of the SOLE analytic application of these data are matched to the relevant activity recognition. In the second half of the table, are a number of other data streams which could be coupled with the current sensing of hand-writing, typing and touching activities. 360 degree video also served as a way to annotate data collected by the wrist-worn sensors to develop an algorithm to then recognise that particular activity in real-time as well as reflective sessions with a teacher to explore the learning analytic application (as demonstrated in the discussion section).

In future work, we intend to integrate wrist-worn bluetooth-enabled GPS, audio sensors, 360 degree video and screen capture of computer use including internet navigation into our study of novel learning environments like SOLE to enable a complete picture in the development of a unified model platforms for teachers. They are included here to demonstrate the potential of the merging of multiple data streams.

A practical illustration from the SOLE code café

As part of EU code week, a SOLE Code café was run at a UK university which involved a day of 6 consecutive SOLEs to help young people learn about computational thinking. There was no specific incentive for the young people’s participation other than a fun out of school club experience. Written informed consent was obtained from the parents and guardians of the participating children.

Twenty-one young people (11 females and 10 males) participated in the SOLEs with an age range from 8 to 15 years of age, 11 (8 females and 3 males) of the 21 participants were asked to volunteer to wear wristbands. The cafe took place in a regular teaching classroom, where the tables are re-arranged and joined to form four big tables (takes up to six children each) distributed in unstructured manner. Each table had a laptop connected to the Internet. There was one large flipchart on which a “Big Question” was displayed after an introduction to stimulate curiosity by the facilitator. This is typical of SOLE-style physical set-up. The reduced number of computers and the shared board were designed to encourage collaboration. The children were free to work with whomever they chose and could change groups at any time. A sharing and collaborative attitude was encouraged and the children were reminded of this at the start of each SOLE.

We, the research team (4 members),were present in the café as participant observers. One team member with experience in SOLE acted as the lead facilitator and other members acted as assistant facilitators. We explained the purpose of the data collection process and the need to install the recording equipment. We were keen not to significantly influence the children’s behaviour by our presence or data collection equipment, but rather normalize its use. We felt the children eased with us quite quickly.

The Big Questions used in the SOLEs were developed to support the participants incrementally. In the first two SOLEs respectively, we asked the participants: What is a thought? What is thinking? In the second session, a further Big Question emerged from the children: Can a computer think for itself? The children had the option of producing their answers using pen and paper, MS Word, or MS PowerPoint applications and exploring in direction they wished. The third SOLE proposed the concept of ‘metaphors’:What is a computer bug? The children were encouraged to produce the answer to the question in LEGO bricks. Inspiration for this was also informed by Mitra’s early work on using the notion of computer bugs in a minimally invasive educational environment to teach programming. The fourth SOLE, we decided to challenge some of the perceptions around the capability of computers: What is the P vs. NP problem? Participants explored the question with only the Internet made available to them. We also integrated a classical graph colouring problem as a practical example for P and NP problems. The fifth and the sixth SOLEs focused on ‘making the computer of the future’ (the hardware and software), one that could solve the P vs. NP problems using LEGO bricks.

Activity recognition results over six SOLEs

Figures 4, 5, 6 and 7 bring the analytic situation (as previously modelled in Fig. 1) into focus to show how an understanding of context facilitated by embedded sensing can lead to a more holistic evidence-based understanding of the learning situation in a SOLE. To demonstrate how we might move from data to design-dependent learning analytic application in SOLE, Figs. 4, 5 and 6 show graphic representations for the activities of hand-writing, typing and touchpad use at three levels of the learning situation: SOLE (in other words, the wholeclass), group and individual in terms of total time, activity time and distribution (proportion of actions sensed) together with a description linked to the proportion and distribution of time spent performing the activities.

Fig. 4

SOLE level Activity leading to design-independent analytic situational queries

Fig. 5

Group level Activity leading to design-independent analytic situational queries

Fig. 6

Individual level Activity leading to design-independent analytic situational queries

Fig. 7

Hand-writing against time per student in one group

SOLE analytic situation

Figure 4 shows the frequency of sensor identifiable activities of handwriting, using the touchpad and typing across the 6 SOLE sessions of approximately 1 h each (0 = SOLE 1 and 5 = SOLE 6). We observe that most handwriting, touchpad use and typing activity occurred during the first hour and decreased until the fourth SOLE, then trailed off and stopped completely at the start of the sixth SOLE session with a slight increase once that session had started. This can be considered alongside the proportion of total time spent on handwriting, touchpad use and typing activity across 6 SOLE sessions. We observe that handwriting, touchpad use and typing activity constituted less than half the total time spent across the 6 SOLE sessions. The rest of the time was spent on ‘other’ – activities other than writing, typing and scrolling/tapping (this could be discussion, lego-building and so on). Additional context-aware sensing has the potential to reveal additional complementary data streams to enable a more holistic view of the students’ participation as discussed in relation to Table 1. In the proportion of total activity time spent on each activity, across 6 SOLE sessions, we observe that out of the three activities under study here, most time was spent on handwriting, followed by touchpad use and finally a small percentage of activity on typing.

Group level (3–4 students per group based on data from 2 groups) analytic situation

Figure 5 shows the proportion of total activity time (14% of 6 h) spent on each activity in 2 groups. Here we observe that as compared to Group #4, Group #2 spent a larger proportion of time using the touchpad and typing. Group #4 on the other hand, spent a larger proportion of time on hand-writing. We also observe that Group #2 spent a larger proportion of time than Group #4 on the activities of handwriting, touchpad use and typing as compared to the rest of the non- sensor identifiable activity time in their SOLE groups.

Individual level analytic situation

Figure 6 shows the different views of Groups #2 and #4 of distribution of individual activity time spent on handwriting, touchpad use and typing per student. These two complementary views can support understandings of how engaged students are in SOLE-specific activities linked to technological devices (in this case internet searches and PC use) (Group #2) and understandings of how engaged students are in SOLE-specific activities alongside other possible non-sensor based activities (like lego building, discussion and possibly off-task behaviour) (Group #4).

For Group #2, Fig. 6 shows the distribution of the students’ total time performing handwriting, touchpad use and typing across 6 SOLE sessions (i.e what the students did most in terms of these activities in total, therefore 100% of sensor-specific activity time). It therefore ignores any time spent not performing any other activity. It shows how each student in Group #2 distributed their activity time. In terms of all 6 SOLE sessions for Group #2, the highest distribution of Student 10’s time was spent using the touchpad as compared to Student 3 whose highest distribution of time was spent on hand-writing activity.

For Group #4, Fig. 6 shows the proportion of total time spent on each activity per student in relation to other non-sensor based activities. The highest proportion of sensor-based activity time was 9% and we therefore also see the proportion of total time spent on each activity per student. As this also takes into account the time spent not performing one of these activities, they are in proportion to each other. For example, here you can see that Students 10, 3 and 2 are those students who spent most of their time performing one of the three activities of handwriting, using the touchpad and typing. Student 10 spent most of their time on using the touchpad, Student 3 spent most of their time hand-writing and Student 2 spent the third most time performing of the activities and a more or less equal time on one of the three activities. Student 7 spent proportionally less time but a more or less equal amount of time shared between hand-writing and touchpad use.

In this demonstration interchangeable role adoption can be probed at individual, group and SOLE level in terms of total and distribution of proportion of time spent on hand-writing, touchpad use and typing.

To examine this even further, we might consider the moment-to-moment dynamics of this process to compare individuals in interaction with each other. A visualisation of the distribution of activity time spent on handwriting at the SOLE level indicates that hand-writing constituted the highest proportion of total activity time spent on each activity, across 6 SOLE sessions. Figure 7 shows a visualisation of what can facilitate enquiry into the roles of note-taker as hand-writing activity for one group engaged in SOLE learning processes. Here, we see the interchangeable activity of hand-writing alongside other activity.

This sequential approach based on an analytical processing and query model for embedded sensing as part of our agenda can be the basis for the development of tools to support a number of ways into the data at the level of both design-independent queries (which may of be specific interest to the multimodality researchers to address issues raised in scope, reliability and and breadth of analysis) and design-dependant queries (of more interest to the smart teacher).

Discussion

Learning analytic application: Questions from the smart teacher

Explorations of these graphic representations provide the basis design-independent queries around the overall question of ‘who did what, where and when?’ (time allocation on activities, chronological order of activities, other types of physical activities which accompany SOLE relevant learning activities not linked to devices). For the smart teacher, these can be further broken down into smaller questions around the use of SOLE-based sensor identifiable activities such as who did what? How often did they do it? When did they do it? They also show how it is possible to divide the capacity to apply these queries analytic situation into different planes of activity (drawing on Dillenbourg et al., 2010) across individual, group and classroom/SOLE. Who did what at the level of the group? Where was activity most prominent?

Making the link to design-dependant queries involves considering the above context-aware questions in light of situational factors relevant to SOLE (such as those included in ‘intellectual amplification’) and the application of the findings and visualisations from the data streams as learning analytics more specifically. An important measure of the impact of embedded sensing for learning analytics in a novel space like SOLE is the extent to which design-independent queries can provide evidence-based enquiry into practice-based questions on “how the learning process in a SOLE works” (p.87).

As part of our work with SOLECODE, we worked with a primary school practitioner to further probe the nature of the practice-driven questions smart teachers might ask and the kinds of representative observations they would make when engaged in facilitating a SOLE. These broader questions would be answered by focussing in on the design-independent queries as mentioned above. In this way, the smart teacher engages at a micro-level on the activity data to answer the higher level questions they actually have when engaged in practice. To that end, we invited the teacher to take and observe a number of photographs from a SOLE and think-aloud about relevant questions he would normally ask. The following were offered:

• 1. Is everyone is engaged? Is anyone sitting back doing nothing? Are they are all focussing on what they’ve decided their job is be it LEGO, be it researching, be it writing something. Is the the whole class is fully engaged?

• 2. Are they all collaborating because they are all focussing in on the one piece of work

• 3. Is there anything I can do support if there is anything going wrong, be it technical with the laptop or be in a misconception?

• 4. If there are no adults present around a group in a SOLE and there is lots of different activity, are the children are still absorbed in what they are doing are they still engaged and still motivated?

• 5. How do I know consolidation is taking place? Does using the word processor mean that the students are starting to consolidate their learning and reformat it into their own understanding?

• 6. How can I gauge wholeclass engagement? This is important as a teacher it tells me that the SOLECODE session has been pitched at the right level, that the children understand, that the activity is suitable for them, that they like it and are enjoying it. It confirms to me that I’ve picked nice resources and ones which are accessible to them and that the Big Question is not too difficult because they are not sitting back doing nothing.

• 7. Which students haven’t really worked as part of the group - have taken their part, created something with lego and brought it back? but I think his understanding of the whole idea was not as good as the team together. I’d like to try to engage students who don’t engage a little bit more.

• 8. Even though students are engaged in creating, I ask myself do they fully understand the activity and can they explain their computer in terms of the Big Question? Can they answer questions can they go further to explain their learning -this is a new area I would like to focus in on.

How we might go about supporting the smart teacher to answer these questions not only means developing the tools to measure learning behaviours in novel learning environments as we have illustrated in this paper but also providing the necessary infrastructures for practical application. In recent years there has been a proliferation in the use of classroom dashboards to support the smart teacher. We propose that our agenda for embedding sensing can be implemented through the use of teacher visualization dashboards and notifiers (of general behaviour trends), Class level dashboards and notifiers (of general behaviour trends), Group and individual dashboards. In addition, whilst a novel environment like SOLE is not designed to be tightly controlled in terms of steps, behaviours and processes (as these are inherently self-organised), learning analytics can lead to reminders to students about the type of activity they need to be engaged with at this level of the process (linked to hand-writing, touching and typing). For the teacher, these learning analytics can also provide a lens on behavioural engagement and provide potential prompts to them to facilitate when engagement may appear to be lost. The reports yielded from embedded sensing data can also provide learning analytics to encourage reflection on the process of a SOLE for both teacher and student based on how the SOLE actually played out, whereby the teacher might want to think about and apply a new design feature in the activity to prompt different types of behaviour/collaboration.

Conclusion

In a review of the drivers, development and challenges of learning analytics, Ferguson identified “building strong connections with the learning sciences” and “developing methods of working with a wide range of data-sets to optimise learning environments” as two of the most important challenges facing the learning analytics field (2012, pp. 12–13). This is not just an issue for the learning sciences but also those working with the development of learning technologies. As Antle and Wise argue, here, learning theory is used “at a very broad level” without specific links between how technological affordances mediate learning processes (2013).

Based on our agenda, we see embedded sensing contributing in two key ways to building strong connections. Embedded sensing for learning analytics can help build a more reliable and systematic picture of novel learning spaces by enabling empirically-driven theory building on how learning takes place within theories of multimodal communication. Embedded sensing also provides a way of considering how technological learning artefacts bring new opportunities for the achievement of ‘distributed intelligence’ (Pea, 1993). Notably, as we are specifically concerned with unstructured and ‘difficult to measure’ learning, this moves theorising around technology in the classroom away from existing theories of ‘distributed cognition’ which seek and measure learning processes in terms of individual-in-context learning processes, more reliant on specifying apriori the objects and representations of these objects. As Pea also points out, “people, not designed objects, “do” cognition” (1993, p. 50). As our demonstration shows, for SOLE, embedded sensing provides a way of identifying analytic constructs to enable the exploration of learning-relevant phenomena and in the future, theory-building.

The challenge of “developing methods of working with a wide range of data-sets to optimise learning environments” in the context of embedded sensing for learning analytics is directly addressed by the potential to involve teachers and students as inquirers into the context-sensitive considerations of their own classroom learning analytics. This echoes Balaam’s (2013) work who extends on the notion of practitioner-theory building to propose that context-sensitive and situational understandings of classroom technology orchestration can also come from considering the classroom itself as a context for design.

Our agenda also addresses wider concerns raised by Selwyn (2000) on the need to ask wider questions through methodological innovation in education computing research. A key area for us is Selwyn’s argument to expand the scope of quantitative analyses to provide more large scale pictures of education technology and strengthen qualitative research approaches with more empirical account of “what does happen (as opposed to what has apparently happened or what could happen) when computers are used in educational settings” (Selwyn, 2000). As regards establishing a more empirically-driven account of what happens when computers (and other technologies are used in the ecology of the classroom), the work in this paper has already started to address the strengthening of qualitative approaches. Considering Selwyn’s calls for larger scale studies, to our knowledge, there is very little investigation into the common measures that could be used across multiple contexts for ‘big data’ on unstructured learning with different types of sensing.

In future work on learning analytics, we look to develop these measures drawing on the processing of embedded sensing. We have already begun to apply other ways to measure SOLE-based learning and other novel learning environments which provide an alternative to sensor-based measures too based on a teacher-friendly observation tool to be used during the SOLE itself (Kharuffa et al. 2017). Having these two lenses and comparing the outcomes of teacher questions ‘in the moment’ in a non-writstband instrumented environment and those based on activity-recognition after the event can also function as an experimental context to measure the impact of the learning analytic application using embedded sensing. Ultimately, the learning analytic application needs to feed back into how the teacher sets-up, plans, orchestrates and facilitates in the novel learning space. We think that both approaches can play a key role.

Therefore, in keeping with the belief that technology should not be used to ‘do’ things to smart teachers and students, we plan to further involve teachers as the lead sense-makers of embedded sensor data-sets and their visualisations - as tools for reflection and learning design on the future of technology in their own classrooms.