Evaluation of smart classroom from the perspective of infusing technology into pedagogy

Abstract

Smart classroom was regarded as one of the essential methods to transform the traditional instructional mode. The research and practice of smart classroom became popular since 2012. However, there were few research on the large scale survey of smart classroom from the both technological and pedagogical perspective to provide baselines for smart classroom building. In this study, a large-scale survey of smart classroom was conducted in China. A total of 13, 495 students from 135 primary and middle schools participated in the survey. Data were analyzed in SPSS 23. Results indicated that participants generally did not perceive much smart learning in classrooms, especially in the dimension of resource and enhancement. The results also confirmed that digital device and Internet were the basis for equipping smart classroom. Technology equipment plus advanced technology could lead to the success of learning. Future research could be conducted to investigate how to match technologies, pedagogies and smart classroom to improve student’s achievements.

Introduction

Since the late 1990s, many governments increased their investments in educational technologies, with the assumption that use of technology in schools could enhance teaching and promote learning outcomes (Huang et al. 2010). However, since the beginning of the investments, some researchers argued the effectiveness of technology in classroom. Researchers had also seen the modest use of technologies without significant influence on teaching and learning in most schools and classrooms (JISC 2009; Cuban 2009; Ramsden 2011). Organization for Economic Co-operation and Development (OECD) released a report with the conclusion that investing heavily in school computers and classroom technology did not improve pupils’ performance (OECD 2015).

The modern classroom started at the end of the sixteenth century when classroom teaching was invented, with instructors lecturing from raised platforms and pupils sitting at fixed desks many rows deep (Song et al. 2014). Rows of seats, instructor front and center, student eyes trained on the teacher, this classroom model worked well for centuries. However, traditional approaches were ineffective for today’s needs (Finch 2018).

Now most classrooms in China remained the same layout as when they were invented (Yang et al. 2013). Technologies in classroom shifted with time. Since chalkboard was introduced into classroom in 1890, technologies such as film strips, overhead projector, desktop computer, interactive white board, smart phones and tablets were gradually used in classroom. With the use of these technologies, teaching in classroom has changed from “blackboard & chalk” mode to the “computer & projection” mode. Although technologies use in classroom had enhanced teaching to some extent, many predicaments still existed in today’s classroom.

First, multimedia console was fixed in front of the classroom, which limited the flexibility of teaching. In classroom, teachers were always busy with operating computers, with few gesture interactions or eye interactions with students (Yang and Lin 2009). Second, the unified and fixed classroom layout enhanced the didactic pedagogy, but hindered teacher’s adoption of student-centered pedagogy. Research showed that classroom with the specially designed layout such as “X-shaped” or “round” shape could meet the need of student-centered pedagogies (JISC 2009). Third, the misuse of slides in multimedia classroom hindered student’s knowledge processing. Slides used by teachers in classroom were normally filled with texts, with no figures, tables or multimedia materials. It was easily for students to distract their attention from slides where the learning contents were listed (Huang et al. 2012). Last, big gaps existed in teacher’s technological pedagogy knowledge and the needs for using emerging technologies in classroom. For example, interactive white boards (IBW) were equipped to lots of classrooms, however, most of them were used just like a projector screen (Huang et al. 2012).

To some extent, the plight of the technology-rich classroom had a close relationship with the design and equipment of the classroom environment. Acknowledging the challenges of technological use in classroom, scholars argued the need of shifting attention from technology and software and learning activities design in smart classroom (Simsek 2005; Aguilar et al. 2015).

The phrase smart classroom had been used since 1995 in San Diego State University when they built the first smart classroom with the aim to enhance learning in big classroom by integrating technologies, like clickers, sympodium, multichannel audio system, etc. (Frazee et al. 2006). In the following years until 2012, researchers investigated various technologies, like multimedia communicational supporting platform (Shi et al. 2003), Ambient intelligence (Augusto 2009), Internet of things (Temkar et al. 2016), etc. to make either physical classroom or virtual classroom smart. In this period, not many research could be found on smart classroom, and smart classroom was not well defined. The existed research of smart classroom were mainly investigated from technological point, however, few studies focused on the pedagogical aspect.

Huang et al. (2012) defined smart classroom as a physical classroom space that was effective for showing teaching content, easy for class management, convenient for accessing learning resources, easy for instructional interaction, and combined with contextual awareness. Although the definition started to integrate the pedagogical issues into consideration, few follow-up researches were found on the design and evaluation of smart classrooms from both pedagogical and technological issues.

Since 2012, smart classroom implementation “is mainly based on active use of mobile technology, learner mobile devices and automatic communications” (Uskov et al. 2015). Now in a smart classroom, mobile devices are owned by students so that all students are able to contribute and become active agents (Jahnke et al. 2017). More and more research on smart classroom/ smart learning environments emerged since 2012. However, pedagogical changed such as innovation in the teaching role, or the new role students take, the role of the contents, who decided on the learning process, resources, etc., was usually overlooked (Bautista and Borges 2013).

In April 2018, China Ministry of Education issued the “The action plan of ICT in Education 2.0”, which emphasized the importance for building smart learning environments to promote smart education (China Ministry of Education 2018). Lots of provinces also announced their local action plans to build smart classrooms for smart learning, like Zhejiang Province and Fujian Province. However, few study revealed the pedagogical issues of smart classroom, especially from the large scale survey perspective. Therefore this study aimed at investigating smart classroom from the combined perspective of both pedagogy and technology through a large scale survey to provide information and reflections on the building and application of smart classroom.

Evaluation of smart classroom

In order to evaluate smart classroom, a large scale survey was conducted in 135 primary and middle schools in China, with 13, 495 students participated in the survey. Data were collected by using the Likert-type smart classroom evaluation questionnaire (SCEQ) that should be illustrated in section 2.1.

The framework and tools for evaluating smart classroom

Few research focused on the evaluation of smart classroom. Li et al. (2015) developed an inventory for smart classroom included the 10 factors of physical design, flexibility, technology use, learning data, differentiation, investigation, cooperation, students’ cohesiveness, equity and learning experience, with the 640 participants survey. MacLeod et al. (2018) developed an instruments for understanding students’ preferences toward smart classroom included the 8 factors of student negotiation, inquiry learning, reflective design, connectedness, ease of use, perceived usefulness and multiple sources, with 507 students participated in the survey. Yang and Huang (2015) developed classroom environment evaluation scale (CEES) evaluating both physical classroom environments and psychological classroom environments, from the 10 dimensions of showing, manageable, accessible, tracking, enhancement, teacher support, involvement, investigation, task orientation and cooperation. Adapted from the 5 factors of physical environment from Yang and Huang (2015) and considering the the inventories by Li et al. (2015) and MacLeod et al. (2018), we created the five dimensions framework for evaluating smart classroom (see Table 1).

Table 1 Framework for evaluating smart classroom

Based on the framework, the smart classroom evaluation questionnaire (SCEQ) with 24 items and 5 dimensions was developed (See Appendix). The survey items were answered by means of a Likert-type scale with five response choices, including “1 = almost never” “2 = seldom,” “3 = sometimes,” “4 = often,” and “5 = almost always”.

Participants

The SCEQ was used in a large scale survey on the status of smart classroom with 13, 495 students from 135 primary and middle schools in China. 12, 514 questionnaires were collected. Data were analyzed in SPSS 23.0. Students from Year 3 (8–9 years) accounted for 32.2.6%; students from Year 5 (10–11 years) accounted for 31.0%; students from Year 8 (13–14 years) accounted for 22.2%; students from Year 11 (16–17 years) accounted for 14.5%, as shown in Table 2.

Table 2 Sample characteristics

Results

For each scale of SCEQ, the Cronbach α ranged from 0.778 of presentation to 0.867 of resource, indicating that the internal consistency of the responses was acceptable for evaluating smart classroom in the 5 constructs. Exploratory factor analysis by principal component analysis with varimax rotation showed that the 5 factors of resource, environment, enhancement, management and presentation accounted for 64.405% of the total 24 variances, as shown in Table 3.

Table 3 Factor loadings and Crobach’s alpha values for the five factors

Table 4 outlined and summarized the average scores and standard deviation for the five factors. As shown in Table 4, “Resource” and “Enhancement” were the two factors with the lowest scores, indicating the overall technology enhanced smart learning were not much perceived by students in classroom. Most students perceived that they could not get access to Internet or digital learning resources freely, and also the assistance from digital devices for learning. “Environment” was the only factor with score above 0.4, indicating students generally satisfied with the temperature, air and light in classroom. The big standard deviation of resource and enhancement indicated that there could be differences for students’ perception of the smart classroom in the five factors.

Table 4 Average scores on the ten scales extracted out on CEES

In terms of the classroom types, 765 students (6.1%) expressed they usually studied in traditional classroom with no computers, with 7603 students (60.8%) in multi-media classroom, 419 students (3.4%) in tablet classroom, 482 students (3.9%) in computer classroom, 3228 students (25.9%) in interactive whiteboard classroom. One-way ANOVA method was used to verify whether there are obvious differences among the different kinds of classrooms (see Table 5). The results showed that there were significant differences (p < 0.001) in different kinds of classrooms for all the five factors.

Table 5 One-way ANOVA for different kinds of classrooms

Students perceived best presentation and enhancement in tablet classroom, however, the mean score was still under 0.4, which indicated that students did not perceived effective technology enhanced learning even in tablet classroom. From Table 5, students usually studied in traditional classroom with no computer scored the lowest among all kinds of classrooms in all factors.

An independent-samples t-test was conducted to investigate if there were significant differences in each scales of SCEQ between no Internet access classroom and free Internet access classroom. The independent samples t-test was significant for all the subscales (p < 0.01) as shown in Table 6. Students in free Internet access classroom generally perceived better smart classroom environments than that in no Internet access classroom.

Table 6 Comparison of students’ perceptions of classroom environments with no Internet access and free Internet access

Independent-samples t-test was also conducted to test the difference between classroom with no devices used and classroom with devices in used every class. The independent samples t-test was significant for all the subscales (p < 0.01) as shown in Table 7. Students perceived significant better classroom environment in classroom with devices used in every class. With digital devices used in every class, the score of presentation is 4.341, indicated that students could share learning outcomes with peers. However, the score of student’s perception of resource was still under 0.4 even devices was used in every class, indicating resources for students in classroom was not abundant.

Table 7 Comparison of students’ perceptions of classroom environments for devices and PPT used

Discussion

This study investigated smart classroom by a large scale survey, from the five dimensions of resource, environment, enhancement, management and presentation. Exploratory factor analysis showed that all the 24 questionnaire items loaded on the 5 factors, with 64.405% explanation of the total variance, indicated the reliability of the questionnaire. Resource dimension emphasized on the convenience for accessing digital learning resources, for example, the Wi-Fi and 1:1 tablets could be used for this purpose. Environment emphasized the convenient level for the indicators of the physical environment, like temperature and air condition. Enhancement emaphasized the facilitation of teachers for adapting new technology in smart classroom environment. Management emphasized on the facilitation of student-teacher and student-student interaction, for instance, the U-layout or X-layout could be used for this purpose. Presentation dimension emphasized on the convenience for sharing learning materials from both teachers and students, for example, the dual screen and wireless display could be used for this purpose.

The findings from the survey indicated that the participants generally did not perceive much technology enhanced smart learning in their classrooms, especially in “Resource” and “Enhancement”. Lots of students could not get access to digital learning resources and share these resources with their peers. Lots of students did not perceive technology enhanced teaching and learning even in tablet classroom, which revealed that the building and applying of smart classroom were only in the primary stage and there was much work need to be done. Pedagogy fit should be a key issue for the application of smart classroom. Technology equipment plus enhanced pedagogy could produce behavior change (Yang et al. 2017).

The independent-samples t-test showed that students perceived significantly better in classroom with free Internet access and digital devices used in every class, which indicated students gained better learning experience in classroom with digital devices and free Internet access. Today’s learners were grown up with digital technologies and Internet, and they had very different characters from the previous generation of learners (Howe and Strauss 2000; Prensky 2001; Gasser and Palfrey 2009). Teo (2013) identified the 4 characters of “grow up with technology”, “comfortable with multi-tasking”, “reliant on graphics for communication”, “thrive in instant gratification and rewards” of the new generation of learners. Smart classroom should meet the learner’s needs (Koper 2014). Digital devices and Internet access were the basis for conducting high order cognitive learning activities for digital native.

For management, the means score of students’ perception was 3.895(the best experience in computer classroom with the score of 4.040), which indicated that the workspace and layout should be improved for digital native students. In fact, 10, 815 (86.4%) students expressed they were sitting at fixed desks many rows deep, indicated that most of the classroom still had the rigid classroom layout with no flexibility for teaching and learning. This result confirmed the conclusion by Yang et al. (2013) that most classrooms in China remained the same layout as when they were invented. Classroom layout should be flexible to accommodate the new learning/teaching trends and to respond to the impacts of massive open online courses (Alwash et al. 2014).

For environment, the mean score of students’ perception was 4.051 (best experience in interactive whiteboard classroom with the score of 4.136), which indicated students generally satisfied with the condition of air, light and temperature in classroom. The item of environment evaluated the basic condition for students’ learning in a classroom. If the mean score was too low, it should be warned that the environment might hinder student’s learning or cause some bad effects. It was suggested the baseline for the mean score of environment factor should be 3.8.

For presentation, the mean score was 3.694 (the best experience in tablet classroom with the mean score of 3.874), which indicated that student did not perceived they could conveniently share learning outcome by using technology. However, the presentation and sharing students’ learning results played an essential role for student’s perception in classroom climate and therefore influence their learning. In fact, research has revealed that presentation methods will affect student’s cognitive results (Moulton et al. 2017), and clear digital presentation could enhance retention and promote learning effects. Future classroom optimization should consider using technology to facilitate student’s presentation and sharing of learning contents. In the design of smart classroom, presentation could be enhanced by dual-large screen, high resolution, wireless display for students, and etc.

All in all, it was apparent that the resource and enhancement were the two factors with the lowest satisfaction from digital native students. It was suggested that digital device equipped for every students and high speed internet should be considered as the basic infrastructure for smart classroom. Digital resources should be provided to both teachers and students, and the resources should be abundant and associated with each specific learning objective for the specific class. Hardware or software should be equipped in classroom to facilitate the sharing of learning materials and learning outcomes for digital native students. Equipping technology is just one of the factors that influence students’ learning experience in smart classroom. Pedagogy adopted by teachers plays another vital role in students’ perception to classroom environment, especially the psychological classroom environments. Physical classroom environments equipped with technologies interacts with pedagogy adopted by teachers. Different pedagogy asks for different learning environment, and vice versa (Radcliffe 2009).

Conclusion

In order to optimize classroom environments to improve student’s learning experience, the present study investigated smart classroom from both pedagogical and technological perspective by a large scale survey considering the five dimensions of resource, environment, enhancement, management and presentation .

The results showed that the five factors questionnaire was a reliable and valid tool for evaluating smart classroom.The survey results confirmed that configuration of Internet access and digital resource for each student in classroom were vital for improving learning experience as these were the basis for digital native conducting inquiry and collaborative learning by using digital resources. The results also confirmed that students did not perceived much smart learning even in tablet classrooms, which implicated that pedagogy fit was the key issue for the application of smart classroom. Technology equipment plus advanced pedagogy could lead to the transformation of teaching and learning modes. The results also confirmed that display of students’ learning results played an essential role for student’s perception of classroom climate.

This study was conducted with a survey research model. Although a survey methodology was appropriate to examine characteristics from a population, it is not as accurate as observing behavior and perception (Archambault and Barnett 2010). The process could be stronger if qualitative data were gathered to validate the results of the survey (Yurdakul et al., 2012). The future study should be conducted by using the methods of classroom observation and interview in different kinds of classrooms, especially the tablet classroom, to interpret the phenomenons and produce concrete solutions. Possible future studies also include designing a smart classroom and carrying out instructions in the classroom, and then evaluate the effects of learning with technologies, and investigate the patterns of teaching activities and learning activities in smart classroom to match technology, teaching and learning space.

Appendix

Smart Classroom Evaluation Questionnaire (SCEQ)

Resource

In class.

1.I can get on internet to search for learning materials

2.I can get on internet

3.I can share digital resources with peers

4.I can get the videos that the teacher uses in class

5.I can get digital learning resources.

6. I can find that computer sockets in classroom when I need to use them

Environment

In class.

7. Temperature in classroom is suitable for concentrating on learning.

8. I don’t feel sleepy in classroom because of fresh air in classroom.

9. No unnecessary noises exist in classroom.

10. Light in classroom is enough for reading books or digital books.

11. I can hear teacher and other students clearly

Enhancement

In class, digital devices enable.

12. I can get the correct answer for questions.

13. I can get learning guidance from devices.

14. I can work with peers for learning tasks.

15. Teachers can assess my practice instantly.

16. I can accomplish tasks by using devices.

Management

In class.

17. I have adequate workspace for the placement of textbooks, tablet PCs and other resources.

18. Adequate space exists for easy movement among workstations, resources and exits.

19. The layout in classroom is suitable for my ways of learning.

20. The podium, blackboard and projector are at the right place for teaching and learning

Presentation

In class.

21. I feel the digital devices promote my sharing.

22. I understand teaching content better with multi-screen display.

23. I fell the network promote my sharing.

24 I can share my learning outcomes with others.