Using an adaptive learning tool to improve student performance and satisfaction in online and face-to-face education for a more personalized approach

Abstract

It is becoming increasingly clear that not all students require the same education, and the requirement of personalized education is increasingly in demand. The incorporation of adaptive learning (AL) has increased in recent years. However, research on this subject is still evolving at the university level. In this study, we investigated the impact of integrating an AL tool (CogBooks®) in a university course (statistics for decision making) taught in an innovative online modality called FIT (flexible, interactive, and with technology), in which the course is designed in the CANVAS® platform and uses Zoom® as a means of communication with students. Learning outcomes were compared between the FIT courses with or without AL and between AL strategies in online and face-to-face courses. It was clear that AL improved the students’ achievement regardless of the modality. In addition, we conclude that students achieve better in AL courses in the classroom than in distance courses. Satisfaction surveys favor a preference for FIT courses with AL over classroom classes with AL. Our results suggest that AL is a solid strategy for teaching undergraduate courses.

Graphical abstract

Introduction

Nowadays, technological developments present an opportunity to generate customized teaching by allowing students to choose their learning path (Alamri et al., 2021). To meet each student’s educational needs to improve their knowledge and increase their engagement, personalized learning emerged (Alamri et al., 2020; Walkington & Bernacki, 2020). Personalized learning refers to an educational strategy where the objectives, the sequence of the content, the learning pace, and the instruction can vary according to the student's needs (Peng et al., 2019). There are different ways of approaching personalized learning, one of which is Adaptive Learning (AL) (Waters, 2014).

However, although AL is not entirely new, it is not easy to define and continually evolves (Cavanagh et al., 2020). According to Martin et al. (2020), the objective of AL is to generate a unique learning experience taking into account the individual differences of the student, either defined as a process (focusing on how the contents are presented to the student, which is adapted depending on the increase in the understanding of the material, measured as the results obtained in the evaluations), as well as their preferences about the type of materials, or technological devices and software. AL strategies are focused on generating learning experiences based on the student's previous knowledge on their learning outcomes; all this is supported by technologies (software) that allow knowing the student's progress and obtaining data to modify instruction according to the results (Peng et al., 2019).

The essence of adaptive systems allows students to approach the contents and advance the material at their own pace, as fast or slow as the instructor or the course structure allows (Dziuban et al., 2016). Additionally, the software is designed to provide continuous feedback, and adapt the content in response to how the student responds to the questions and activities; all this helps him/her to command the content (Bailey et al., 2018).

The incorporation of AL systems has been carried out with different objectives by the educational institutions; some use it as a tool for the students to do their homework, perform exercises, and be a support mechanism throughout the course; in other cases, as part of remediation mechanisms and for those who need to reinforce their knowledge (Tyton Partners, 2016). Derived from the use of AL in universities, research has been generated focused on assessing the impact of its use in terms of improvement in learning outcomes, engagement, and retention, among others.

In a study conducted by Diziuban et al. (2017), the researchers sought to determine how students react and adjust to flexible learning environments that use AL in two universities (the University of Central Florida and Colorado Technical University). The results showed that although there were differences between institutions, students from both universities gave AL a high mark regarding effective education. In addition, most believe that AL gives them greater flexibility and helps them strengthen knowledge acquisition.

On the other hand, Wang et al. (2019) used a hierarchical knowledge structure to automatically organize updated and compiled learning materials from the internet. They provided students with personalized materials that adapt to their language (Japanese) proficiency. The results showed that adapting the materials to the needs of each student increases their engagement with the course.

Additionally, in the study by Liu et al. (2017), the researchers sought to evaluate the impact of the use of AL as a tool for the remediation of knowledge of biology, chemistry, information literacy, and mathematics in students entering the pharmacy career (pharmacy undergraduate program). The results showed that using AL significantly increased the knowledge of the remedial chemistry content; however, it did not have the same effect in the other disciplines.

Savio-Ramos (2015) investigated the effectiveness of personalized learning in increasing high school algebra competence. One hundred and seventeen students participated (between 10 and 12th grade). They were assigned to two groups: (1) computer-based learning with the incorporation of a personalized learning platform and (2) the same learning environment without the AL platform. Both groups were subjected to a pre-test and post-test. The results showed that there was no learning gain in either group. However, those who used personalized learning had a more positive perception of personalized learning than those of the traditional group.

In a case study conducted by the Boston Consulting Group in conjunction with Arizona State University, the use of AL in Biology courses was incorporated, increasing by 2% the number of students who obtain grades between A and C in the groups using AL in mixed mode (face to face with online), concerning the traditional mixed modality (Bailey et al., 2018). In the case of Algebra, AL was incorporated in the online mode groups, and the increase in the percentage of students with A-B-C scores was 11% higher than in the mixed mode groups. On the other hand, at Georgia State University, the use of AL in introductory writing courses contributed to lowering the DFW rate (percentage of grades of D, F, or students who Withdraw from the course) in minority students (Bailey et al., 2018).

Similarly, Colorado Technical University integrated its AL tool into its face-to-face Trigonometry and Pre-calculus courses, increasing the average pass rate from 76 to 94%, and average withdrawal rates decreased from 36 to 17% in Trigonometry. Similarly, the average pass rate in Precalculus increased from 66 to 94%, and average withdrawal rates decreased from 45 to 13%. Additionally, students who took courses with AL performed better in the following calculus courses (Daines et al., 2016).

Research gap and study objectives

Research-based evidence on the results of the incorporation of AL is still evolving at the university level. There is a lack of quantitative studies that demonstrate a difference in student’s grades and satisfaction when using integrated (sequence and structure) and contextualized AL strategies in higher education (Xie et al., 2019). The purpose of this research is to quantitatively evaluate the impact of incorporating the use of AL in higher education. We take, as a subject of study, a business statistics course, which is a subject belonging to the first semester of the Business program.

The main objective was two-fold: Firstly, to determine the effect of incorporating an AL strategy in the “Statistical Methods for Decision-making” course offered in a flexible, interactive, and with technology (FIT) modality (synchronous online). Secondly, assessing in which modality the AL has the most significant effect on student performance, in courses in the classroom or FIT classes.

The hypothesis of our work is that students improve their academic performance when an AL strategy is used within the course. For this reason, we sought to evaluate AL in the learning modalities that were employed at our university at the time of the study. We wanted to answer the questions:

• Q1. Using an AL strategy, do students perform better (have better grades)?

• Q2. In what type of course does an AL strategy help students perform better?

• Q3. Does using an AL teaching strategy impact evaluating satisfaction with the learning experience?

Theoretical framework

Education is evolving with technology to improve quality and increase potential. Adaptive learning systems using machine learning offer a solution to individualized learning paths that can be time-consuming. By assessing knowledge and considering social-emotional characteristics, education can be diversified and more effective, leading to fewer dropouts (Osadcha et al., 2021). The paradigm shift conveyed by AL consists of using data-based technology to identify specific needs based on the levels of performance achieved by each student to provide the most appropriate educational resources, activities, instruction, and feedback that the student needs at a specific time to reinforce his/her performance (Dron, 2018).

Adaptive learning theory follows a teaching approach that tailors the educational experience to the individual learning style of each student. By leveraging data to create a customized program, it considers various abilities and needs, rather than adhering to a predetermined path. This results in a personalized, data-driven learning experience (Sezgin & Yüzer, 2022).

The framework for adaptive learning posits that individualized instruction is a crucial component in achieving sustained academic progress and satisfaction (Clark et al., 2022). Considering the student learning performance, the theoretical framework adopted in this work corresponds to the Digital Technology—Personalized and Adaptive Learning student learning framework, developed by Singh and Alshammari (2021), which establishes three postulates:

• 1. Digital technology creates a smart learning environment, enabling efficient, effective, and comfortable personalized learning.

• 2. Digital technology can provide personalized and flexible learning to improve student performance.

• 3. The environment greatly influences educational institutions and complements the relationship between digital technology-enabled personalized and adaptive learning and student performance.

This theory is based on the Technology Organization Environment framework, developed by Tornatzky and Fleischer (1990), which presents a valuable instrument for scrutinizing the adoption and integration of various information technology innovations (Oliveira & Martins, 2011).

In this case, the environment (online or face-to-face modalities) enriched by AL technology could impact the student performance, this is, the test scores and final grades. On the other hand, to create an effective learning environment, it's important to continuously assess and improve the course design process. Student feedback and learning theories should also be taken into consideration. Evaluation should be done in phases to address technical challenges and immediate concerns, and subsequently learner satisfaction (Kruger, 2020). In this regard, learner satisfaction is how users perceive an information system's usefulness in achieving their goals. It reflects how students feel about their learning experiences and can impact their commitment to a program. High satisfaction leads to lower drop-out rates and higher persistence (Lim et al., 2022).

Study context and problem

Tecnologico de Monterrey launched the Tec21 Educational Model, which aims to develop skills and abilities required in the professional field. It offers challenge-based learning, inspiring Faculty, memorable experiences, and flexibility for students. The online FIT courses account for the last Tec21 component (Castillo-Reyna et al., 2020). In FIT courses, students from all national campuses from Tecnologico de Monterrey take a class (90 min) with nationally renowned professors via webconference in real time on defined days and times. Usually, these classes are taught through the video chat software program Zoom, twice a week. Students can find the course content, exercises, materials, and strategy on a Learning Management System (CANVAS; Instructure Inc., Salt Lake City, UT) platform.

The advantages of these courses are that the student has the flexibility to take the sessions from any geographical location, interact with classmates from different campuses, as well as have personalized follow-up from the professor through various digital means. It includes a digital learning environment that integrates content, resources, and activities, as well as digital media for interaction.

In a routine review of the outcomes of the FIT courses, we found that the subject Statistical Methods for Decision Making was complicated for students (students didn’t have a good performance—non-passing averages). This is a basic course (first third of the career) that does require the student to have coursed basic mathematics in the business curriculum but may be difficult for its abstract concepts content. Thus, it was necessary to give students tools so that they could increase their level of approval, level their knowledge, and get a better understanding. To solve this problem, we incorporated AL into the course. For this, we employed the CogBooks platform, which is an AL tool available on the market. However, as the tool was integrated into the didactic sequence of the class, the model was completely developed by us internally.

For the incorporation of CogBooks, it was necessary to redesign the course entirely. For this, FIT course faculty members were trained in AL and CogBooks. Additionally, they had the support of an instructional advisor to design a global strategy to integrate the class experience and the student duties and tasks in CANVAS and CogBooks. The AL strategy observed the following characteristics:

• 1. The course is organized hierarchically by topics covering specific learning objectives; each includes a set of concepts.

• 2. Within the AL platform, each concept presents a base content that can include video, text, and infographics, among other educational resources, and that has been designed with micro-learning principles.

• 3. In each basic content, the student is offered the option of indicating their degree of understanding of the material presented.

• 4. At least two additional reinforcement resources are included for those students who cannot understand the base content.

• 5. At the end of each topic, after reviewing all the concepts, an automated evaluation test is presented that the student can take two times.

• 6. The adaptive system allows for obtaining analytics on the evaluation tests and the students’ progress in the content review.

• 7. The instructor consults this information before each class session.

Figure 1 indicates the didactic sequence we follow in the present work.

Fig. 1

Didactic sequence in the described AL course

The CogBooks platform gives students a personalized learning experience and immediate feedback. Each subject of a course is divided into concepts and sub-concepts (Fig. 2).

Fig. 2

Screenshots of the different levels of information from the course on the CogBooks platform

For each sub-concept of the course, the student enters the platform and reviews the first teaching support entirely prepared by the instruction designers (Fig. 3).

Fig. 3

Screenshots of the explanation of a topic in CogBooks

After reading, the student shows his/her level of understanding based on a thermometer included in the same tool (Fig. 4).

Fig. 4

Screenshots of the learning evaluation performed by the student regarding a certain topic in the CogBooks platform

In case of having a degree of understanding greater than 50%, the platform displays a quiz about the topic (Fig. 5).

Fig. 5

Screenshots of a question from a quiz performed by students at the CogBooks platform

The student has two attempts to take the quiz, and the final grade is recorded. If the student gets a good grade (> 70/100) after the first attempt, the next topic will be reviewed. If he/she has a failing grade or enough but wants to improve it, the system recommends checking another booster support and retaking the quiz. The next topic will be reviewed if the student gets a good grade the second time. If he/she has a failing grade, the system recommends emailing the instructor through the platform. In case of having a degree of understanding less than 50%, the platform shows a second reinforcing didactic support with a different format than the previous one. For example, if the first support was a text, now, a video can be displayed. This has the goal of impacting all the different types of student learning. After finishing reviewing the second resource, the student will have to self-select once again the degree of understanding through the thermometer. If it is more than 50%, the quiz will appear with the previously described process; otherwise, it will show you the third resource of reinforcement. After the revision of the third resource of reinforcement, whose format is different from the first two, the student must say if his/her degree of understanding is higher than 50% and therefore take the quiz corresponding, or if there is still doubt, the platform allows the student to contact the instructor via email or REMIND (a direct message communication tool with the instructor) platform for personalized advice before taking the quiz.

In this work, we attempt to investigate if the proposed personalization strategy improves student performance and satisfaction, not only in FIT courses but also in traditional face-to-face formats.

Methodology

In this work, we employed a comparative quantitative methodology, since we measured the learning performance (exams and final term averages) and satisfaction (using a Likert scale), of students taking a course with AL, compared to a control group.

Participants

This work was conducted for one year and six months, divided into three semesters: August-December (Year 1), January-May, and August-December (Year 2). Table 1 includes the sample distribution regarding the semester, groups, enrolled students, and gender. All students pursue a business career.

Table 1 General data of the analyzed courses

This was a voluntary response sampling. During Year 1, a call was launched for all faculty members who had previously taught the subject in FIT format to see if they wanted to include AL in their course. Those who accepted were trained and included in the FIT + adaptive learning experiment; the others were considered as a control group. For the experiments of Year 2, a call was launched for faculty members who had previously taught the subject in FIT format with adaptive learning and who also had a group (the same class) in the face-to-face format. From those who accepted, a random selection was made. The selected instructors were included in the experiment. In the face-to-face traditional format, they changed their material for that of FIT + AL.

Procedure

The course where our study was focused was Statistical Methods for Decision Making, which belongs to the first third of t the Business undergraduate program at the Tecnologico de Monterrey (Mexico) that has a midterm exam and a final exam, designed in a collegial manner. The exams were the same in each of the semesters of the study, the questionnaires were multiple-choice to standardize the results and make the analysis more objective. They include distractors based on the most common errors of the students, according to the topic of the question. Both exams are taken in person at the Campus in which the student is enrolled. Table 2 shows the characteristics of the midterm and final exams.

Table 2 Midterm and final exam features

The final grade also includes activities carried out by students on the CogBooks AL platform and other activities carried out in class or outside, individually or collaboratively. Table 3 shows the composition of the final evaluation.

Table 3 Final grade evaluation

The first implementation of the AL courses studied here occurred in the August-December (Fall) semester of Year 1; four FIT groups participated, two in which AL was incorporated and two without AL as the control experiment. In the case of the groups where AL was integrated, the students had to carry out reading activities and exercises on the CogBooks platform before the class. The platform generates a learning path for each student according to their initial assessment and progress. Students had to cover specific topics before arriving at class. The instructor reviews the progress of the students in CogBooks before the class with two goals:

• (1) To send a message, using the same CogBooks or the REMIND platforms, to the students who had not finished the last activities.

• (2) To adjust the class to focus specifically on the topics where the students had a weak understanding.

According to the above and the results of the CogBooks tests, the instructor designed the class considering the scenarios in Table 4.

Table 4 Scenarios for class preparation using CogBooks as an AL system

The students could review the same material on CANVAS in the groups where the AL was not incorporated. However, reading it before the class was optional.

The courses contained the same topics in all cases and were addressed simultaneously according to the syllabus. Table 5 shows the breakdown of contents and activities carried out in the course, comparing FIT courses with or without AL (August–December of Year 1).

Table 5 Comparison of the features of the FIT and FIT + AL courses

In all courses, the instructors who participated in this study are identified with a number throughout the study (Instructors 1–12).

Considering the results obtained in the first experimental settings, it was decided to extend the use of the AL tool to the face-to-face (classroom) modality during the January-May and August-December semesters of Year 2. For these experiments, there were eight groups for the first period (four in the FIT and four in face-to-face modalities) and twelve groups for the second period (six in FIT and six in face-to-face modality). Both types of groups used the same CANVAS course design using CogBooks.

Table 6 shows the breakdown of content and activities carried out in the course, comparing the face-to-face courses with AL with the FIT courses with AL in two consecutive semesters, Jan-May and Aug-Dec (Year 2).

Table 6 Comparison of the characteristics of the Face-to-face AL and FIT + AL courses

Data collection and analysis

In the August-December semester of Year 1, the results of the mid-term exam, final exam, and the final grades of students who took the FIT course with AL (FIT + AL) were compared to those who took the same FIT course without this teaching technique (FIT). In the following year semesters, the comparison included students who took the FIT course with AL and students who attended the same course in a face-to-face format (face-to-face + AL).

The first tool consists of an analysis of the midterm exam, final exam, and final grades, as well as the percentage of students passing and scoring above 90. The second tool analyzes the results of the Student Opinion Survey (SOS) of the instructors who participated in the study. The SOS included the questions below (Table 7).

Table 7 Questions of the SOS survey

The SOS uses a Likert scale from 0 to 10, where 0 is worst, and 10 is best, except for the 08MEJ question which is a dichotomic question (1 = Yes 0 = NO).

It was necessary to verify the normality of the data because its distribution determines the best way to compare them. We perform Shapiro–Wilks tests to determine if the grades’ distribution is normal. The Shapiro–Wilk test is a more appropriate method for small sample sizes (< 50 samples). The null hypothesis for both tests states that data are taken from the normally distributed population. When P > 0.05, the null hypothesis is accepted, and data are called normally distributed. These tests were included in our study (Mishra et al., 2019). Next, inferential statistics with t-student and Mann–Whitney U tests were used, which allowed us to contrast the averages of the study groups. Since several instructors are involved in the study, an analysis is carried out for the instructors who participated in both modalities. Finally, the final evaluation of the students was analyzed using the Alpha-Cronbach method. The analysis was performed in SPSS version 25 (Cronbach, 1984; Mishra et al., 2019; Supriyadi et al., 2020).

Results

Quantitative data was collected directly from the CANVAS platform. Table 8 shows the results obtained by students enrolled in two groups of a FIT course compared with two groups of the same FIT course but with AL (FIT + AL) in the same semester (August–December Year 1).

Table 8 Comparison FIT with FIT + AL courses

The comparison is made regarding the midterm exam, final exam, and final grades, as well as the percentage of students passing and scoring above 90. The average grades for the midterm exam, final exam, and final grade of the FIT + AL course were 63.9, 80.6, and 80.3; in all three cases, the grade point average was higher than the FIT course without AL 59.3, 61.4, and 79.9. This can be explained as students need to understand the methodology and its advantages. For this reason, the mid-term exam results are usually lower than the final ones. An interesting aspect is that even though the percentage of students approved (with a grade higher than or equal to 70) in the FIT course (97%) is higher than the FIT + AL course (89.2%), the percentage of passers with grades higher than 90/100 is higher in the FIT + AL (24.3%) compared to FIT (15.2%).

Table 9 shows the results obtained by the students enrolled in the FIT + AL compared with students having the face-to-face format with AL.

Table 9 Comparison FIT + AL with Face-To-Face + AL courses from Jan–May (Year 2)

The average grade in the midterm exam of the FIT + AL courses was 71, which is higher than the face-to-face AL, which is 68.7. However, the final exam average grades and the course final grade are higher in the face-to-face AL (70.4 and 83.7) than in the FIT AL (59.8 and 77.3). Similarly, the percentage of students approved and those with grades above 90 are higher in the face-to-face AL (92.4% and 29.3%) compared to the FIT AL (85.7% and 15.5%).

Finally, Table 10 shows the results obtained by the students enrolled in the FIT course with AL (FIT + AL) vs. students who enrolled in the same course but in a face-to-face format (Face-to-face + AL). The comparison is made regarding the midterm exam, final exam, and final grades, as well as the percentage of students who passed with scores above 90.

Table 10 Comparison FIT AL with Face-To-Face AL courses during Aug-Dec (Year 2)

The average grades for the midterm exam, final exam, and final grades of the FIT AL course were 57.5, 71.4, and 80.8; in the three cases, the grade average was lower than the face-to-face AL of 62.8, 77.9, and 84.2. Similarly, the percentage of students approved and those with grades above 90 are lower in the FIT AL (90.8% and 18.5%) compared to the face-to-face AL (96.1% and 31.8%).

The results of the SOS were also analyzed. This survey is anonymous, applied at the end of each period, and its purpose was to detect possible differences between the different modalities of teaching the course. Table 11 shows the results of the SOS of the four instructors who participated in the study.

Table 11 Average results of the SOS survey after having a FIT or a FIT + AL course in August-December (Year 1)*

In all cases, 80 to 100% of the enrolled students answered the survey. The results show that except for question 2, where instructor I2, who taught the FIT course without AL, obtains an assessment similar to that of the instructors of the FIT course with AL, in the rest of the questions, the opinion of the students favors instructors who used AL.

The SOSs surveys were applied to students with FIT courses with AL (synchronous online sessions), and the results were compared with those from face-to-face courses with AL. Table 12 shows the results of the eight instructors participating in the January-May 2019 semester. In all cases, 80 to 100% of the enrolled students answered the survey.

Table 12 Average results of the SOS survey after having a FIT + AL or a Classroom + AL course in January–May (Year 2)*

Regarding question 01 MET on the methodology, question 02 PRA on the practical application of the concepts, question 04 EVA on the evaluation, and question 05 RET on the intellectual challenge, the results show that the three least favorable evaluations are obtained by the instructors I7, I8 and I9 of the face-to-face model. Regarding the role of the instructor as a learning guide in question 06 APR, the results obtained by instructors in both modalities are very similar. However, in question 03 ASE, regarding the interaction with the instructor and the advice received during the learning process, instructors I2 and I9 of the face-to-face model obtained the three highest evaluations. This may mean that students value the closeness of having the instructor in the classroom. It does not escape our attention that when the same instructor (in this case, I2) taught the same course during the same semester in two different modalities, FIT and in-person (classroom), both with AL, the opinion of the students was consistently better in the classroom than in the FIT mode. Noteworthy, when the average of the answers to the 08 MEJ question on all instructors in this period is compared, the AL course is always higher than those taught without AL.

We replicated this same experiment during the following semester. We applied the SOS questionnaire to 12 groups; the results are shown in Table 13.

Table 13 Average results of the SOS survey after having a FIT AL or a Classroom AL course in August-December (Year 2)*

The internal consistency (reliability) of each test or questionnaire should be reported and used only if it is high enough; for this, we conducted the Alpha Cronbach study (Cronbach, 1984) on the final grade, and the results were 0.779 for the Fall Semester of Year 1, 0.7694 for the 2019 Spring Semester and 0.8190 for the 2019 Spring Semester. These results implied an excellent reliability of the evaluations, according to Fraenkel and Wallen (2006), who stated that the reliability item could be accepted if the alpha is within 0.70–0.99.

A descriptive statistical analysis of the scores was performed, followed by Shapiro-Wilks tests to determine if the distribution of the scores was normal. Next, inferential statistics were used with Student's t-tests and Mann–Whitney U to contrast the average grades of the study groups. We decided to conduct comparative statistical studies between the FIT and FIT + AL groups to determine whether there were significant differences. As shown in Fig. 6A, the boxplot shows a higher standard deviation in the FIT AL groups in the mid-term exam and the final grade (26 and 10.76) compared to the FIT groups (19.9 and 8.52), contrary to what was observed in the final exam where the standard deviation (20.26) was lower concerning the FIT groups (23.17).

Fig. 6

Statistical analysis of midterm, final, and final grade exam scores. A Comparison between FIT groups with and without AL. B Comparison between FIT AL groups and classroom AL. Statistical analyses are found in the bottom table of each boxplot

The analysis of the final score by the Shapiro-Wilks method suggested a significant difference between the FIT and FIT + AL courses, with FIT + AL being higher (Fig. 6A). Since not all the data have a normal distribution, parametric and non-parametric statistics were used with students’ t-tests and Mann–Whitney U tests to contrast the average of FIT vs. FIT + AL grades in the final grade. As shown in Fig. 2A, the results aligned with the Shapiro-Wilks method, suggesting a significant difference between the FIT and FIT + AL courses (Fig. 6A).

In the same way, when we compare the grades of the FIT courses with those of the classroom courses, we observe that the FIT groups presented higher standard deviation in the mid-term exam, final exam, and final grade (20.3, 24.9, and 13.8) concerning the face-to-face modality (19.7, 21.4 and 11.6). In Fig. 6B, the boxplot shows that the standard deviation of the data collected in the FIT groups has a higher dispersion than in face-to-face courses. It is noteworthy that there are some outliers. The analysis of the final grade by the Shapiro-Wilks method suggested a significant difference between the FIT and Face-to-face courses, with the face-to-face course in the classroom being higher (Fig. 6B). As in the previous analysis, parametric and non-parametric statistics were used with student's t-tests and Mann–Whitney U tests to contrast the average of FIT scores vs. classroom in the final grade. As shown in Fig. 6B, there was a significant difference between the FIT and classroom courses (Fig. 6B).

Figure 7 summarizes our quantitative data, considering all the averages of the data studied. If we take the achievement of a FIT course as 1, the use of AL has positive consequences in the areas studied here: midterm, final exam, final grade, and passing rate. However, a significant improvement is observed in the quality of student achievement since, in both cases, in classroom + AL and FIT + AL, the percentage of students with more than 90 marks was almost 1.5 times higher than in the FIT course without AL.

Fig. 7

Effect of AL in several areas of this study. The averages of the achievement of the students who underwent FIT, FIT AL, and in-classroom (face-to-face-) AL were normalized, considering that 1 is the average of all the students who took a FIT course. Midterm, final, grade awarded, passing percentage, and segment scores of students exceeding 90 are shown

Discussion

Empirical research on the impact of AL is still limited because the technology for it is still developing (Weber, 2019). On the other hand, the complex nature of a subject matter such as statistics can prove daunting to first-semester students who are still acclimating to the demands of higher education. Using AL presents excellent opportunities for universities regarding student learning and satisfaction in these and other courses. The nature of the systems requires a didactic strategy that allows taking advantage of the benefits of AL in the incorporation of a course. Moreover, the use of AL requires specialized preparation of teachers in the use of technology and in the instructional design of their courses, as well as a scheme to provide continuous support during delivery for adequate adoption and implementation (Johnson & Zone, 2018).

In this work, we employed CogBooks, an AL system developed between the years 2015 and 2020 that stands out among other AL systems because of its size and functionality (Osadcha et al., 2021). Nonetheless, there are only a few works that document its use. This software has been used previously in biology and physics courses (O’Sullivan et al., 2020; Youngblood et al., 2022). The authors of these works have empirically and qualitatively measured the student's and faculty's perceptions of satisfaction.

CogBooks offers nine academic disciplines (agricultural sciences, biological sciences, business, health, and well-being, history, mathematics, philosophy, physics, and psychology.). However, some users find it difficult to navigate due to too many buttons and links. Also, the authors recommend that it could be improved by providing better explanations for incorrect answers (Vasyliuk & Lytvyn, 2023). In this work, the CogBooks content was curated by an instructional team and adapted to a FIT course or to a traditional face-to-face statistic for business course.

Regarding the first research question, Using an AL strategy, do students perform better (have better grades)? The results showed here that when incorporating CogBooks, an AL strategy, in the FIT courses, there is an improvement in the results obtained by the students in the midterm, final exams, and even in the final grades. However, there was no improvement in the percentage of passed. This may be because of the incorporation of mandatory activities in CogBooks. If the student did not carry out all the activities, his/her final grade was affected, and in some cases, could cause him/her to fail the course. This did not impact the students of the traditional FIT course because reviewing the materials was not mandatory.

Similar studies show positive effects in increase in average and decrease in dropout (Daines et al., 2016). According to Wang et al. (2023), students from two provinces of China who were randomly chosen to use an AI learning system displayed higher progress on a mathematics test when compared to those who were assigned to a face-to-face course taught by an expert. This was irrespective of the group size (Wang et al., 2023). A similar result was obtained by Hwang et al. (2020) when comparing the results of mathematics tests of an experimental group using an AL strategy with the results of a control group. However, in other works, AL technology has been found to be unrelated to test scores, but to the students' satisfaction when used on demand (not as an obligation of the course) (White et al., 2020).

Regarding the second research question, In what type of course does an AL strategy help students perform better? In our work, by incorporating the AL strategy in the face-to-face mode, the results, in general, are higher than those obtained in the FIT mode with AL; the average in the final exam in person with AL was 10.6% higher than the FIT with AL. Likewise, the final grade is higher at 6.4%; the pass rate is better at 6.7%, and the percentage of students with grades above 90 increased by 13.8%. The results suggest that students are still mainly accustomed to the instructor’s presence in the classroom. They value and take better advantage of the explanations and support given to solve doubts, which could be reflected in their grades. This agrees with the highest assessment that the students give in the satisfaction survey to the face-to-face model with AL concerning the question related to the advice received by the instructor during their learning process. Students may find that face-to-face counseling is more valuable than distance counseling. In this regard, in a recent study conducted by Daugherty et al. (2022), students have expressed concerns over the possibility of losing opportunities for social interaction with their peers. To mitigate this concern, experts recommend integrating activities that foster collaboration among students, whether through in-person or online means, as opposed to relying solely on software-based solutions. By doing so, students can engage with one another, a crucial element in their holistic development (Daugherty et al., 2022).

This takes us to the third research question, Does using an AL teaching strategy impact evaluating satisfaction with the learning experience? It is interesting to note that in the satisfaction survey, the results on FIT courses with AL were better than those without AL and face-to-face courses with AL. This is in agreement with other works. For instance, Hooshyar et al. (2021) used a learning satisfaction questionnaire to evaluate the students' satisfaction with an adaptive educational computer game AL system developed in-house. They found that students using the adaptive educational computer game showed a significantly higher level of satisfaction than the control group. In other work, learner´s satisfaction levels with personalized learning resources were relatively high (93.27% on a scale of 0–100%) (Peng &Fu, 2022).

It is important to note that a statistical analysis was made between instructors who taught AL and not AL courses at the same time, and we did not find a clear difference between the average of all instructors. This supports the idea that the AL strategy rather than the instructor is the critical difference in the student´s achievement. The role and the use of AL in undergraduate courses require more research. However, our study suggests that using an AL strategy improves the grades and the percentage of passing, regardless of the mode of delivery. It also allows the leveling of knowledge of students.

Implications

The use of adaptive learning is very recent, and there is little research on the results of its incorporation into the teaching–learning process, as well as the appropriate methodology to do so. Adaptive learning technology for education has focused on personalized instruction, but not enough on learner satisfaction as a key indicator of success (Lim et al., 2022). The ratings and satisfaction levels provided by the learner are considered direct feedback, while the scores are considered indirect feedback on AL systems usability (Raj & Renumol, 2022). This document contributes to the understanding of the use of AL, to the process carried out for its incorporation, and to its effect on learning gain and student satisfaction.

On the other hand, in educational systems, we treat all students equally within a classroom, mainly because the instructor in isolation does not have the tools to provide each student with what they need. Adaptive learning is a tool to personalize learning and provide the student with what they need according to their level of knowledge. This article contributed to better knowing and understanding of how to personalize learning using AL tools.

Limitations and future work

We must stress that even though the number of individuals in the sample is acceptable for statistical analyses, as in all educational innovation studies, our study has obvious limitations, such as the period of study, which is only three semesters, the fact that it is focused on a single subject.

The study was carried out in a private university with a high focus on enriching the teaching process with the use of educational technology, and with a very specific student profile. The results and challenges in incorporating AL may vary when considering the tradition of the educational institution in the use of technology, the skills of instructors and students in the use of technology, and the willingness of instructors to adjust the instructional design and infrastructure of the institution (technology and support for instructors and students) (Mirata et al., 2020).

Another limitation is that not all instructors participating in the research necessarily teach the class in the three modalities (FIT, FIT + AL, and face-to-face). That is, an instructor could teach in FIT + AI and Face-to-face, but not in FIT without AI. This can cause the results to be influenced by the instructor. Moreover, the characteristics of a professor in terms of technology management, teaching experience, and predisposition to innovation can influence the results of the research.

Furthermore, student characteristics may also play a role. In this case, these are first-semester students. The results could be different for students in more advanced semesters who already have a higher level of maturity and are more accustomed to self-study. Finally, all students took the same quizzes and tests in the courses. This makes it possible to compare scores between students and groups of students, but only for the same test. We cannot compare tests because the scores do not mean the same thing (they are not on the same scale). This is a significant limitation on the research questions we can ask within this study design.

In future work we propose to increase the sample to give it greater statistical validity, ensuring that it is the same instructor who teaches in the three modalities. Also, include the student's opinion about the use of this technology for learning, as well as the advantages and disadvantages of its use. We may repeat the study with another subject other than statistics, too. Finally, we plan to review whether the acceptance of AL depends on students' motivation and self-management capacity. That is, consider these characteristics as moderators of the effect.