Sequence analysis and process mining perspectives to goal setting: What distinguishes business students with high and low self-efficacy beliefs?

Abstract

This study investigates the relationship between students' self-efficacy beliefs, goal-setting, and learning tactics in an online business course. Using sequence analysis and process mining techniques, we analyzed log data from 209 students to identify distinct learning tactics and their association with self-efficacy beliefs, inferred from initial goal-setting and final grades. Four learning tactics were identified, with significant differences observed between students with high and low self-efficacy beliefs. High self-efficacy students demonstrated more varied and adaptive tactics, including greater use of quizzes and project-focused activities. In contrast, low self-efficacy students and those who didn't set goals showed less engagement and fewer monitoring activities. The project-focused tactic had the strongest correlation with final grades and goal achievement. Goal-setting at the course's start was linked to more effective learning behaviors and better outcomes. The study reveals how students' online learning behavior changes based on initial goal-setting and subsequent reflection. These findings contribute to research on self-regulated learning in online environments and offer practical implications for designing online courses and learning analytics interventions. Understanding these differences enables the development of targeted interventions to improve learning tactics and self-efficacy beliefs among students, ultimately enhancing their ability to achieve learning goals in online educational settings.

Introduction

Self-Regulated learning (SRL) is a process including goal setting and planning, monitoring the progress of learning, and reflecting on outcomes and the process influencing learning outcomes (Pintrich, 2000). Goal-setting is a subprocess included in the first phase of SRL, namely as a part of the planning process, and depending on the goal quality it can lead to improved SRL (Boekaerts & Corno, 2005; Winne & Hadwin, 1998). This effect has been found in different levels of education, e.g., in primary level (Hirsto et al., 2022) in vocational high schools (Chang et al., 2018) and in higher education MOOCs (Wong et al., 2021). Students who excel in goal-setting are likelier to complete coursework (Handoko et al., 2019), since goals help students to monitor their progress at the learning task. Decades of research has shown that SRL is not easy (McCardle & Hadwin, 2015). Especially in online studying, skills in SRL are particularly needed (Heikkinen et al., 2023).

Recent studies have further emphasized the importance of SRL and goal-setting in online learning environments. Motivation, cognition, metacognition, and self-efficacy are significant components in the SRL process (Gambo & Shakir, 2021). This underscores the multifaceted nature of SRL and the need for comprehensive approaches to support learners in online settings. The relationship between self-efficacy and SRL has been a focus of recent research. Shin (2024) found that self-efficacy can predict self-regulation and course outcomes, suggesting that students' beliefs in their capabilities play a crucial role in their ability to effectively manage their learning. The effective use of metacognitive strategies can lead to an increase in self-efficacy, indicating a potential reciprocal relationship between SRL skills and self-efficacy beliefs (Khosravi et al., 2023). In the context of online learning, recent studies have explored innovative approaches to support SRL. Hsu et al. (2023) investigated how students allocate time during exam preparation, finding that consistency in study time throughout the preparation period led to better performance. This highlights the importance of time management skills within the broader framework of SRL. Implementing adaptive learning strategies with diverse didactic techniques can lead to improved learning outcomes and achievement of both disciplinary and transversal competencies (Rincon-Flores et al., 2024).

The need for SRL is recognized in project management and business education, although ways to support SRL are still few (De Oliveira Fassbinder et al., 2017; Marcelino-Sádaba & Perez-Ezcurdia, 2020). The research of SRL in business education started more than a decade ago. For instance, Schloemer and Brenan (2006) looked at developing the SRL skills of accounting students. Within the study, the students received a list of competencies established by the university, upon which they set personal learning goals to achieve them. Instructors used the learning goals to perform prompting interventions to support learners in considering alternative strategies to achieve said goals better. Surveys were used to help students monitor their progress based on the amount of studying time in the past week. In the middle of the course, students did a mid-term evaluation to assess the need for change in learning strategy. The self-evaluation was accompanied by teacher feedback. The results showed that goal-setting and monitoring helped students to improve effective learning behaviors, such as class preparation and collaboration.

A proper learning goal must be specific, measurable, and challenging, but yet attainable (Zimmerman & Schunk, 2011), just like the definition of the project goal (Doran, 1981). In practice, goal-setting is an excellent task for students at the beginning of the course; it combines SRL and project management perspectives. There are two elements connected to goal setting which explain the students’ ability to achieve the goal: self-efficacy beliefs and expectancy value. Self-efficacy beliefs are an important factor impacting students' motivation and academic achievement (Pajares, 1996). Educators play a key role in developing stronger self-efficacy beliefs with educational practices. Self-efficacy beliefs have a relation with SRL, especially via self-assessment, and factors enhancing it include the amount of feedback and support for a self-assessment process (Panadero et al., 2017). The expectancy-value theory explains individuals' attitudes and behaviour based on their beliefs and evaluations of certain outcomes. It proposes that people's attitudes are influenced by two main factors: expectancy and value (Wigfield & Eccles, 2000). Expectancy refers to a student’s subjective belief about the likelihood or probability of a particular outcome occurring because of learning activities. For example, if someone believes that studying hard will result in good grades, they have a high expectancy regarding the relationship between studying and academic success. Value refers to the subjective importance or desirability that a student attaches to a specific outcome or goal. This is to say, when students believe they can succeed and see the value in the learning task, they are more likely to persist in their efforts, leading to improved learning outcomes.

Learning analytics to support learning

Learning analytics (LA) has a rich collection of methods that can be used to analyse online learning processes. LA is “the measurement, collection, analysis and reporting of data about students and their contexts for purposes of understanding and optimising learning and the environments in which it occurs” (Siemens & Long, 2011). The data used in LA can come from clicks, various assessments such as quizzes, or log traces; these are usually gathered from a learning management system (LMS). Different ways to analyse this data include, e.g., measuring (Elmoazen et al., 2023) and predicting student performance (Namoun & Alshanqiti, 2020) or drop-out (de Oliveira et al., 2021), evaluating the elements of networking (Malmberg et al., 2022; Saqr & Alamro, 2019), and different learning patterns (Juhaňák et al., 2019).

Recent advancements in LA have expanded our understanding of student engagement and learning processes in online environments. Nakamura et al. (2024) employed change point detection algorithms to analyze student engagement in classrooms, demonstrating the potential of such techniques to provide valuable insights for improving teaching and learning processes. This approach offers new possibilities for identifying significant shifts in student behavior and engagement, potentially allowing for more timely and targeted interventions.

However, it's important to note that while LA interventions can improve online engagement, they do not necessarily lead to improved learning outcomes (Villalobos et al., 2024). This highlights the need for a more nuanced approach to implementing LA, one that goes beyond simply increasing engagement to focus on enhancing learning effectiveness. In this context, (Tepgec et al., 2024) argue for the importance of developing students' feedback literacy skills, emphasizing that the ability to understand and apply feedback is crucial for translating increased engagement into improved learning outcomes. Recent research has also explored the application of LA in problem-based learning (PBL) environments and revealed that students' levels of self-efficacy were related with their success rates in solving problems (Liu et al., 2023), while LA dashboards can significantly impact self-efficacy in virtual reality simulation-based training (Gallagher et al., 2024).

Learning patterns can be analysed as sequences to identify learning tactics students use while studying online (Hadwin et al., 2007). This sequence analysis approach uses students’ trace data to construct sequences of the student's actions studying online. The differences in the ways students act in LMS can be interpreted as the learning tactic students use (Jovanovic et al., 2020). This approach has been used to analyse learning tactics of business students (Heikkinen et al., 2022). There is a well-established connection between the usage of learning tactics and strategies, and the outcomes of learning processes (Jovanović et al., 2017; Siadaty et al., 2016). Another perspective to study patterns is process mining, which can be used, e.g., to cluster sequences to show the students’ interactions with the content and activities in LMS (Bogarín et al., 2014). An understanding of students’ learning processes enables the development of teaching processes and learning designs (Alqaheri & Panda, 2022; Stoyanov & Kirschner, 2023).

Building on these approaches, recent research has explored more personalized and adaptive learning systems. Takami et al. (2023) proposed a personality-based tailored explainable recommendation system for trustworthy smart learning. This innovative approach suggests that by considering individual differences in personality and learning preferences, we can create more effective and engaging learning experiences. Such personalized systems could potentially address the varying needs of students with different levels of self-efficacy and goal-setting behaviors, offering targeted support and interventions.

Sequence analysis methods can be used to understand the needs of students with low self-efficacy beliefs—the ones who believe that higher grades are either too hard to be achieved or require too much work. We do know that sometimes students are struggling with their assignments (López-Pernas & Saqr, 2021), but we are not sure how we can help them. It is also known that students have different goals, i.e. not all students want to achieve the highest grades (e.g., Knouse et al., 2014), e.g., due to low self-efficacy beliefs. Still, there may be students who are completely satisfied with the grade that is enough to pass the course. When designing interventions to support learning, it is essential to identify the students who are aiming for higher grades but do not have sufficient skills and distinguish these students from the ones who are satisfied with unpretentious learning outcomes. By aiming the intervention at the former group, the instructor’s effort is expected to produce a better output (Dahling & Ruppel, 2016).

Clustering is an approach which can be used to distinguish the differences in students’ learning profiles. Clustering has been applied to distinguish learner profiles in online management education. Das and Bhuwandeep (2022) used the Online Student Engagement (OSE) scale (Dixson, 2015) as a framework for clustering. The clustering was done to identify cohorts of students with similar learning preferences. The clusters differed based on the importance of attributes (e.g., the timing of online classes and teaching methods).

The power of LA has not been fully harnessed to effectively support SRL of business students’ learning in project management. There are few LA studies conducted to foster SRL in the field of business studies. In software engineering project management, LA is used to assess and predict teamwork (Petkovic, 2016). James et al. (2020) use LA to explore social factors influencing learning performance with the help of social network analysis. Virtual business projects are supported with scaffolding customised with automated reflection and feedback. These studies put their focus on the reflection phase of the SRL cycle. Thus, the understanding of goal-setting, self-efficacy beliefs, and how those affect learning remains unstudied in the field of business education.

Purpose and aims of the study

In light of these recent developments in LA, SRL, and adaptive learning strategies, there is a clear need for research that integrates these approaches in the context of business education. The current study aims to address this gap by examining how LA can help understand different learning tactics between students with low and high self-efficacy beliefs in a project management course. By understanding these differences, we can create targeted interventions capable of supporting low self-efficacy belief students to improve their learning tactics and learning processes to achieve their individual learning goals.

RQ1:

What kind of learning tactics do project management students use when studying online (RQ1.I), could the different learning tactics be used as an indicator towards achieving their goals (RQ1.II), and what are the differences in learning processes between different learning tactics (RQ1.III)?

RQ2:

How do learning tactics differ between low and high self-efficacy belief students (RQ2.I) and what are the differences in learning processes between low and high self-efficacy belief students (RQ2.II)?

By addressing these research questions, this study aims to contribute to the growing body of knowledge on SRL in online environments, particularly in the context of business education. The findings will not only enhance our understanding of the relationship between self-efficacy, goal-setting, and learning behaviors but also inform the development of more effective and personalized learning interventions in project management education.

Materials and methods

We apply a data-driven approach with log data captured from LMS to trace the actions students take studying online. This approach is widely used (Elmoazen et al., 2022; López‐Pernas et al., 2021; Zhang & Paquette, 2023). Sequence analysis and process mining can be studied using statistical software, e.g., R. There are several libraries which have been used to research patterns, including TraMiner, seqHMM (López-Pernas & Saqr, 2021; Saqr & López-Pernas, 2021); pMineR, rENA (Uzir et al., 2020); and BupaR (López‐Pernas et al., 2021; Uzir et al., 2020).

Study context

The current study was conducted at the Finnish University of Applied Sciences. The context was a project management course organised entirely online. There were three consecutive course implementations included in the research materials. Each implementation, lasting 14 weeks, was arranged during the academic year 2021–2022. The students participating in the course were first-year business students from the organising UAS and students from different UASes, who participated in the course during the summer semester. Only the students (n = 209) who gave their informed consent are included in the research.

The extent of the course is 5 ECTS (European Credit Transfer System, equals c. 130 h of student’s active participation). The course contains two themes: creative problem-solving and project management. These themes are divided into sub-themes following the phases of problem-solving (Lubart, 2001) and project planning (Meredith et al., 2017). In the problem-solving phases, students identify a problem, gather information, create ideas for the identified problem, and evaluate the created ideas to choose the best ones, whereas the phases for project planning include scope and work packages, schedule, budgeting, and compiling project plan documents. The project planning theme included an assignment focusing on SCRUM (Hron & Obwegeser, 2022; Takeuchi & Nonaka, 1986), aiming to provide tools for agile project execution. Students were expected to follow the order of these phases during their studying.

The course was organised as an asynchronous course in Moodle LMS. The course materials contained 17 video lectures. Trouble-shooting material was available for the students who struggled with the assignments, along with supplementary material for those who were eager to learn more. The students could use all the materials from the beginning of the course. Along with the video lectures, there was one assignment for each sub-theme, in total, ten assignments. In the first assignment, students chose the problem they drove to solve during the course in the following assignments. Moreover, to cultivate goal-setting (the first phase of SRL), each student was expected to set an individual learning goal for the course. Here a principle of formative assessment (Wiliam et al., 2009) was implemented even though it is not commonly used in business education (Ochuot & Modiba, 2018). Formative assessment was applied by using the number of accepted deliverables as criteria for the final grade of the course (five-tier numeric scale, 0 = failed, 5 = excellent). Within the instructions of the assignments, the criterion for accepted deliverables was defined, thus enabling the students to monitor (the second phase of SRL) their actions while preparing deliverables. If the criteria were not achieved, the instructor gave feedback to students on how to improve performance to achieve the level of an accepted deliverable. In the last assignment, the students were asked to recall their goals set in the first assignment and reflect (the third phase of SRL) on their learning and how they would improve their learning if they could start the same course again. These learning design elements can be seen as a tool to fulfil the need for interventions supporting all phases of SRL (Heikkinen et al., 2023).

Data sources

Three data sources were used from each student: (1) the student’s numeric goal set (the overall grade for the course) in the deliverable of the first assignment, (2) LMS log data, and (3) the final grade. LMS log data included a timestamp for students’ actions in LMS, user IDs, course module IDs, and event names. We used the log data as a basis for the analysis as it can unveil the learning processes (Siemens & Long, 2011) and it has been proven to be useful in the contexts of different LMSs (Viberg et al., 2018). Furthermore, time-stamped log data is a valuable tool for assessing the temporality of learning (Chen et al., 2018).

The final grades were used both as an indicator of course performance and as a benchmark to assess if a student had achieved their goal related to self-efficacy beliefs in terms of their intended goal set at the beginning of the course.

Data preparation

The first step of the analysis was cleaning the log data. The non-learning activities were removed from the LMS log data. These included teacher actions and learners’ actions not directly linked to active learning (e.g., profile views, viewing download centre, logging into the course). This was followed by the anonymization of the student IDs. The output of this step was re-coded into categories (Table 1) to illustrate the different contents and activities students use while studying in LMS. The coding scheme was grouping similar actions into categories (Molenaar & Chiu, 2014; Saqr et al., 2023) “assignment”, “course viewed”, “deliverable”, “feedback”, “interaction”, “learning resource”, “monitoring”, “quiz”, and “reading forum” to simplify the interpretation of analysis results. The cleaned and re-coded data included coded logs, timestamps, and student IDs, resulting in 56,047 rows of data. The descriptive statistics display the different activities students performed during studying (Table 2). This data was used in the following steps of analysis.

Table 1 Recoding LMS event to learning actions

Table 2 Descriptive statistics of students' usage of learning content and actions

The students were allocated into three cohorts by comparing the student’s final grade with the grade student set as a goal for themself in the first assignment. The coding scheme for this step is presented in Table 3. These cohorts are used to analyse the distinctive features between low and high self-efficacy belief students.

Table 3 Coding the goal achievement

Data analysis

We analysed the data with sequence analysis to find how students apply the different features of LMS during learning. This approach has been used numerous times to identify online learning tactics and strategies (López‐Pernas et al., 2021; Saqr et al., 2023).

Identification of learning tactics

To answer RQ1, to find what kind of learning tactics learners use and what is the distinction between low and high self-efficacy belief students, we applied sequence analysis. Sequence analysis is used to identify learning sessions with similar sequences of learning actions using the TraMineR R package (Gabadinho et al., 2011). The re-coded trace data was organised timewise to model and visualise the learning (Bogarín et al., 2018; Saqr et al., 2023). The re-organised data was grouped into sessions. Different learning actions are determined to belong to a certain session based on the interval between LMS actions. Every LMS action within the interval of 30 min was considered to belong to the same session, which is the interval used in similar studies previously (Jovanović et al., 2017). We trimmed the excessively short sessions, with less than three actions performed and the excessively long learning sessions with numerous activities were identified as outliers and therefore were trimmed to 50 actions (Jovanović et al., 2017). Before trimming the learning sessions, the maximum session length was 266, with a mean length of 8.38 (SD = 12.13, median = 4). After trimming the overly long and short sessions, the mean session length decreased to 8.03 (SD = 9.75, median = 4). The total number of sessions was 6,688. We used these sessions to describe and interpret what students do during online studying.

To identify students' learning tactics (RQ1.I), each session resulted in a sequence object with timewise organised events. These sequence objects are representations of learning tactics and display how the learning patterns between students differ. We used agglomerative hierarchical clustering (AHC) and tried different clustering methods: optimal matching, longest common subsequence, Hamming distance and Euclidean distance and their applications (Gabadinho et al., 2011) with Ward’s algorithm to cluster the sessions with similarities and find the optimal number of clusters (López‐Pernas et al., 2021; Uzir et al., 2020). Spell-length-sensitive optimal matching (OMslen) method resulted in four interpretable clusters with the help of fit indices (Gabadinho et al., 2011). OMslen is a modification of the optimal matching algorithm that weights optimal matching elementary operations inversely with episode length (Halpin, 2009). We used three plots to interpret the results: sequence distribution, index, and implication plots (Saqr et al., 2023). A sequence distribution plot illustrates the proportion of different activities at each point in time, an index plot displays each sequence as a stacked bar plot, and an implication plot visualises the difference between the target cluster and other clusters in each step of the sequence. We used fit statistics (R squared and average silhouette width) to support the selection of the best number of clusters (Studer, 2013). We aimed to find distinct clusters that could be used in plotting to interpret the differences in online learning behaviour. The labels for the tactics are based on the context, thus following the procedure used in previous studies (Fincham et al., 2019; Matcha et al., 2020; Saqr et al., 2023).

To find out if there are differences between applying different tactics and achieving the goal a student has set (RQ1.II), we tested Spearman’s correlation between different tactics, goal-setting, and final grade to see if these could be used as indicators towards achieving learning goals.

In order to find what are the differences in learning processes between different learning tactics (RQ1.III), we applied process mining methodology to visualise the consecutive order of actions students take during learning sessions. The clusters representing learning tactics were analysed using process mining to create process maps presenting differences between different learning tactics. Process maps illustrating students’ actions during their learning sessions were created using process mining (Peeters et al., 2020; Uzir et al., 2020) with the BupaR package (Janssenswillen et al., 2019). The resulting visualisations present the probabilities of different consecutive actions students take during online studying (Chiu & Reimann, 2021), and it is used to explain the order of actions students perform online. This was done to map the relative flow (edge percentage) between different actions (nodes) and the relative frequency (percentage within nodes) of the actions. In plotting, we used 0.02 as a threshold to suppress the noise in process maps with processmapR and DiagrammeR libraries. This action cuts off the transitions used least often. Thus, threshold limits the number of edges in graphs and makes it easier to interpret the plots.

Identification of differences between high and low self-efficacy belief students

To answer RQ2, we applied the same methodology as in the previous step. We continued by grouping the students into three cohorts: (a) students with high self-efficacy beliefs, (b) with low self-efficacy beliefs, and (c) who did not set learning goals for themselves, thus whose self-efficacy beliefs could not be determined (RQ2.I). This was done to find out if there are differences in applying different tactics between cohorts. We used process mining to create process maps of applying different learning activities, presenting differences between low and high self-efficacy belief students, and students who did not set their learning goals (RQ2.II).

Results

In this section, we present the results for each research question in subsections following the order of the research questions.

Identification of learning tactics

The sequence distribution plot (Fig. 1) displays the sequences (n = 6688) of LMS data. The sequences represent the actions students take during a learning session. The x-axis represents the order of actions (the first action in the first slice, the second in the second slice, etc.) and the y-axis represents the distribution of different actions of all sessions at the particular step of the sequence. The plot shows that most students start (the first slice on the x-axis) their learning session by viewing the course or performing monitoring activities (the dark blue slice). Then, students continue using learning resources (the yellow slice). This is the dominating action type throughout sessions. Feedback is applied most often as the second step of the learning process. As the sessions unfold, the frequency of assignments and quizzes increases. This might indicate that students first check the status of their learning process and then check the feedback they have received before continuing to study. Once they have gotten familiar with the current theme, they check the instructions for the assignment and perform quizzes before submitting deliverables. The frequency of interaction and reading forum activities are very low. The thin lines present these on the sequences’ 19th, 21st–23rd, 34th, 43rd, and 46th steps.

Fig. 1

Plot of overall sequence distribution of learner actions in the LMS

We used AHC to identify distinct learning tactics (RQ1.I). As a result of the OMslen method, we identified four distinct sequence clusters representing different learning tactics (Fig. 2):

• Short-focused (n = 3746, 56.0%): This is the shortest sequence cluster where students’ learning tactic is focused mostly on viewing the course. Some effort is put into course materials before returning to viewing the course at the end of the learning session.

• Quiz orientation (n = 1698, 25.4%): A quiz is the most used activity in this sequence cluster. The relative frequency of quizzes increases as the sessions unfold. Prior steps before the quiz include using learning resources and monitoring and feedback activities. After the peak proportion of quiz actions, students return to check learning resources and monitoring activities.

• Project focused (n = 689, 10.3%): Within this sequence cluster, the relative frequency of the assignment actions is highest compared to the other clusters. In these sessions, students check the assignment’s requirements, study the learning resources, and then submit the deliverables. However, the sessions do not end with submission. Instead, students check the assignment (perhaps to see the instructions of the consecutive assignment) once more before returning to check learning resources.

• Long diverse (n = 555, 8.3%): In the sessions belonging to this sequence cluster, students keep using the feedback throughout the learning sessions. Even though the relative frequency of feedback is not high, it is constantly used along with learning resources, monitoring activities and assignments.

Fig. 2

Sequence distribution plot of learning tactics (left), sequence index plot (middle), and implication plot (right) of the distinct learning tactics

We examined the correlation between goals, grades, and students’ learning tactics (RQ1.II) (Fig. 3). There was a significant correlation (p < 0.05) between almost every factor, except short-focused tactic and goal r(167) = 0.13, p > 0.05. The highest correlation coefficients were between grade and project focused tactic r(211) = 0.78, p < 0.05; short-focused and quiz orientation tactics r(211) = 0.73, p > 0.05; short-focused and project focused tactics r(211) = 0.71, p < 0.05; and grade and quiz orientation tactic r(211) = 0.71, p < 0.05. The associations between different tactics had lower correlation coefficients: Long diverse and grade r(211) = 0.62, p < 0.05; quiz orientation and project focused r(211) = 0.65, p < 0.05, long diverse and quiz orientation r(211) = 0.61, p < 0.05; long diverse and short focused r(211) = 0.44, p < 0.05; goal and grade r(167 = 0.43, p > 0.05; long diverse and project focused r(211) = 0.33, p < 0.05; project focused and grade r(211) = 0.29, p < 0.05; quiz orientation and grade r(211) = 0.24, p < 0.05; long diverse and grade r(211) = 0.24, p < 0.05. These results show that all tactics are important towards higher grades, while project focused seems to be the most important learning tactic.

Fig. 3

Correlation matrix between goal, grade, and students' learning tactics

Identification of differences in learning processes

Sequential process maps of students’ learning tactics (Fig. 4) illustrate the flow of learning activities and display the differences between students’ learning tactics (RQ1.III). The percentages in the figure display the proportion of learning sessions which include the transition (edges) between different activities (nodes) included in the particular session type.

• Project focused students are likely to start sessions with monitoring (38.5%), course-viewed (36.7%), or assignment (16,8%) actions. Then, students tend to move from monitoring to an assignment (32.3%) and back (34.8%), or from monitoring to course viewed (27.3%) and back (29.8%). This tactic rarely includes visits to learning resource actions (17.4%).

• Long diverse this tactic starts with course viewed (50.9%) or monitoring (32.1%). This is followed by the second step from course viewed to monitoring (34.0%) or from monitoring to course viewed (30.2%). An assignment (33.3%) is less frequently used when compared with project focused (59.6%), whereas learning resource actions have increased (from 17.4% to 23.3%).

• Quiz orientation starts with course viewed (58.8%), monitoring (27.8%), or learning resource (13.4%). From the course viewed the flow continues to either monitoring (62.0%) or learning resource (40.1%). Within this tactic, the learning resource usage has the highest value (60.6%). This process map shows a loop between course viewed, monitoring, and learning resource activities. An assignment (5.3%) and reading forum (5.9%) are visited rarely. This is the only process map, where the quiz (9.6%) is visited.

• Short-focused students start sessions with course viewed (94.5%). As the next step, students continue with monitoring (85.1%), which is followed most likely by returning to course viewed (80.1%) or learning resource (17.4%). Students using this tactic take more action to read forums (5.5%) than interacting with assignments (5.0%).

Fig. 4

Sequential process maps of students' learning tactics

Differences between high and low self-efficacy belief students

The learning tactics distribution plot between student cohorts (Fig. 5) illustrates the differences in the usage of different learning tactics between student cohorts (RQ2.I):

• High self-efficacy beliefs (n = 107, 51.2%): This is the largest cohort and the sequence of different tactics used is the longest. Students with high self-efficacy beliefs use the quiz orientation tactic more often than the rest of the cohorts. The short-focused is the most used tactic, while long diverse and project focused tactics are used during almost the entire span of the sequences.

• Low self-efficacy beliefs (n = 60, 28.7%): The length of the overall distribution plot for this cohort is shorter than for the students with high self-efficacy beliefs. These students use fewer learning sessions during the course, and the proportion of the short-focused tactic is slightly more used than students with high self-efficacy beliefs.

• Goal-not-set (n = 42, 20.1%): The most distinct feature of this cohort is the low frequency of the project focused tactic and higher frequency of the long diverse tactic. This suggests that these students wonder through the learning space but do not put significant effort into assignment activities.

Fig. 5

Learning tactics distribution plot between student cohorts

Differences in relative-case process maps between low and high self-efficacy beliefs students, and students who did not set their learning goals are illustrated in Fig. 6 (RQ2.II).

• High self-efficacy beliefs start their learning processes with course viewed (96.3%). This is followed by monitoring (97.2%), learning resource (70.1%), or assignment (52.3%). There is a strong flow between each of these four actions. Students with high self-efficacy beliefs are also the most active cohort to read the forum (12.2%) and the only cohort that uses quizzes (16.9%).

• Low self-efficacy beliefs Start every learning session from the course viewed (100%), which is followed by monitoring (96.7%), learning resource (50%), or assignment (40%). From monitoring, students tend to return to the course viewed (96.7%). When compared to students with high self-efficacy beliefs, low self-efficacy beliefs students tend to use fewer assignments, learning resources, and reading forum actions and do not use quiz activities.

• Goal-not-set The students within this group start with course viewed (95.2%), which is the only step before ending the session (76.2%) or monitoring (35.7%). Within this cohort, the usage of monitoring, learning resource, and assignment actions are the lowest, whereas reading forum is higher than non-achievers.

Fig. 6

Sequential process maps of student cohorts

Discussion

Goal-setting is a subprocess of SRL and can positively impact students’ academic achievement (Handoko et al., 2019). Our study contributes to the current body of research by implementing sequence and process mining techniques to better understand the differences in online learning tactics between low and high self-efficacy beliefs students. We identified the learning tactics of students based on sequenced events from their learning sessions. We also used the goals students set at the beginning of the course to identify differences in learning behaviour and learning outcomes. Our results indicate a positive effect of goal-setting at the beginning of the course, expressed through better learning tactics employed, but also more sessions and time spent by students overall in the LMS.

Overall, the most used activity was accessing learning resources, followed by monitoring, and assignment access. Quizzes can be used as both a summative assessment method and a reflection tool; students fill them out to get a better image of their progress at various points in the course which can then be used to reflect on their learning behaviour and change it accordingly. We also notice that course viewing decreases as learning sessions unfold. This finding suggests that throughout the learning session, students start with more general learning behaviour, such as viewing the course and its learning resources. The next actions within the session show clear differences between the low and high self-efficacy student beliefs. The students with high self-efficacy beliefs move slowly towards more specific or reflective activities, such as quizzes, whereas students with low self-efficacy beliefs struggle to figure out what they should do next to progress with their learning process.

Our first research question was divided into three parts, all building upon the preceding part in a logical progression to give a comprehensive answer. For the first part, we used all the activities previously described to identify four distinct learning tactics used by students during the course. More than half of the learning sessions were short-focused, as students simply viewed the course or the available materials before ending the session. Moreover, such sessions are constant across student goal-setting categories, as they are the most common type of learning sessions for both students that set goals or not, and for low and high self-efficacy beliefs students. We argue such sessions can play a role in the planning process of SRL (Winne & Hadwin, 1998), as students could access the course to clarify certain aspects of the course to plan future sessions. A more obvious explanation is that students simply get distracted or lose motivation after accessing the LMS; more research should follow up on this, especially considering how much overall time is spent during these sessions.

On the other side, we have the long-diverse tactic, where students are the most varied in their LMS usage. At first glance, we would expect such sessions to be the main learning activity, especially considering the constant usage of feedback, one of the main indicators in predicting academic performance in educational psychology (Hattie & Timperley, 2007). In the context of this course, the formative feedback received for the assignments gives students a perspective on the assessment criteria. After the evaluation, the student can assess their performance with feedback. Students with high self-efficacy beliefs use this as a tool to evaluate and, if needed, to enhance their performance, while other cohorts do not apply reflection at the same amount. However, long-diverse sessions have the second lowest correlation with the final grades and lowest with goal-setting. The quiz and project focused categories are the most straightforward; students use these sessions to focus on specific quizzes or assignments. Such sessions have the highest correlation with grades with project focused having the highest correlation with goals. Considering this difference between long-diverse and quiz or project focused sessions, we argue that having clearly set goals helps the students better focus during their learning sessions, such as focusing on specific assignments or quizzes, which is a more efficient and effective way of learning due to their testing nature. Such a difference could also portray the testing effect (Rowland, 2014), as completing an assignment or quiz would lead to better learning than rereading the materials associated with rote learning. While long-diverse represent a more varied learning behaviour, these could also hide confusion, students accessing everything with no clear purpose in mind. This could be the explanation for the students who did not set their learning goals: they lack the SRL skills, which would both help them to set precise goals for learning and to find ways how they could navigate themselves in the learning space. A final explanation stems from the stable form of the sessions. Long-diverse sessions have a steady structure throughout the course, while quiz and project focused sessions change drastically around the middle and end of sessions. Changing learning behaviour is important as it shows a clear SRL of students who need to adapt to the course features as a response to new material or feedback. Further research is needed to understand better why such differences occur.

Looking at the sequential process maps (Fig. 4) we see that viewing the course, accessing learning materials and monitoring are used the most across tactic categories. Monitoring is an essential subprocess of SRL (Pintrich, 2000; Winne & Hadwin, 1998), and such processes are missing from the only tactic category not correlating with goal-setting (i.e. short-focused); this is a clear indication of their importance. This is further corroborated by the fact that students that did not set any goals also had the lowest number of monitoring activities. As students have a clear goal in mind throughout the course, they know how to better monitor their learning behaviour, as they have a better image of what their final goal should be. This, in turn, offers clear criteria to which they can reflect and adapt the learning tactics used. The fact that a simple goal set at the beginning of the course can lead to such changes in the students’ SRL behaviour brings further evidence to the importance of supporting students in their SRL, changing their zone of proximal development, and allowing for further growth.

When we categorise students based on their self-efficacy beliefs, we see that students with high self-efficacy beliefs access the LMS the most, monitor their progress, access assignments and quizzes to test their progress, and apply feedback to improve their performance. This shows clear SRL behaviour, which further leads to successful learning and high academic performance. In contrast, students that did not set any goals have the most long-diverse sessions and read forums the most. This indicates an almost “confused” behaviour, as not having a clear goal in mind makes them not manage to employ effective learning methods and constantly check for others’ help. Moreover, the fact that they have almost no project focused sessions, with a sudden increase towards the end of the session seems to suggest blocked learning behaviour, which is a non-effective and harmful learning behaviour (Krasnoff & Overkott, 2022). The students with low self-efficacy beliefs have somewhat similar behaviour to students with high self-efficacy beliefs, however, they study less overall and make less use of assignments and forums. All these categories suggest a clear increase in learning and efficient learning tactics when students set a clear goal at the beginning of the course.

Relevance

Our study aligns with recent research in smart learning environments, particularly in the areas of SRL and LA. For instance, Gambo and Shakir (2021) conducted a systematic review on SRL in smart learning environments, highlighting the importance of motivation, cognitive, metacognitive, and self-efficacy components in the SRL process. Our findings on the differences between high and low self-efficacy students in their learning behaviors and goal-setting practices corroborate these components' significance in SRL.

Furthermore, our findings on the importance of goal-setting and its impact on learning behaviors align with recent studies in smart learning environments. For example, Hsu et al. (2023) investigated how students allocate time during exam preparation and found that consistency in study time throughout the preparation period led to better performance. This supports our argument that clearly set goals help students focus better during their learning sessions, leading to more efficient and effective learning.

The differences we observed in learning behaviors between students with high and low self-efficacy beliefs are also relevant to recent research on adaptive learning strategies. Rincon-Flores et al. (2024) found that implementing an adaptive learning strategy with diverse didactic techniques led to better results in students' learning and achievement of disciplinary and transversal sub-competencies. This suggests that tailoring learning experiences based on students' self-efficacy and goal-setting behaviors could further enhance the effectiveness of online learning environments.

Our study's use of LA to identify and analyze student learning tactics contributes to the growing body of research on using data-driven approaches to improve educational outcomes. For instance, Nakamura et al. (2024) used change point detection algorithms to analyze student engagement in classrooms, demonstrating the potential of such techniques to provide valuable insights for improving teaching and learning processes. Similarly, our sequence mining approach offers a way to automatically identify significant patterns in student learning behaviors, which can inform instructional design and personalized interventions.

The implications of our findings extend beyond just understanding student behavior. They also suggest potential avenues for developing more effective learning support systems. For example, Takami et al. (2023) proposed a personality-based tailored explainable recommendation system for trustworthy smart learning. Our findings on the differences in learning tactics between high and low self-efficacy students could inform the development of similar adaptive systems that provide personalized support based on students' goal-setting behaviors and self-efficacy levels.

This study sheds more light on how business students use LA tactics (Heikkinen et al., 2022). The results give more evidence of the differences in self-efficacy beliefs (e.g., Knouse et al., 2014) affecting the student's ability to achieve their expected goals. The sequence mining approach gives another perspective to clustering students (Das & Bhuwandeep, 2022), thus, offering a better understanding of the learner perspective. Furthermore, the results show another way to analyse the project management study process besides assessing and predicting teamwork (Petkovic, 2016). The sequence mining and process mining approaches have been previously used in different educational settings (Elmoazen et al., 2022; López‐Pernas et al., 2021; Zhang & Paquette, 2023), and this study verifies that these approaches are applicable also in business.

Our results show the importance of helping business students learning project management set clear goals and follow up on them, mainly through changes in online learning behaviour and SRL. We argue that such interventions should be encouraged and promoted more often due to their simplicity but large effects. If a student cannot set a proper goal, this can be seen as an indicator for a teacher to intervene. Such a student needs intervention at the very beginning of the course, which could greatly help students to first set goals for themselves. This should then be followed by a set of interventions aiming to help the students with low self-efficacy beliefs to improve their learning behaviour and match the patterns the students with high self-efficacy beliefs apply. As the meta-analysis by Heikkinen et al. (Heikkinen et al., 2023) discovered, less than half of the SRL interventions have positive significant effects, with very few targeting all phases of SRL. Our study aimed at filling this gap, as we also asked students to reflect on their work at the end of the course.

It is important to note, however, that while LA interventions, such as LA dashboards, can improve online engagement, they do not necessarily improve learning outcomes (Villalobos et al., 2024). This suggests that while feedback may help students improve their activity, it does not always help them understand what they should do to achieve better results. Therefore, as Tepgec et al. (2024) argue, feedback literacy skills should be supported to help students improve their ability to apply feedback and enhance their learning.

Our findings on the relationship between self-efficacy and learning behaviors are further supported by recent research. Shin (2024) found that self-efficacy can predict self-regulation and course outcomes, which aligns with our observations of high self-efficacy students demonstrating more effective SRL behaviors. Additionally, Khosravi et al. (2023) showed that effective use of metacognitive strategies can lead to a rise in self-efficacy, suggesting a potential reciprocal relationship between self-efficacy and SRL skills.

Theoretical and practical implications

This study has four theoretical implications. Firstly, our findings contribute to SRL theory by demonstrating how self-efficacy beliefs and goal-setting behaviors manifest in specific learning tactics within an online project management context. Secondly, this study showcases the potential of LA techniques, particularly sequence and process mining, in providing detailed insights into SRL processes, extending the methodological toolkit for SRL research. Thirdly, by examining SRL in the context of project management education, our study bridges these two fields, offering a new perspective on how SRL principles apply in professional skill development. Lastly, our identification and analysis of four distinct learning tactics contribute to the theoretical understanding of how students approach online learning, potentially informing future research on learning strategies.

Practical Implications focus on course design, targeted interventions, scaffold development, feedback mechanisms, LMS design, and student guidance. Instructors should incorporate explicit goal-setting activities at the beginning of online courses and provide opportunities for students to reflect on and revise these goals throughout the course. LA can be used to identify students with low self-efficacy beliefs early in the course, allowing for timely and targeted interventions to support their learning. Based on the identified learning tactics, educators can develop scaffolding strategies that guide students towards more effective learning behaviors, particularly for those with low self-efficacy beliefs. This can done by implementing regular feedback mechanisms that not only address content knowledge but also promote self-reflection on learning strategies and goal progress. LMS developers should consider incorporating features that support goal-setting, self-monitoring, and adaptive learning pathways based on students' self-efficacy levels and chosen tactics. This requires developing resources to help students understand different learning tactics and how to adapt their approaches based on course demands and personal goals.

Limitations

While this study provides valuable insights into the learning tactics and SRL behaviors of business students in project management courses, it is important to acknowledge several limitations that may affect the generalizability and interpretation of our findings.

Firstly, our study focused on business students in a project management course at a single institution. While this allows for a deep understanding of this specific context, it may limit the generalizability of our findings to other disciplines, educational levels, or institutional settings. Future research should consider replicating this study across various disciplines and institutions to validate the broader applicability of our findings.

Secondly, the self-efficacy beliefs and goal-setting data were collected through self-reported measures. While widely used in educational research, self-reported data can be subject to social desirability bias and may not always accurately reflect students' true beliefs or behaviors. Future studies could incorporate additional objective measures or observational data to triangulate self-reported information.

Lastly, there were limited timeframe and intervention testing applied in this study. Our study analyzed student behavior over the course of a single academic term. A longitudinal study spanning multiple terms or years could provide more insight into how learning tactics and SRL behaviors evolve over time, especially as students progress through their academic programs. Although our study identified differences in learning tactics between high and low self-efficacy students, we did not test specific interventions to improve self-efficacy or learning tactics. Future research could focus on designing and testing targeted interventions based on our findings.

Despite these limitations, our study provides valuable insights into the relationship between self-efficacy beliefs, goal-setting, and learning tactics in online project management education. By acknowledging these limitations, we hope to encourage future research that builds upon our findings and addresses these areas to further advance our understanding of SRL in online environments.

Conclusion and future work

The current study makes a novel contribution to the body of literature by portraying in detail the changes in business students’ online project management learning behaviour based on assignment goal-setting at the beginning of the course, followed by a reflection at the end.

There are multiple possibilities for future research. One of the main “weaknesses” of LA research, in general, is the lack of context, such as not knowing what the students do or intend to do behind the monitor. The previously discussed short-focused learning tactics are the perfect example. These short sessions were the most common by far, even disregarding the goal-setting factor. While we did give multiple explanations, we believe research should focus more on such “contradicting” behaviour by possibly also including the context. It is expected that the students do not study with 100% efficiency, all learning sessions have a clear goal which is always achieved. However, we also believe the high number of such short-focused sessions mandates further study.

The studied course had quizzes and assignments the students had to complete. While they did seem to represent a more effective way of learning, we are not sure why. One argument we gave was them being associated with the testing effect, a proven method of effective learning. However, we are not sure exactly if there is an actual effect of such assignments and how it could influence online LMS learning. We believe future research should study this effect in particular, as it could lead to high academic performance.

Furthermore, there is a need for the development of interventions and research to study the impact of different interventions for goal-setting of students who cannot set learning goals, and interventions for students with low self-efficacy beliefs to help them regulate their learning processes towards achievement.

Lastly, the trace data, per se, is insufficient to explain the learning processes. The cognitive aspect of learning should be evaluated jointly with the traces. This could be done by researching survey data gathered from learners to find out how they perceive the state of SRL and the impact of interventions.