Implicit modeling of learners’ personalities in a game-based learning environment using their gaming behaviors

Abstract

While the most used method to model the learner’s personality is the self-report using questionnaires, this study presents and validates a newly developed framework for implicitly modeling the learners’ personalities within a game-based learning environment using their gaming behaviors. This framework is based on an online role-playing game for teaching the computer architecture subject and a learning analytics system. To evaluate the efficiency of the proposed framework, an experiment was conducted with forty four participants (34 learners and 11 teachers) in a Tunisian University. The obtained results showed that this framework has a high accuracy level in correctly modeling both the extraversion and openness personality dimensions. In addition, these results highlighted a “good” and “moderate” agreement degree in modeling the extraversion and openness personality dimensions respectively compared to the Big Five Inventory (BFI). The findings of this study can advance research in game-based learning and educational psychology by developing environments which can be used for both learning and modeling learners’ personalities instead of using questionnaires.

Introduction

Many studies have reported the benefits of learning using digital games instead of the traditional method in classrooms which became boredom and not motivating (Prensky 2006; Squire and Jenkins 2003), especially for this new digital and gamer generation of learners. Machado et al. (2018) state that digital games are good tools to make learners more engaged and motivated to learn. Teachers also believe that using digital games for learning can enhance learners’ motivation and learning outcomes (Huizenga et al. 2017). In this context, 67% of American learners learn using digital games in classrooms (Statista 2018). Game-based learning (or also digital games for learning) are used to deliver several pedagogical objectives, such as learning English (Wu and Huang 2017), artificial intelligence (Denden et al. 2017a), mathematics (McLaren et al. 2017) and computer architecture (Tlili et al. 2016a, 2016b). Huizenga et al. (2017) highlight several important elements that contribute to cognitive learning outcomes in game-based learning, namely: (1) learning in safe environments which means that learners can try new things without worrying about the consequences of their actions, hence they can learn from their mistakes; (2) receiving direct feedback about their choices; and, (3) active learning which means that learners can make more than one trial, hence they can correct their errors if needed. Besides, one of the important factors to consider in game-based learning is the learners’ individual characteristics which make them behave differently while learning and playing. Tlili et al. (2016a, 2016b) stated that the main important indicator of individual differences is personality. Several studies show that personality differences can affect preferences for game genres (Schimmenti et al. 2017), playing style (Bartle 2004) and learning strategies (Pavalache-Ilie and Cocorada 2014). Chen and Lin (2017) state that knowing the personality of each learner can make the interaction within learning systems more effective. The traditional method to model the learners’ personalities is self-report using questionnaires, which is a subjective method. The main limitation of this method is that the learners may not reveal their true information if they think that they will not benefit from responding (Chen and Lin 2017). Tekofsky et al. (2013), on the other hand, show that individuals’ actions in games are related to their personalities. For instance, learners who react quickly in games are more likely to be low in conscientiousness. In this context, we hypothesize in this study that the learners’ actions and choices during a game might reveal their personalities. Therefore, this study focuses on implicitly modeling the learners’ personalities using their gaming behaviors and actions in game-based learning environments. Traditional analytic tools have been proved to be insufficient when dealing with these environments, since they are dynamic, packed with action and learning is an integral part of the game play. Such a specific learning environment requires a specific real-time analytical tool that will adequately match the dynamic game environment (Minović and Milovanović 2013). Thus, this study applies Learning Analytics (LA) to identify learners’ personalities using their gaming behaviors. LA is an emerging area that focuses on obtaining information by analyzing learners’ interactions with online educational contents.

This study particularly focuses on the extraversion (low extraversion, balanced extraversion and high extraversion) and openness (low openness, balanced openness and high openness) dimensions because they are the most corrected dimensions that affects user’s behaviors toward a system, including learning systems (Matzler et al. 2006; Duff et al. 2004). Extraversion personality trait is the first and widely used dimension proposed in the literature (Jung and Baynes 1921). Costa and McCrae (1992) consider introverts (people low in extraversion) as people who are less sociable and outgoing, while he considers extroverts (people high in extraversion) as people who have more positive emotions. Openness on the other hand is considered as people who are more open to experience and imagination (Chittaranjan et al. 2011). In this context, previous studies showed that these two personality dimensions are positively correlated (Matzler et al. 2006), which means that learners who are found to be high in extraversion are more likely to be high in openness to experience.

The rest of the paper is structured as follows: Section “Literature review” conducts a literature review about personality and how it is modeled in computer-based learning environments. Section “Proposed framework for modeling the learner’s personality” presents the developed framework to implicitly model the learners’ personalities using their gaming behaviors. Section “Method” presents the followed experimental method to validate this framework, while section “Results” presents the obtained results. Section “Discussion” discusses these results and finally, section “Conclusion, limitations and future directions” concludes the paper with a summary of the findings and potential future directions.

Literature review

Personality

Since there is still no broadly accepted definition of personality, various definitions of it have been proposed in the literature. For instance, Rose (2010) defines personality as “the enduring emotional, personal, interpersonal, experiential, attitudinal and motivational style that explains individual’s behavior in different situations.” Mount et al. (2005) define personality traits as stable psychological characteristics which define person’s behavior and cognitive style. Previous studies showed that personality affects learners in many ways, such as Internet use and addiction (Samarein et al. 2013; Tan and Yang 2012), preferences for game genres as well as their emotions while playing (Felicia and Pitt 2009; Schimmenti et al. 2017) and attitudes toward using game elements in educational environments (Denden et al. 2017b; Denden et al. 2018a). For instance, learners high in extraversion prefer using leaderboard, points and avatar game elements more than learners low in extraversion (Denden et al. 2018a). Studies have also highlighted that learners have different responses to educational methods based on their personalities (Cohen and Baruth 2017; Irani et al. 2003). For instance, learners high in conscientiousness and openness are more likely to prefer using online courses (Cohen and Baruth 2017). In addition, learners’ performances can be also affected by personality differences (Anderson et al. 2018). For instance, Shuto et al. (2017) showed that learners high in openness are more likely to have better performance in lecture courses.

In order to predict learners’ personalities, various personality models have been proposed in the literature, such as Myers Briggs (Myers et al. 1985) and Hans Eysenck (Eysenck 1990), but one of the most used and prominent psychological model is the Five Factor Model (FFM) (Franić et al. 2014; McCrae and John 1992; Tlili et al. 2016a, 2016b). FFM contains five dimensions which describe diversities of people (Chittaranjan et al. 2011), namely extraversion, agreeableness, conscientiousness, neuroticism and openness to experience. Since extraversion is the first and major dimension of psychological tests (Watson and Clark 1997) and it is highly correlated with learners’ openness to experiences (Barrick et al. 2001), this study focuses only on modeling extraversion and openness dimensions.

Extraversion or surgency as mentioned by Norman (1963) is the first proposed dimension in the literature by the psychological theorist Carl Jung in 1921. McCrae and Costa Jr. (1989) stated that the extraversion dimension varies between dominance and warmth. It refers to the degree of being sociable, activist, warmth, outgoing, have positive emotions and gregarious (Costa and McCrae 1992). In particular, extroverts (people high in extraversion) are optimist, confident, assertiveness, activist, sociable and friendly (Costa and McCrae 1992). In addition, they are considered as persons who prefer warm colors (Choungourian 1967) and move their energy toward the external word (Jung and Baynes 1921). Introverts (people low in extraversion) on the other hand are more likely to prefer cool colors (Choungourian 1967), afraid of taking risks (Walsh 2012), prefer loneliness and characterized by moving their energy toward the inner word (Costa and McCrae 1992; Jung and Baynes 1921). Therefore, people high in extraversion tend to be more engaged in different aspects of their lives than people low in extraversion (Watson and Clark 1997).

Openness to experience was first proposed by Tellegen and Atkinson (1974) and called “Openness to Absorbing and Self-Altering Experience” or Absorption. This dimension refers to the degree of being intellectually curious, original, wise, logic, foresighted, conscious, imaginative, insightful and open to new experiences (McCrae and John 1992; Watson and Clark 1997). Since openness contains terms that are consistent with intellect interpretation, such as being wise and logic, researchers linked the openness personality with learning motivation (Barrick et al. 2001). In this context, Farsides and Woodfield (2003) highlight that openness is strongly correlated with academic success, where students high in openness are more likely to have better academic results. In particular, people high in openness are more creative, curious and seek out new experiences, while people low in openness tend to be more familiar with old experiences instead of new ideas (Watson and Clark 1997).

Several studies have investigated the correlation between extraversion and openness dimensions (Matzler et al. 2006; George et al. 2011). For instance, Matzler et al. (2006) showed that extraversion and openness personality traits are positively correlated in marketing to affect users’ behavior toward a product. This means that people high in extraversion and openness are more likely to have a positive response toward a given product. Furthermore, Aluja et al. (2003) stated that extraversion and openness dimensions are strongly related with the sensation seeking construct of individuals. In particular, people with strong sensation tend to be high in extraversion and openness. The next subsequent section discusses how individuals’ personalities, including extraversion and openness, are modeled in computer-based learning environments.

Personality modeling in computer-based learning

Tlili et al. (2016a, 2016b) found that questionnaires are the most used method in the literature to model the learners’ personalities in computer-based learning environments. Questionnaires are a self-report measure where learners have to select the statements that best describe them (Ganellen 2007). However, learners can have low self-knowledge, which can negatively influence the validity of their responses (McDonald 2008). In addition, questionnaires are typically too long, which can make learners stressed and unmotivated. Moreover, learners might not give their true information or try to respond in a fashionable way, especially when they feel that they are being assessed by others (Okada and Oltmanns 2009) or think they can be penalized for giving their real opinions (Chen and Lin 2017). Therefore, many researchers have found that using behavioral patterns is more efficient in modeling personality (Chen and Lin 2017; Scherer and Giles 1979; Vinciarelli and Mohammadi 2014). In this context, studies showed that hand writing (Chen and Lin 2017), speech features such as speaking rate (Mairesse et al. 2007), social media behaviors (Gao et al. 2013), learners’ behaviors in Massive Open Online Course (MOOC) (Chen et al. 2016) and gaming behaviors (Bunian et al. 2018) can be used to model personality. Specifically, to the best of our knowledge, only digital games which are mainly designed for fun without any pedagogical objective are used to model the learner’s personality. Thus, this study uses a newly designed role-playing game-based learning, which aims to teach the computer architecture subject, to model the learners’ personalities using their gaming behaviors during the learning-playing process. Since analyzing learners’ gaming behavior requires a real-time analysis tool which can match the dynamic game environment (Minović and Milovanović 2013), this study uses a newly designed LA system to analyze the collected game behavior traces. In this context, Learning Analytics (LA) has emerged as an area which involves artificial intelligence, machine learning, information retrieval, statistics, and visualization, in order to automatically analyze educational data to enhance learning experiences (Chatti et al. 2012). Despite the importance of LA and game based learning, combining them remains a challenge (Serrano-Lagunaa et al. 2013). Therefore, the next section presents the proposed framework to implicitly model the learner’s personality in a game-based learning environment using LA.

Proposed framework for modeling the learner’s personality

To implicitly model the learners’ personalities using their game behaviors, a newly framework is developed which is composed of two parts, namely a Computer Architecture Game (CAG) and an LA system, as shown in Fig. 1. CAG was developed and deployed on an OVH web server. It contains the learning content as game activities which are added by the teacher and the implemented game elements to model the learner’s personality. During the learning-playing process, the learners’ gaming behavior traces are stored in game variables, and are then automatically saved in a MySQL database deployed on an OVH web server as well. This is done using typical client-server architecture with PHP scripts. After that, the collected traces are exported to an eXtensible Markup Language (XML) file, and then fed to the LA system to predict learners’ personalities. Finally, the obtained results of the modeled personalities are also saved in the MySQL database. Further details about CAG and the LA system are presented in the next subsequent sections.

Fig. 1

The proposed framework to model the learner’s personality

CAG game

CAG is a 2D role-playing game that aims to help learners learn the computer architecture subject (Tlili et al. 2015). It represents a virtual realistic environment of a city named “Kairouan”, which is located in Tunisia, to make learners feel like they are in a real situation and behave like they always do in reality. The game environment contains several areas and places that the learners can explore and visit. The main goal of the learner is to install the antivirus in the central computer of “Kairouan” city in order to bring it back to life. To ensure the learning process, CAG includes different game elements. For instance, it provides the learning content in a narrative game story which can make learners more motivated to learn (Dickey 2006; Nicholson 2015). In addition, an immediate feedback about the learners’ performance while playing is presented (This feedback is mainly to support learners while learning within the game). For example, when the learner responds correctly to a learning activity in the game, a positive feedback to make him/her more motivated will appear with additional details about that activity. Furthermore, a rewarding system, which consists of bonus money within the game, is used to make learners more engaged in order to gain more rewards (Werbach and Hunter 2012). The learners start by choosing the avatar that will represent them in the game. After that, in order to achieve the CAG goal, they are required to finish various activities, namely collecting coins, shooting, quiz, buying and battle, as described in (Tlili et al. 2015). The efficiency of CAG as a learning tool has been validated in another research work (Tlili et al. 2015).

CAG also offers a session for teachers where they can update/modify the delivered learning content (e.g., the learning content can be updated to fit the next course chapter that the teacher wants to deliver). This criterion allows the reusability of CAG with different learning contents. Figure 2 presents the interface where the teacher can modify the learning content of the “quiz” game activity. The teacher has to write in the first text box the question that the Non Player Character (NPC) will ask to learners. In addition, he/she has to define the three answers, which will be given by the NPC, where learners should only choose the correct answer among them. Finally, the teacher has to set up the feedback message that will appear to learners in case the answer is correct or wrong.

Fig. 2

Game interface to manage the learning content of the “quiz” game activity

For the modeling process, CAG also includes five game elements which can help identify the learner’s personality using games. These elements are validated in our previous work (Essalmi et al. 2017) and are: (1) modeling reality aims to bring the reality model into games. This element can influence positively on the degree of the learner’s immersion (Rouse III 2010); (2) avatars aim to represent the learner and reflect his/her image within the game (Bjork and Holopainen 2004); (3) goal is the ultimate game goal that the learner tries to achieve (Bjork and Holopainen 2004); (4) non-linearity means that the game should allow the learner to make his/her own choices regarding the way of achieving the game goals and paths to take (Rouse III 2010); and, (5) interaction which is in two forms, namely player to computer and player to player. The latter highlights the social side of games (Prensky 2001). Besides, various game behavior traces are collected during the learning-playing process in order to model the learners’ personalities (extraversion and openness). These traces are identified based on the different features of extraversion and openness dimensions (presented in section “Personality”). For instance, learners high in extraversion are more likely to prefer hot colors than learners low in extraversion who prefer cool colors (Choungourian 1967), therefore we have used the “color” trace (CR), as shown in Table 1, for the extraversion dimension. Moreover, learners high in openness are more likely to have better academic results (Farsides and Woodfield 2003), thus we have used the “score” trace (SCR) for the openness dimension. Table 1 presents the full corresponding gaming behavior traces for each personality dimension and its definition.

Table 1 The related gaming behavior traces for each personality dimension

For instance, to collect the “risk” trace, the learners, while navigating in the game environment, will have to freely choose one of two paths to take. The first path is written on it “dangerous” and the second path is written on it “safe”. The taken path will help in identifying if the learner is a risk taker or not, hence identify if he/she is low or high in extraversion. Additionally, as shown in Fig. 3(a), in order to collect the “color” trace (to model extraversion personality), learners have to visit a clothes shop to choose clothes for their avatars where only two colors are available, namely red and sky blue (hot and cool color). The choice of the learner can help deduce if he/she is high or low in extraversion. Figure 3(b) also presents another game scenario to collect the “accessed areas” trace (to model openness personality). In this context, while navigating within the game environment, the learners can visit different areas, such as disco or libraries. The type of the visited areas can help deduce if the leaner is sociable and open or not, hence identify if that learner is low or high in openness.

Fig. 3

(a) CAG scenario to collect the “color” trace (b). CAG scenario to collect the “accessed areas” trace

Finally, the collected traces are automatically saved in a MySQL database. Specifically, each trace has a game threshold, if the learner overpasses it then the game stores the value of that trace in the database as high; if the learner reaches the threshold without overpassing it, the game stores the value of that trace in the database as medium; Otherwise, it is stored as low. For instance, the game threshold for the “number of friends” trace is four. If the learner makes more than four friends in the game, the “number of friends” value will be “high” in the database. If the learner makes only four friends, the “number of friends” value will be “medium” in the database. Otherwise, the “number of friends” value will be “low” in the database. Finally, all the collected traces are fed to the developed LA system, which is presented in the next subsequent section, to model the learners’ personalities.

Learning analytics system

The collected traces are then fed to the developed LA system to model learners. Many approaches based on various artificial intelligence methods were proposed in the literature in order to model learners, such as the overlay model (Stansfield et al. 1976), stereotypes (Rich 1979), machine learning techniques (Webb 1998), fuzzy logic (Zadeh 1996), bayesian networks (Conati et al. 2002) and ontology-based student modeling (Winter et al. 2005). Bayesian networks are considered as one of the most used methods which deal with the uncertainty of learners’ model (Chrysafiadi and Virvou 2013). They are a powerful tool for knowledge representation (Cheng et al. 2002) due to their sound mathematical foundations and natural way to represent the probabilistic relationships among a set of variables (Heckerman 1997). Therefore, our LA system uses Bayesian networks to analyze learners’ gaming behavior traces in order to model their personalities. In this context, Naïve Bayes classifier algorithm, which is based on the below Bayesian rule, is used.

$$ \mathrm{P}\left(\mathrm{Cj}|\mathrm{d}\right)=\frac{\mathrm{P}\left(\mathrm{d}|\mathrm{Cj}\right)\mathrm{P}\left(\mathrm{Cj}\right)}{\mathrm{P}\left(\mathrm{d}\right)} $$

P(Cj|d) is the posterior probability of instance d being in class Cj.

P(d|Cj) is the likelihood, which is the probability of generating instance d given a class Cj.

P(Cj) is the prior probability of occurrence of class Cj.

P(d) is the prior probability of occurrence of instance d.

Naïve Bayes classifier is a supervised algorithm which may be developed using expert opinion instead of requiring historical data. Therefore, to identify the personality (class) that each learner belongs to, namely extraversion (low extraversion, balanced extraversion and high extraversion) and openness (low openness, balanced openness and high openness), several classification rules were prepared by four experts, which are academics specialized in psychology, based on the learners’ gaming behavior traces. Table 2 presents examples of these rules which are used within our developed LA system.

Table 2 Examples of the given classification rules by the experts

The LA system also uses data visualization techniques to show the teachers in a global view the personality distribution of their classes using pie chart representation. In addition, it considers several strategies to overcome several issues highlighted in (Tlili et al. 2018) during the data preparation for LA. For example, the LA system uses authentication method to allow only authorized persons to have the access to the collected traces and results, hence ensure the privacy of the learners. In addition, the collected traces and generated reports are stored for a pre-defined period (one academic year) before they are automatically deleted. Furthermore, to ensure the transparency of LA, the students have the possibility to see their collected gaming behavior traces. The next section presents the followed experimental method to validate this framework.

Method

In order to prove or disprove the following hypothesis the learners’ actions and choices during a game might reveal their personalities, an experiment was conducted in a public Tunisian University. This section presents the participants of this experiment. In addition, it describes the followed procedure and the used instruments.

Participants

The participants of this study were forty five participants divided as follows: thirty four undergraduate learners, who were all majoring in computer science and aged between nineteen and twenty two, participated in validating the accuracy of the proposed LA system in modeling personality. Eleven teachers on the other hand, who were also from different public Tunisian Universities and aged between twenty six and thirty six, participated in validating the technology acceptance toward using the LA system in their classrooms. The followed experimental procedure is presented in the next subsequent section.

Procedure

The learners started by playing CAG which is deployed online for twenty minutes. After that, they took a break for ten minutes, and then they answered the Big Five Inventory (BFI) to model their personalities. This assessment questionnaire took between ten and fifteen minutes. Finally, the modeling results obtained from the BFI and the LA system are analyzed and compared to evaluate the accuracy level and agreement degree. As a second step (after the validation process of implicitly modeling the learners’ personalities), the teachers answered the Technology Acceptance Model (TAM) questionnaire which took about ten minutes, in order to see their technology acceptance degree toward the newly developed LA system. This was also done to investigate if the teachers are willing to use this LA system in classrooms to model their students’ personalities. The used instruments are detailed in the next subsequent section.

Instruments

To identify each personality, learners had to answer the original English BFI version which is a validated and widely used questionnaire in the literature (John et al. 1991). It contains 44 statements regarding the five traits presented in the Five Factor Model. BFI is a five points Likert-scale questionnaire from 1 (strongly disagree) to 5 (strongly agree). This study used only the given answers of the eight statements which cover the extraversion dimension and the ten statements which cover the openness to experience dimension (defined in the BFI scoring). BFI scoring is divided into three groups for each dimension (low extraversion, balanced extraversion and high extraversion and low openness, balanced openness and high openness). In particular, the values between 1 and 2 indicate that the learner has a low personality dimension (extraversion or openness); the value of 3 indicates that the learner has a balanced personality dimension; and, the values between 4 and 5 indicate that the learner has a high personality dimension. Specifically, decimal values are rounded to the nearest whole value (e.g., the value 2.9 is rounded to the nearest value which is 3).

To identify the teachers’ technology acceptance degree toward the developed LA system, they had to answer the Technology Acceptance Model (TAM) questionnaire (Davis 1989). TAM questionnaire is a five points Likert-scale questionnaire from 1 (strongly agree) to 5 (strongly disagree) which covers four variables, namely: (1) perceived usefulness (U) refers to the degree to which teachers find the LA useful for modeling personality; (2) perceived ease of use (EOU) refers to the degree to which teachers believe that using the LA system is free of efforts; (3) attitude toward using the LA system (ATT) refers to the degree to which teachers report a favorable and positive attitude toward the LA system after using it; and, (4) intention to use the LA system (INT) refers the degree to which teachers are willing to use the LA system again in the future to model learners’ personalities.

Results

Accuracy results

To evaluate the accuracy level of the personality modeling results using the proposed framework, each learner’s personality result is compared to his/her results obtained from the BFI. In particular, no missing data was reported in the learners’ BFI responses. Accuracy is defined as the percentage of correctly classified learners and it is calculated using the below formula:

$$ \mathrm{Accuracy}=\frac{\mathrm{correctly}\ \mathrm{predicted}\ \mathrm{class}}{\mathrm{total}\ \mathrm{testing}\ \mathrm{class}}\times 100\% $$

As shown in Table 3, in both the extraversion and openness dimension respectively, only 7 and 10 learners (out of 34) had wrong personality results using our framework compared to the BFI. Consequently, the accuracy level of modeling both personality dimensions (extraversion and openness) results using our framework was high (79.41% and 70.58%).

Table 3 Accuracy level results

Agreement degree results

To determine the agreement degree between the proposed framework and the BFI, the Kappa (K) variable is calculated (Cohen 1960). It is used to validate newly developed instruments compared to already validated instruments, such as the instruments to measure depression (Ekeroma et al. 2012) or Anhedonia symptoms in Parkinson’s disease (Nagayama et al. 2012). According to Landis and Koch (1977), kappa < 0 indicates no agreement, from 0.0 to 0.2 indicates slight agreement, from 0.21 to 0.40 indicates fair agreement, from 0.41 to 0.60 indicates moderate agreement, from 0.61 to 0.80 indicates good agreement, and from 0.81 to 1.0 indicates perfect agreement. Table 4 presents the results of the Kappa variable.

Table 4 Kappa agreement degree results

As shown in Table 4, the level of agreement between the proposed framework and the BFI for modeling both personality dimensions (extraversion and openness) is statistically significant (p = 0.000 < 0.05). Furthermore, the agreement level is “good” for the extraversion personality since the obtained Kappa value is greater than 0.5 (Kappa = 0.651) and “moderate” for the openness personality since the obtained Kappa value is less than 0.60 (Kappa = 0.57).

Technology acceptance results

Prior to conducting further analyzes, it is important to evaluate the reliability of this questionnaire’s measurement. Thus, the Cronbach’s alpha for each variable (presented in the TAM questionnaire) was calculated. Furthermore, the median and mean of the teachers’ answers to the questionnaire were calculated. In general, a mean and median near 1 indicate that teachers are satisfied with the LA system. However, a mean and median near 5 indicate that they are dissatisfied with the LA system. Table 5 presents the mean, median and Cronbach’s alpha values for the variables EOU, U, ATT and INT.

Table 5 Technology acceptance results

If Cronbach’s alpha is equal or greater than 0.7, it means that the instrument is reliable (Yu 2001). Therefore, as shown in Table 5, the questionnaire’s measurements are reliable since all the values of the four TAM variables are greater than 0.7. Besides, as shown in Table 5, all values of the mean and median are around 1 and completely far from 5, thus the teachers were very satisfied with this LA system. The next section discusses the obtained results.

Discussion

The results of this study showed a “good” and “moderate” agreement degree in modeling extraversion and openness dimensions compared to the BFI. These findings support our hypothesis which suggests that the learners’ personalities can be identified based on their actions and choices in the games. However, few of the personality results using CAG game were wrong. Specifically, most wrong results are found when we are trying to model low personality dimensions (low extraversion and openness). This can be explained by personality can affect differently learners’ perception towards game genres (Peever et al. 2012). This can make learners with low extraversion and openness not comfortable while playing our CAG game, hence not show their real personalities while playing. Additionally, further investigation is needed regarding the used classification rules to identify learners with low personality dimensions in Table 2. This may enhance the accuracy level of modeling specifically learners with low extraversion and openness.

Besides, to evaluate the efficiency of our proposed framework which implicitly models the learners’ personalities, Table 6 compares this framework with similar systems reported in the literature. These systems are presented in different papers which were collected from different databases, including the ACM Digital Library, Taylor & Francis Online and Cornell University Library. The terms “modeling”, “implicitly” and “personality” were used to search all these papers. The conducted search highlighted a large number of papers, which were then reduced to 11 papers using the following criteria: (1) must be an empirical research which reports its findings; and, (2) must involve human participants. The comparison between our proposed system and the obtained systems (in the 11 research papers) was based on six features, as follows: (1) learning means that the proposed system is for learning purposes or not; (2) fun means that the proposed system stimulates the fun side or not; (3) traces means the used type of traces to implicitly model the learners’ personalities; (4) personality model specifies the used personality model; (5) data analysis method refers to the applied method to analyze the collected traces; and, (6) accuracy refers to the accuracy level of the proposed system in correctly modeling personality. For the learning and fun features, if the system contains one of these features, it will be represented within the cell with “+”, if not it will be represented with “-”. Table 6 also aims to summarize and give readers the research works reported in the literature which aims to implicitly model the learners’ personalities using information systems.

Table 6 Summary of the used systems for implicit modeling of personality

As shown in Table 6, several systems were used for implicitly modeling the learners’ personalities based on different traces. For instance, MOOC is designed for teaching, but it did not include the fun side. Psyops on the other hand is a game which is designed for fun but it is not educational. Therefore, compared to these proposed systems presented in Table 6, our proposed framework is the only one that combines both the learning and fun sides while implicitly modeling personality. In addition, most of these systems, including ours, refer to the FFM personality model, since it is the most used one in computer based learning environment (Tlili et al. 2016a, 2016b).

In addition, Fig. 4 presents the distribution of the used traces to implicitly model personality, based on Table 6. It is seen that games can help more in identifying learners’ personalities. For instance, the choice of the strategy to follow or the way of interacting in the game can make learners behave like in a real situation. As a result, collecting more behavioral information that can be used in the modeling process of individual characteristics, not only personality but also, working memory capacity (Khenissi et al. 2015), emotions (Conati and Zhou 2002) and learning style (Stathacopoulou et al. 2007). However, using gaming behaviors traces for personality modeling is critical since many factors may impact individuals’ behaviors in games, hence affect the accuracy level of the personality modeling results (Milam et al. 2012), as shown in Table 6. For instance, Milam et al. (2012) state that the learner’s previous gaming experience can affect the accuracy of personality results. Erfani et al. (2010) also highlight that age, gender and gaming experience can impact the learner’s performance in games. Moreover, gender differences affect players’ play style and emotions during the game (Hartmann and Klimmt 2006; Yee 2006). For instance, male players were found to be more aggressive than female players in games (Williams et al. 2009). Furthermore, the game genre may impact the learners’ perception of the game (Billieux et al. 2011) and the content of the learning activities (Chong et al. 2005; Prensky 2005), consequently affect their gaming behaviors.

Fig. 4

Distribution of the used traces to implicitly model personality

Conclusion, limitations and future directions

This study presents a newly developed framework for implicitly modeling the learners’ personalities, namely extraversion and openness, using their gaming behaviors. This framework is composed of both a game-based learning CAG and an LA system which uses Naïve Bayes classifier algorithm. The obtained results showed that this framework has a high accuracy level with a “good” and “moderate” agreement degree compared to the BFI for modeling the extraversion and openness personalities respectively. In addition, the teachers found the LA system easy to use and useful as well as they were willing to use it again in the future.

This study can advance research in game-based learning by developing environments which can be used for both learning and modeling learners’ personalities. In particular, this study highlights several game behaviors to be collected for personality modeling as well as the way of collecting those behaviors using different game scenarios which researchers and practitioners can use while designing their games. This research can also advance research in educational psychology by providing a new method for modeling the learners’ individual differences instead of using the traditional method, namely questionnaires.

This study on the other hand has several limitations which should be acknowledged and further researched. For instance, it covers only the extraversion and openness personality dimensions. Furthermore, this study contains a limited number of participants (only 45 participants). Moreover, the developed LA system uses only one algorithm (Naïve Bayes classifier) to model the learners’ personalities.

Future work could focus on: (1) modeling the remaining personality dimensions, namely Agreeableness, Conscientiousness and Neuroticism; (2) using other personality instruments (in addition to the BFI) to model the learner’s personality and compare the obtained results with the results of our LA system. This will further help evaluate the accuracy of the obtained results; (3) trying to implement our proposed framework as a service on the cloud. This can help provide an open learner model which can be reused by other learning systems or even universities, resulting in a low cost of learner modeling; and, (4) making CAG adaptive by providing a personalized game environment based on each identified learner’s personality. For instance, learners high in extraversion prefer hot colors (Choungourian 1967). This could be used for texts which will appear on the game interfaces to make learners more engaged and motivated. Moreover, learners high in openness are curious and open to experiences (Watson and Clark 1997), thus implementing some game elements, such as stories, may attract them more to use the game.