An expectancy value theory (EVT) based instrument for measuring student perceptions of generative AI

Abstract

This study examines the relationship between student perceptions and their intention to use generative artificial intelligence (GenAI) in higher education. With a sample of 405 students participating in the study, their knowledge, perceived value, and perceived cost of using the technology were measured by an Expectancy-Value Theory (EVT) instrument. The scales were first validated and the correlations between the different components were subsequently estimated. The results indicate a strong positive correlation between perceived value and intention to use generative AI, and a weak negative correlation between perceived cost and intention to use. As we continue to explore the implications of GenAI in education and other domains, it is crucial to carefully consider the potential long-term consequences and the ethical dilemmas that may arise from widespread adoption.

Introduction

Artificial intelligence (AI) has been applied in various industries, such as healthcare (Topol, 2019), finance (Königstorfer & Thalmann, 2020), the transportation industry (Iyer, 2021), and education (Zhai et al., 2021). Generative AI (GenAI) is a subset of AI that has tremendous potential to revolutionize human-AI interactions and solve complex problems within educational settings (Russell & Norvig, 2016). ChatGPT, a type of GenAI, was released in November 2022 (Schulman et al., 2022), it has impressive capabilities to generate coherent and contextually appropriate responses that closely mimic human-like communication with an advanced language model based on the Generative Pre-trained Transformer (GPT) architecture. This has sparked significant interest in academic and industry circles (Agrawal et al., 2022; Chui et al., 2022; Cotton et al., 2023; Mucharraz y Cano et al., 2023) as well as among the public (Nah et al., 2023). It has potentials to provide personalized learning experiences and tailor instructional content to individual students’ needs and abilities (Chan & Lee, 2023; Chassignol et al., 2018; Crompton & Burke, 2023). It can also foster collaboration and peer interaction by generating context-aware prompts and responses, creating a dynamic learning environment that fosters engagement and deeper understanding (Zawacki-Richter et al., 2019).

However, the emergence and upgrades of GenAI have also brought new challenges for both teaching and learning (Chan & Hu, 2023). This calls for a human-centered approach to education (UNESCO, 2023) as higher education institutions adapt to this changing landscape. It is crucial to understand students’ intentions to use GenAI tools and ensure students are adequately prepared for their personal and professional pursuits in this fast-paced world (Chan, 2023a, 2023b). Intention, an important indicator of human behaviour (Ajzen, 2002), has been widely studied in educational technology (e.g., Ifinedo, 2018) but often within general contexts such as AI learning intention (Chai et al., 2021; Wang et al., 2023) or in non-GenAI settings, for example, AI-enabled automatic scoring applications (Fu et al., 2020) and AI teaching assistants (Kim et al., 2020). Since the adaptation of GenAI in higher education is still in the exploratory stage, past research on AI cannot fully illustrate students’ perceptions in GenAI-specific context.

Therefore, this study aims to address student-centered factors influencing their intentions towards GenAI by using the Expectancy Value Theory (EVT) to understand students’ preliminary knowledge and their perceptions of using the newly launched technology. EVT posits that individual’s motivation to engage in a behaviour is influenced by their expectations of success and the value they place on the behaviour (Wigfield, 1994; Wigfield & Eccles, 2000), working as a framework to examine students’ intentions and perceptions from knowledge, perceived value, and perceived cost of the technology in the early stage. To facilitate educators and policymakers to make informed decisions about the integration of GenAI into higher education, the research will validate an instrument specifically designed for GenAI context and further explore the following questions:

• 1. Is there a correlation between students’ knowledge of and familiarity with GenAI and their intention to use GenAI?

• 2. Is there a correlation between students’ perceived value of using GenAI and their intention to use AI?

• 3. Is there a correlation between students’ perceived cost of using GenAI and their intention to use AI?

Literature on student’s perception of AI and GenAI

There has been a growing interest in students’ perceptions of AI in education ranging from general AI use to specific applications such as AI teaching assistants and ChatGPT. In Chan and Hu (2023), student voices from a survey involving 399 Hong Kong undergraduates and postgraduates across disciplines revealed five benefits and six challenges of GenAI in education. Perceived advantages included personalized learning, writing help, and research abilities. Yet, concerns around accuracy, privacy, ethics, and its impact on personal growth, career, and societal values were voiced.

Zou et al. (2020) employed a sequential explanatory mixed-methods design, which comprised of a survey assessing student perceptions of their current usage and effectiveness of AI-English Language Learning apps for speaking skills enhancement, followed by qualitative interviews to elucidate and interpret the findings from the questionnaire. The sample included 113 Year 1 and Year 2 English for Academic Purposes (EAP) students from an English-speaking university in China. The primary findings reveal that participants expressed positive opinions regarding AI technology’s role in developing speaking skills, albeit with certain limitations, such as the absence of personalization and feedback. The potential implications of this study suggest that AI technology may serve as a valuable tool for supporting EAP students in improving their speaking skills.

Haensch et al. (2023) analyzed TikTok the social media content to better understand how students perceive and use ChatGPT. The findings suggest that students are interested in using ChatGPT for various tasks, but there is also a concern about its potential impact on academic integrity. The study highlights the need for educators to consider how they incorporate or regulate AI technologies like ChatGPT in universities to raise awareness among students about ethical considerations when using AI technologies. More AI regulatory in education information on governance, pedagogical and operational are found in Chan (2023a, 2023b).

A recent study in India (Kumar & Raman, 2022) surveyed 682 students enrolled in full-time business management programmes to gather their opinions on the usage of AI in various aspects of higher education, including the teaching learning process, admission process, placement process, and administrative process. Students generally had positive perceptions of AI usage in higher education, particularly in administrative and admission processes. However, they were more hesitant about AI being used as a partial replacement for faculty members in the teaching–learning process. The study also found that students’ prior exposure to AI influenced their perceptions.

Several studies on students’ perceptions of AI adopt the Technology Acceptance Model (TAM). TAM posits that the perceived usefulness (PU) and perceived ease of use (PEOU) of a technology are key determinants of its acceptance and use (Abdullah & Ward, 2016; Davis, 1989). Using this model, Kim et al. (2020) investigated students’ perceptions of the usefulness of and ease of communication with AI teaching assistants in the United States. The study included 321 college students, and the findings suggest that perceived usefulness and ease of communication with an AI teaching assistant positively predict favorable attitudes, which consequently leads to stronger intention to adopt AI teaching assistant-based education. Students who perceived positively with AI teaching assistants mentioned an increase in efficiency and convenience in online education. However, some students also expressed concerns about the lack of human interaction and the potential for errors or technical glitches. Another example is Hu’s (2022) study that examined the factors affecting students’ use of an AI-supported smart learning environment system and found that perceived ease of use and perceived usefulness influenced students’ behavioural intention.

Bonsu and Baffour-Koduah (2023) explored the perceptions and intentions of Ghanaian higher education students towards using ChatGPT, using a mixed-method approach guided by the Technology Acceptance Model (TAM) with a sample size of 107 students. The study found that although there was no significant relationship between students’ perceptions and their intention to use ChatGPT, students expressed the intention to use and supported its adoption in education, given their positive experiences. Social media was identified as a key source of students’ knowledge about ChatGPT, and they perceived more advantages than disadvantages of using it in higher education.

Gado et al. (2022) used an integrated model based on TAM and the unified theory of acceptance and use of technology (UTAUT) to investigate psychology students’ acceptance of and intention to use AI in German universities. Perceived usefulness, perceived social norm, and attitude towards AI were shown to predict intention to use AI; however, perceived ease of use was found to have no significant influence on intention to use. Although perceived knowledge of AI did not have a significant impact on attitude, it showed a relationship with intention to use. In Raffaghelli et al.’s (2022) study in Spain, the UTAUT model was adopted to examine students’ reaction to an early warning system, an AI tool that monitors student progress and detects students who are at risk of failing, in a fully online university. Their results show low expected effort in the tool’s usage was correlated with high perceived usefulness. Students’ perception of the tool also changed over time with the post-usage survey showing a lower acceptance level than the pre-usage survey.

Students’ intention to use AI-driven language models like ChatGPT in India was also explored by Raman et al. (2023). This study, framed by Rogers’ Perceived Theory of Attributes and based on Expectancy-Value Theory (EVT), aimed to explore the factors that determine university students’ intentions to use ChatGPT in higher education. A sample of 288 students participated in the study, which focused on five factors of ChatGPT adoption: Relative Advantage, Compatibility, Ease of Use, Observability, and Trialability. The results revealed that all five factors significantly influenced ChatGPT adoption, with students perceiving it as innovative, compatible, and user-friendly. The potential implications of this study suggest that students are open to using AI-driven language models like ChatGPT in their education and perceive them as valuable resources for independent learning.

In the Netherlands, a study (Abdelwahab et al., 2023) was conducted using a survey completed by 95 students from 27 higher education institutions. The survey questions were categorized into four factors based on a conceptual framework, including students’ awareness of AI, teacher’s skills in AI teaching, teaching facilities for AI, and the AI curriculum. Respondents were asked to provide their answers using various methods, such as a 5-point Likert scale, ranking, yes or no, or open-response answers. Business students in the Netherlands have expressed concerns regarding their higher education institutions’ readiness to prepare them for AI work environments. They feel that the institutions are ill-equipped or have not fully utilized their resources to provide adequate AI-related training. There is an urgent need to update the curriculum and educational facilities for AI work environments and provide more comprehensive training and education on AI-related topics.

A study involved 102 physics students from a German university who evaluated ChatGPT responses to introductory physics questions (Dahlkemper et al., 2023). This study aimed to evaluate how physics students perceive the linguistic quality and scientific accuracy of ChatGPT responses to physics questions. The study used a survey instrument based on the Unified Theory of Acceptance and Use of Technology (UTAUT), and included three statements about students’ expectations of AI performance and their attitudes towards AI in general. The items were answered on a 5-point Likert scale. The UTAUT model (Venkatesh et al., 2003) identifies four key factors that influence technology adoption: performance expectancy, effort expectancy, social influence, and facilitating conditions. The key findings suggest that while students generally perceived the linguistic quality of ChatGPT responses positively, they were more critical of the scientific accuracy. Additionally, students who had prior experience with AI were more likely to have positive attitudes towards AI in general.

Several factors have been identified in the literature as influencing students’ intention of using GenAI in education. Familiarity with AI technologies, personal innovativeness, and perceived usefulness have been shown to positively affect students’ attitudes toward AI (Chassignol et al., 2018). Furthermore, perceived ease of use, which relates to the user-friendliness of AI tools, has been found to be a crucial determinant of students’ willingness to adopt AI technologies (Venkatesh et al., 2003). Table 1 shows some previous studies on students’ perceptions of AI and ChatGPT Our study intends to employ the expectancy-value theory (EVT) to investigate the correlation between students’ intention to use GenAI and their knowledge, familiarity, perceived value, and cost of GenAI. The EVT postulates that self-efficacy and perceived value affect technology adoption. While the more commonly used TAM views that perceived ease of use (self-efficacy) impacts perceived usefulness (utility value), and perceived usefulness (utility value) influences behavioural intention in a cascade mechanism; the EVT presents both expectancy (self-efficacy) and perceived value as having a direct impact on technology adoption in a concurrent manner (Backfisch et al., 2021a, 2021b). In addition, the concept of “cost” in the EVT, which is the sacrifice, effort, and the negative aspects of engaging in an activity (Wigfield & Eccles, 1992) is either overlooked or underdeveloped in models such as TAM and UTAUT. Although the EVT has been used in many areas, there has been limited research into the predictive utility of this framework in relation to students’ attitudes and intention towards GenAI use. Hence, this study aimed to investigate whether students’ expectancy and their perceived value of GenAI concurrently affect their intention to adopt GenAI. It is hoped that the findings of this study would shed light on the factors that influence students’ adoption of GenAI and provide an alternative perspective on how the relationship between student perception and intention to adopt GenAI can be understood.

Table 1 Some previous studies on students’ perceptions of AI and ChatGPT

Expectancy-value theory and other frameworks

In the previous section, different frameworks such as TAM, UTAUT to explore student perception of AI have been mentioned. For our study, Expectancy-Value Theory (EVT) will be used. EVT posits that individuals’ decisions to engage in a particular activity or task are influenced by their expectations of success (expectancy) and the perceived value they attach to that activity (value).

Expectancy refers to an individual’s belief in their ability to succeed in a task, while value encompasses several components, such as attainment value, intrinsic value, utility value, and cost (Wigfield & Eccles, 2000). When examining students’ intention to use GenAI, this framework can be utilized to address the research questions as follows:

RQ1: Is there a correlation between students’ knowledge of and familiarity with GenAI and their intention to use GenAI?

According to the expectancy-value theory, students’ knowledge and familiarity with GenAI may influence their expectancy beliefs. The more familiar and knowledgeable students are with the technology such as how they generate outputs, the higher their expectancy beliefs may be, leading to a higher likelihood of adopting GenAI in their learning processes (Wigfield & Eccles, 2000). Previous research has also shown a positive correlation between students’ knowledge of technology and their intention to use it such as via the UTAUT model (Venkatesh et al., 2003).

RQ 2: Is there a correlation between students’ perceived value of using GenAI and their intention to use AI?

Value is a crucial component of the expectancy-value framework, and it is hypothesised that students who perceive higher value in using GenAI will be more likely to adopt it (Wigfield & Eccles, 2000). Studies have shown that perceived usefulness and perceived ease of use are significant determinants of technology acceptance (Davis, 1989; Teo, 2009; Venkatesh et al., 2003). Maheshwari’s (2021) study also highlights the impact of institutional support and perceived enjoyment on students’ intentions to continue studying courses online. Specifically, the perceived value components that influence these intentions are:

• Attainment value which refers to the belief that engaging in a behavior will lead to an important goal or outcome. For example, students who believe that using GenAI will improve their academic performance or digital competence may be more likely to use it.

• Intrinsic value refers to the personal enjoyment or satisfaction that a person derives from engaging in a behavior. For example, students who enjoy exploring new technologies or feeling comfortable using GenAI due to the anonymity.

• Utility value refers to the belief that engaging in a behavior will lead to practical benefits, such as improved skills or knowledge. For example, students who believe that using GenAI will help them save time or provide them with unique feedback may be more likely to use it.

RQ 3: Is there a correlation between students’ perceived cost of using GenAI and their intention to use AI?

Cost refers to the negative aspects or barriers associated with engaging in a particular behavior such as effort, time, undermining the value of education, limiting social interactions, or hindering the development of holistic competencies (Chan & Hu, 2023; Chan, 2023b).

Cost can be seen as a factor that influences an individual’s motivation and intention to engage in a behavior. If students perceive the costs of using GenAI to outweigh its benefits, they may be less likely to adopt the technology. Previous research has shown that perceived barriers, such as cost, can negatively affect students’ intentions to use technology in education (Chan & Hu, 2023; Flake et al., 2015; Regmi & Jones, 2020; Stüber, 2018). The expectancy-value framework has been widely used in educational research to examine students’ motivation, learning, and achievement (Cheng et al., 2020; Sin et al., 2022).

Why expectancy-value theory?

The Expectancy-Value Theory (EVT) is widely used and has been adopted across various domains. It is chosen as the theoretical framework for this study over other models such as the Unified Theory of Acceptance and Use of Technology (UTAUT), Technology Acceptance Model (TAM) and Theory of Planned Behavior (TPB) because EVT specifically focuses on the factors that drive individuals’ motivation and decision-making processes related to their choices, goals, and performance, which is a major focus of this study. While other models like UTAUT, TAM and TPB offer valuable insights into technology acceptance and adoption, they do not fully capture the motivational factors that are central to EVT.

EVT is considered more suitable for this study because it takes into account the perceived value and cost associated with using GenAI, which are critical factors in determining students’ intentions to use such technology. Moreover, EVT also emphasizes the role of students’ knowledge and familiarity with GenAI, which is an essential aspect of this study’s research questions.

Methodology

The main purpose of this study was to examine the correlations between students’ intention to use GenAI (e.g. ChatGPT) and their knowledge, perceived value, and perceived cost of using the technology, subsequent to the validation of the questionnaire developed upon EVT.

Sampling

The methodology for this study employed a cross-sectional survey design using an online questionnaire to gather data on students’ familiarity, knowledge, perceived value, perceived costs, and intention regarding the use of GenAI technologies in teaching and learning at universities in Hong Kong. In Nov 2022, these universities were confronted with the unexpected intrusion of GenAI, just like other industries. This disruption led to varying policies among universities, with some allowing students to use ChatGPT while others temporarily banned the technology due to perceived risks (Mok, 2023). The research was also inspired by the need for long-term solutions to address the challenges posed by GenAI by investigating its potential value and cost among students.

Convenience sampling was applied, wherein the participants were selected based on their accessibility and willingness to participate. To reach the participants, the questionnaire was distributed throughout March of 2023 via a bulk email sent to all the students studying in a university in Hong Kong, including undergraduate and postgraduate students from STEM and non-STEM disciplines. While this approach may not ensure a representative sample of the target population, it allows for the efficient collection of data from a large group of respondents. In total, 405 (out of 460 total responses) participants (as summarized in Table 2) provided valid information for data analyses with an average age of 23.87 years, consisting of 51.4% males (n = 208) and 48.6% females (n = 197). These participants were at different levels of degrees, with 50.9% (n = 206) at an undergraduate level and 49.1% (n = 199) at a postgraduate level. They also had various academic backgrounds, with 52.8% (n = 214) from STEM majors such as engineering, science, and architecture, and 45.7% from non-STEM majors such as education, business, law, medicine, and arts.

Table 2 Demographic information

Instrument for analyses

The questionnaire was specifically designed for the purposes of this study. It was informed by a review of relevant literature and existing questionnaires on teachers’ and students’ perceptions of educational technologies based on the EVT framework (e.g., Backfisch et al., 2021a, 2021b; Ball et al., 2019; Chen, 2011; Ranellucci et al., 2020). Since there are no currently available questionnaires on students’ perceptions of GenAI which are based on the EVT framework, some of the questionnaire items in this study were adapted from other similar instruments. For example, in Chen’s (2011) questionnaire on students’ acceptance of e-learning, one of the items measuring students’ e-learning performance expectancy is “If I use the CUS (Cyber University System), I will increase my chances of getting more competence”. To measure students’ GenAI attainment value, the item was adapted as “I believe generative AI technologies such as ChatGPT can improve my digital competence” (Q8). Further, most of the items were constructed to reflect issues and challenges associated with GenAI use within the EVT framework. For instance, Q20 “Generative AI technologies such as ChatGPT will hinder my development of generic or transferable skills such as teamwork, problem-solving, and leadership skills” and Q21 “I can become over-reliant on generative AI technologies” were designed to measure the perceived cost of GenAI, that is, the potential risks posed by GenAI cautioned by authors such as Bai et al. (2023) and Nah et al. (2023). To ensure that the questionnaire items were relevant and clear, pilot studies were conducted before the formal data collection. The questionnaire was modified based on the feedback received during the pilot study to ensure its accuracy and clarity.

The instrument consisted of four main sections: knowledge of GenAI, perceived value of using GenAI, perceived cost of using GenAI, and intention to use GenAI. The items and factors are already grounded in the EVT framework (Wigfield & Eccles, 2000), which has been well-established in previous research on technology adoption (Venkatesh et al., 2003). Table 3 shows the factors, their corresponding questionnaire items, and the analysis methods used and Table 4 shows the survey items. The participants’ opinions were assessed using 23 five-point Likert scale questions (Q1 as frequency scale; Q2-Q23, with response options ranging from 1-Strongly Disagree to 5-Strongly Agree). This allowed the participants to express their level of agreement or uncertainty on each statement.

Table 3 Summary of factors and questionnaire items

Table 4 Descriptive analysis results (N = 405)

Rationale for analyses

The analyses were conducted in three stages. The first stage focused on descriptive analyses of the responses to show the normality of the data and to reveal participants’ perceptions mainly by comparing means. The second stage involved the validation of each factor as specified in the EVT section (Table 3), where the validity and reliability of the scales were tested. The final stage analysed the correlations between students’ knowledge of and familiarity with GenAI, students’ perceived value of using GenAI, students’ perceived cost of using GenAI, and their intention to use AI, which aligned with the research questions.

Regarding the validation stage (i.e. stage 2), due to the strong theoretical basis of EVT, it was decided to use only Confirmatory Factor Analysis (CFA) without using Exploratory Factor Analysis (EFA), following the calculation of Cronbach’s alpha. According to Brown (2006), CFA is a hypothesis-driven method that allows for direct testing of the proposed factor structure. This method will enable us to focus on hypothesis testing and confirming the hypothesized factor structure, rather than exploring new and unknown factor structures that EFA would provide. Moreover, the use of CFA in this study can be considered a parsimonious approach, ensuring that our research findings are concise and easier to interpret. For example, in the context of our study, we will be able to assess whether the survey items measuring Students’ Knowledge of AI (Q2-Q6) indeed load onto a single factor, as theorized. The validity of constructs was measured by Average variance extracted (AVE), composite reliability (CR) (Fornell & Larcker, 1981; Hair et al., 2015), and Heterotrait-monotrait (HTMT) ratio (Henseler et al., 2015).

The analyses in the research were made through IBM SPSS 27 and IBM AMOS 28. The missing data were imputed by the multiple imputation procedure in SPSS.

Results

Stage 1: descriptive analysis

The survey study was conducted among students from Hong Kong to explore their perceptions of using GenAI technologies like ChatGPT for teaching and learning in higher education. The use frequency and familiarity with GenAI technologies among participants varied (never = 33.6%; rarely = 22.0%; sometimes = 28.9%; often = 9.6%; always = 5.9%) based on Q1 (“I have used generative AI technologies like ChatGPT”). With a mean as low as 2.32 (see Table 2), Q1 demonstrated that many participants had limited user experience with GenAI by the date the research was conducted.

As summarized by Table 4, S.D., skewness, and kurtosis values indicate a normal distribution of the dataset, which allows further calculations in stage 2 and stage 3. To better understand participants’ opinions towards Q2-Q23, means were interpreted by referring to the Likert Scale interval recommended by Pimentel (2010), where a point mean falls into the range from 1.00 to 1.80 can be regarded as strongly disagree, 1.81 to 2.60 as disagree, 2.61 to 3.40 as neutral, 3.41 to 4.20 as agree, and 4.21 to 5.00 as strongly agree.

Thus, based on the means of Q22 (mean = 3.92) and Q23 (mean = 3.86), many of the participants believed that the integration of generative AI technologies in higher education will have a positive impact on teaching and learning in the long run and they envision integrating generative AI technologies into their teaching practices in the future. They showed an overall agreement perception of the statements regarding their knowledge of ChatGPT (Q2-Q6), with means ranging from 4.15 to 3.8. Additionally, they also recognized the value (Q7-Q17) of ChatGPT in various teaching, learning, and working occasions; except for Q11 (“I think generative AI technologies such as ChatGPT can help me become a better writer.”) and Q10 (“I can ask questions to generative AI technologies such as ChatGPT that I would otherwise not voice out to my teacher.”), with means at 3.31 and 3.38, which tended to be more neutral. Whereas students’ responses on the cost of using generative AI were more neutral, with means settling between 2.85 and 3.15.

Such responses may imply the consensus value of ChatGPT amongst the majority of the participants. They did have a certain level of understanding and concerns over the limitations of generative AI technology in terms of handling complex tasks. Meanwhile, the participants recognized the technology’s attainment value such as improving efficiency, and its foreseeable utility value in the workplace as well as the long-term impacts on the learning outcomes (for example, academic performance, creativity- and emotion-related competencies).

Stage 2: reliability and validity of the scales

Driven by the theory of EVT, the questionnaire aimed to understand students’ perceptions of using GenAI with regard to knowledge, value, and cost. To measure the reliability of the constructs, Cronbach’s alpha coefficient was first calculated to test the internal consistency. Cronbach’s alphas for the 5 knowledge, 11 perceived value, and 4 perceived cost items are 0.812, 0.876, and 0.746. As shown in Table 5, the Cronbach alpha values are all greater than or closer to 0.7, indicating an acceptable internal consistency within the three scales.

Table 5 Cronbach alpha coefficient results (Total N = 405)

CFA tests were then conducted to test the construct reliability of the knowledge, perceived value (with three sub-scales), and perceived cost. Since χ² is very sensitive to sample size and its p-value will tend to be small in a big sample, we decided to report the χ²/df ratio instead as recommended by Schermelleh-Engel et al. (2003). The results in Table 6 indicate a good model fit regarding students’ knowledge, student-perceived value, and student-perceived cost with all less than 3 (Schermelleh-Engel et al., 2003), the root mean square error of approximation (RMSEA) all lower than 0.07 (Steiger, 2007) and standardized root mean square residual (SRMR) lower than 0.080 (Hu & Bentler, 1999). TLI of perceived value (= 0.948) are less than 0.95, yet still higher than 0.90 and its CFI is higher than 0.95.

Table 6 CFA results (N = 405)

Average variance extracted (AVE), composite reliability (CR) (Fornell & Larcker, 1981; Hair et al., 2014), and Heterotrait-monotrait (HTMT) ratio (Henseler et al., 2015) were further used to examine the convergent validity and discriminant validity. AVE and CR were calculated based on the factor loading of the items and the HTMT ratios were generated with the “HTMT plugin” developed by Gaskin et al. (2023).

As shown in Table 7, though some of the AVE values are lesser and closer to 0.50, yet with the CR value of the knowledge scale, value scale, and cost scale as 0.822, 0.763, and 0.873 accordingly, which exceed the cutoff point of 0.60, it may be able to conclude that the convergent validity is acceptable (Fornell & Larcker, 1981). The values of HTMT ratios (see Table 8) further denote good discriminant validity, as all the values range from 0.660 to 0.002, lower than the threshold of 0.85 (Henseler et al., 2015).

Table 7 Convergent validity results (N = 405)

Table 8 Discriminant validity results (N = 405)

Stage 3: correlations among the variables

The correlations between students’ knowledge, perceived value, and the perceived cost of using GenAI were analyzed using bivariate correlation with Pearson’s correlation coefficient (r). Pearson’s correlation coefficient is used to measure the linear relationship between factors derived from EVT and students’ intention to use GenAI in higher education for a sample of 405 participants. The results (see Table 9) suggested a relatively high and positive correlation between student-perceived value (r = 0.606, p < 0.001) and students’ intention to use. The three subscales—attainment value (r = 0.587, p < 0.001), intrinsic value (r = 0.459, p < 0.001), and utility value (r = 0.506, p < 0.001)—were also positively correlated with students’ intention to use GenAI.

Table 9 Correlation analysis results (Pearson’s correlation coefficient) (N = 405)

The correlations between students’ past use frequency (r = 0.339), students’ knowledge (r = 0.178), student-perceived cost (r =  − 0.295) of GenAI, and their intention to use were more moderate but still significant. Compared with knowledge, the connection between the student-perceived cost of using generating AI and students’ intention to use GenAI was stronger, though in a negative way.

Discussion

The findings from Table 7 demonstrates that EVT-related factors, such as knowledge, perceived value (including attainment, intrinsic, and utility values), and perceived cost, are all significantly correlated with students’ intention to use GenAI in higher education. This finding resonates with Kim et al.’s (2020) and Hu’s (2022) studies that students’ perceived ease of use and perceived usefulness of AI tools positively affect their intention to use the tools. The perceived value has the strongest positive correlation with intention to use, while the perceived cost has a weak negative correlation.

The student-perceived value of GenAI emerged as the most significant factor influencing their intention to utilize such technologies in an educational context. The majority of participants acknowledged the potential advantages of GenAI in the workplace and its capacity to enhance learning outcomes, encompassing the improvement of academic performance and the development of digital competence. Moreover, students identified utility value in aspects such as increased efficiency, provision of personalized and immediate feedback, and facilitation of idea generation. Similarly, Gado et al. (2022) and Hu (2022) found that perceived usefulness of AI had a significant relationship with students’ intention to use AI. It shows that a positive perception of how GenAI can assist or benefit students’ academic work and their future professional life is key to its adoption.

The correlation analysis between students’ knowledge of GenAI and their intention to use it revealed a statistically significant, albeit weak relationship. Previous experience with GenAI, on the other hand, had a moderate correlation with intention to use. While Dahlkemper et al (2023) showed that students who had prior experience with AI tended to have a positive attitude towards AI, our study was able to demonstrate a significant relationship between students’ frequency of GenAI use and their intention to use GenAI tools. The findings suggest that in addition to providing students with basic knowledge about GenAI, such as its definition, limitations, and benefits, it is also important to create opportunities for students to utilise GenAI or integrate its use in their university life to encourage adoption of the technology.

As perceived cost was negatively correlated with the intention to use, it suggests that reducing the perceived costs associated with the use of GenAI could potentially increase students’ intention to use it. Compared to previous studies which utilised TAM to explore the relationship between perceived usefulness, perceived ease of use, and intention to use (e.g., Bonsu & Baffour-Koduah, 2023; Hu, 2022; Kim et al., 2020), this EVT-based study shows that perceived cost is also an important factor affecting students’ intention to use GenAI. As shown in the responses to the questionnaire, students were concerned that the use of GenAI could undermine the value of a university education, deprive them of the opportunities to interact with others, and hinder the development of transferable skills. To tackle these apprehensions, the study advises fostering social and experiential learning (Chan, 2022) as well as promoting interpersonal interactions within higher education environments.

Implications

The implications of this study, which employed a validated instrument grounded in Expectancy-Value Theory (EVT) to evaluate student perceptions of GenAI in higher education, are multifaceted and have far-reaching consequences for researchers, educators, and educational institutions alike.

First and foremost, our study contributes to current understanding of factors that influence students’ acceptance and intention to use GenAI by highlighting the role of cognitive (knowledge of GenAI) and affective (perception of value and cost) factors as well as previous experience of using GenAI. By identifying these factors and their relationships with intention to use AI, the study provides valuable insights for educators and institutions looking to foster AI adoption in higher education. By emphasizing the potential value of GenAI, addressing concerns related to perceived costs, and enhancing students’ knowledge about these technologies, institutions can develop strategies and interventions aimed at promoting positive attitudes towards AI and ultimately improving the learning experience for students.

Second, the study has implications for the design of educational curricula. The findings suggest that institutions should focus on fostering AI literacy, particularly knowledge and awareness of the benefits of GenAI as these two factors were found to be correlated with students’ intention to use GenAI in this study. Knowledge, a key component of AI literacy, can increase students’ confidence and readiness in using GenAI (Ng et al., 2021). In addition, having a good understanding of AI can reduce students’ apprehensions about GenAI (Jeffrey, 2020). There should also be opportunities for students to integrate the use of GenAI in individual and collaborative tasks to allow them to explore how GenAI can enhance their learning experience without compromising interaction with peers. By incorporating these elements into their curricula, institutions can ensure that students are ready to adopt GenAI and are equipped with the skills and knowledge necessary to make the most of AI in their academic pursuits and future careers.

The role of motivation in shaping students’ adoption of GenAI is apparent in this study as demonstrated by the correlations between students’ knowledge of GenAI, previous experience of using GenAI, perceived value, and intention to use GenAI. Wigfield (1994) refers to expectancy and value as motivational constructs that determine an individual’s decision to perform and persist in tasks. Students who have used AI applications and have sound knowledge of AI tend to have a positive view of the technology (Chen et al., 2022), thus resulting in high expectancies for success. Similarly, beliefs about the importance, usefulness, and value of a task mediate one’s motivation to participate in the task (Wigfield & Eccles, 1992). Hence, classroom pedagogy should integrate motivational strategies targeting at enhancing students’ expectancies for success and instiling positive value beliefs. Integration of GenAI in academic tasks should be personalised so that students can derive satisfaction and enjoyment from its use while realising its importance to them as students and as future-ready workers upon graduation. In addition, teachers can provide guidance and advice to students on how they may tackle the challenges posed by GenAI in a task to reduce their anxiety and apprehension about GenAI use.

Additionally, the development of a validated instrument based on EVT represents a significant contribution to the field. To date, there has been a lack of robust, theoretically grounded instruments to assess students’ attitudes towards GenAI adoption, making it challenging to systematically understand the factors that influence their intention to use the technologies. The EVT-based instrument addresses this gap in the literature and provides a strong foundation for future research and practice in this area.

Limitation

Despite the valuable insights provided by this study, it is important to acknowledge its limitations. Firstly, the sample size was restricted to 405 participants, which may not be fully representative of the larger student population. Additionally, the study was conducted at a single point in time, and thus, it may not account for potential changes in students’ attitudes and perceptions towards GenAI over time. The focus on higher education students also limits the generalizability of the findings to other age groups or educational contexts. Furthermore, the study primarily relied on self-reported data, which may be subject to response biases, such as social desirability or recall bias. Lastly, the study did not explore the influence of individual differences, such as cultural background, personal experiences, and learning styles, which could also impact students’ perceptions of GenAI.

It is essential to acknowledge that skipping EFA in the validation can have some limitations, such as the potential to overlook alternative factor structures or issues with item loadings (Fabrigar et al., 1999). However, given the strong theoretical basis, prior research, and focus on hypothesis testing, using only CFA in this study can be considered a justifiable decision. To address potential concerns, it may be helpful to consider conducting additional validation studies in the future to further explore the factor structure and psychometric properties of the survey instrument.

Future research should aim to address these limitations by employing larger and more diverse samples, conducting longitudinal studies, and examining the impact of individual differences on the relationship between perceived value, perceived cost, and the intention to use GenAI.

Conclusion

This study explored students’ perceptions of GenAI using an EVT-based instrument. The findings reveal a significant correlation between students’ knowledge of GenAI, previous use of GenAI, perceived value, and intention to use, thus highlighting the role of motivation in shaping students’ decision to adopt GenAI. Educational initiatives to promote GenAI use should focus on enhancing expectancies for success and fostering positive value beliefs through personalised learning experience and strategies for mitigating GenAI risks. As GenAI is rapidly becoming a global trend and reshaping the practices of various industries, higher education has an important mission to prepare a future-ready workforce that is able to Malechwanzi et al. (2016). utilise and collaborate with GenAI effectively. To achieve this goal, students’ acceptance and willingness to adopt GenAI are crucial and there is an urgent need for research into this area. This study offers a validated instrument for measuring students’ perceptions of GenAI, which can be used by researchers for further studies of GenAI adoption.