Skip to main content

On the relationship between mobile-based extramural activities and vocabulary development of EFL learners: a mixed-method study

Abstract

Mobile-based extramural English (EE) activities, including watching movies, playing digital games, and listening to music outside the classroom, have played an important role in increasing learners' interest in English as a foreign language (EFL). This mixed-method design study aimed to investigate the role of EE activities in the vocabulary development of intermediate EFL learners. The study included a group of 60 learners, including 30 male and 30 female students from two language institutes. A pretest, posttest, and delayed posttest were used. The experimental group utilized WhatsApp to receive electronic vocabulary instruction four days per week for two months as part of the EE activities. In contrast, the control group received classroom instruction using the conventional approach typically employed in EFL contexts for teaching the same textbook vocabulary. A semi-structured interview was conducted to investigate the participants' beliefs regarding their preference for EE-based activities. The results of the EG group revealed that EE-based activities significantly impacted the students' vocabulary learning (p < 0.001). The findings also revealed no significant differences in vocabulary knowledge between male and female students after using EE activities (p = 0.34). The themes that emerged from the qualitative phase suggested that participants preferred EE-based activities as they can result in less mental stress, more vocabulary learning, improved conversation, and improved pronunciation and grammar. Also, the participants preferred various mobile-based EE activities, including watching movies, listening to music, playing multiple games, watching cartoons, and focusing on movie subtitles, which resulted in their vocabulary development. The findings call for language educators to integrate more virtual and online-based EE activities to supplement traditional vocabulary teaching practices.

Introduction

Extramural English learning encompasses naturalistic learning, self-instruction, and self-directed learning (Benson, 2011). It refers to any learning that occurs outside of the traditional classroom environment. Such learning activities, including digital games, movies, music, and other extracurricular English (EE) activities, have contributed significantly to students' interest in English and their motivation to achieve high academic performance. In teaching English as a foreign language (EFL) in Iran, the researchers observed a significant gap in English proficiency between students who routinely participated in EE activities and those who did not. With the development of technology and the Internet, there are now more opportunities to practice English outside of the classroom than twenty years ago. In addition, online communication platforms and social media have facilitated students' daily language use. Among the technological advancements in mobile communication, WhatsApp (henceforth, WA) has become a popular instrument in numerous academic disciplines. WA, initially introduced in 2009 as an instant messaging platform, now has approximately two billion monthly active users. In recent years, a growing body of research has examined the use of WA in diverse educational domains, such as computer science (Guler, 2016), language learning (Farahian & Parhamnia, 2022; García-Gómez, 2022), midwifery (Gonenc et al., 2021), and psychology (Alshaibani & Qusti, 2021). In a recent review by Manca (2020) concentrating on social media platforms as language-learning tools, WA was identified as the platform with the most research. Several studies in the field of English as a foreign and second language have demonstrated the positive effects of using WA on students' autonomous motivation (Alamer & Al Khateeb, 2021), engagement (Saritepeci et al., 2019), and vocabulary acquisition and retention (Ankeny, 2019; Cetinkaya & Sutcu, 2018). In addition, WA has been identified as advantageous for the professional development of language teachers, as it facilitates collaboration, material development, and the enhancement of English language knowledge (Motteram et al., 2020).

The difficulties Persian speakers face when acquiring English are well documented. The acquisition of foreign languages, especially English, has become a major concern for language learners. To effectively learn the language, students spend countless hours in classrooms and at home engaging in activities such as viewing videos, listening to audio files, and listening to music.

In the past, the primary source of information for learning a foreign language was the classroom. However, the current landscape has expanded to include learning in various non-traditional contexts, particularly for English as a second language, which is readily available online, in the media, and the swiftly expanding entertainment world. The Internet has facilitated access to resources in multiple languages and provides numerous communication instruments in the target language. In contrast to the traditional learning paradigm, the current study concentrates on computer-assisted language learning in extracurricular and extramural settings where learning is linked to educational institutions (Sylven & Sundqvist, 2017).

In addition, research indicates that playing video games, which provide entertainment for children, adolescents, and young adults, can also improve English language skills. Previous research has established a positive correlation between extensive gaming and success on English vocabulary tests for EFL students (Brevik, 2016; Olsson, 2016; Sundqvist, 2009, 2019; Sundqvist & Wikström, 2015; Sylvén & Sundqvist, 2012). In the context of EFL in Iran, however, the potential contribution of extramural learning to the language proficiency of Iranian EFL learners, particularly in terms of vocabulary development, warrants further study. Consequently, this study aims to investigate the relationship between EFL students' language knowledge and EE-based activities, with a particular concentration on the proficiency of intermediate EFL learners in receptive vocabulary at an English Language Institute (ELI) in the Iranian EFL context.

A further significant drawback in language learning education is the absence of a robust educational mobile messaging application designed for use in school curriculums. This shortcoming hinders the effective incorporation of the energy exerted by these applications' users into education and learning. It is hoped that a significant portion of language learning can occur outside of traditional classrooms, in the comfort of learners' residences, by incorporating an engaging mobile messaging application such as WhatsApp into language learning curricula for all age groups. Adult students who spend lengthy time in the classroom learning English frequently experience discomfort and disinterest (Dong, 2004). Learning vocabulary requires stimulation, which can be reinforced by associating it with meaning (Miramontes et al., 1997). Recognizing the existing research gap regarding the role of mobile messaging applications in enhancing L2 learners' vocabulary knowledge in Iran, this study seeks to investigate the potential impact of E-learning on enhancing the vocabulary knowledge of Iranian EFL students. In addition, the study seeks to determine whether there are significant differences between male and female Iranian EFL students in their use of E-learning. This study aims to contribute to the existing body of knowledge in applied linguistics and provide language educators and policymakers with useful insights by examining the relationship between extramural English activities and language proficiency, particularly in terms of vocabulary development, as well as the impact of using WhatsApp as an E-learning tool.

Literature review

It has been demonstrated that exposure to a language incidentally is beneficial for vocabulary learning. Researchers contend, however, that intentional/explicit learning generally results in faster improvements and greater retention, leading to higher levels of mastery (Schmitt & Schmitt, 2020). Even though explicit vocabulary instruction is more advantageous and effective, some measures can be taken to help students achieve their objectives more efficiently. Most textbooks lack a transparent and systematic approach to vocabulary instruction (Schmitt & Schmitt, 2020), so relying solely on textbooks for language learning is problematic making.

Extramural English (EE) has been demonstrated to enhance learning a new language. Several studies have explored the beneficial relationship between frequent exposure to extramural English and language learning effectiveness, highlighting the importance of out-of-class activities for achieving good grades and language proficiency (Olsson, 2016; Lai et al., 2015; Sundqvist & Wikström, 2015; Smemoe et al., 2012; Olsson, 2012; Sylvén & Sundqvist, 2012). Due to the increased need for repetition, Matsuoka and Hirsh (2010) discovered that authentic texts are superior for vocabulary learning. Cobb (2007) also emphasized the significance of reading authentic texts rather than relying solely on textbooks to recycle vocabulary for effective learning. Therefore, teachers should modify their curricula to more effectively cover textbooks and consider adding supplements to introduce new vocabulary (Schmitt & Schmitt, 2020).

The benefits of learning vocabulary outside of the classroom are remarkable (Nation, 2013). It not only relieves the teacher of challenges but also enables material to be related to real-world contexts, resulting in more fruitful learning. It has been discovered that incidental learning, which closely resembles most first-language vocabulary development, is more effective than direct, purposeful learning for vocabulary acquisition (Nation, 2013). This holds for L2 learners as well. Lindgren and Muñoz (2013), Jensen (2016), and De Wilde and Eyckmans (2017), for instance, conducted research demonstrating that young infants can acquire a substantial vocabulary through exposure to the target language. De Wilde and Eyckmans (2017) found that children in Belgium without formal or informal training in the target language scored 66.20 out of 108 on the Peabody Vocabulary Test, highlighting the potential advantages of incidental learning. Lefever (2010) discovered that most learners can comprehend fundamental spoken English, with many able to engage in simple conversations even before formal education. This demonstrates how young language learners acquire L2 vocabulary predominantly through extramural English, frequently through media exposure (Lefever, 2010; Schmitt & Schmitt, 2020).

With the prevalence of extracurricular English activities such as social media, online gaming, viewing television, movies, and YouTube, young language learners have numerous daily opportunities to acquire incidental vocabulary. Several studies have demonstrated (Peterson, 2013) that playing English-language computer games significantly enhances learning outcomes. Many websites also facilitate explicit instruction and vocabulary acquisition (Schmitt & Schmitt, 2020). Schmitt and Schmitt (2020) contend that research on extramural English activities and their effects on language development will likely become a major focus in the coming decades. De Wilde et al. (2020) studied Dutch secondary school students. They determined that speaking English, playing video games, and utilizing social media are the three most important forms of L2 input for various linguistic components, including vocabulary. These extracurricular English activities facilitate second language acquisition by fostering social connections and authentic participation. In the past decade, several studies have investigated the incidental acquisition of L2 through online gaming (Jensen, 2016; Kuppens, 2010; Sylvén & Sundqvist, 2012).

In addition, several studies have highlighted the significance of watching brief, subtitled films for vocabulary development (d'Ydewalle & Van de Poel, 1999; Koolstra & Beentjes, 1999). Peters and Webb (2018) discovered that viewing a single, full-length television program with contextual vocabulary learning significantly improved vocabulary knowledge. According to Ryu (2013), gaming can also aid in language acquisition through continuous exposure to the target language and interactions with native or more proficient peers in massively multiplayer online games (MMOGs) (Peterson, 2010). In the field of second language acquisition (SLA), research on vocabulary learning in gaming has garnered prominence (Jabbari & Eslami, 2019; Reinders, 2017). Published meta-analytic analyses on game-based vocabulary teaching and learning support the efficacy of gaming as an instrument for vocabulary acquisition (Chen et al., 2018; Chiu, 2013). In addition, qualitative research has demonstrated that learners are motivated to acquire the language used in games and engage in language-related explanations and feedback with other players (Bytheway, 2015; Turgut & İrgin, 2009). In a recent study by Zeidan (2020) investigating the relationship between extracurricular English activities and performance among Swedish school students, gamers performed better than non-gamers on vocabulary tests, indicating a potential link between gaming and vocabulary development.

Previous studies (Grant & Rong, 1999; Wong et al., 2002) have attempted to explain the significance of gender differences in L2 academic achievement. In addition to the social context, gender as an individual factor determines significant differences in the failure or success of second language acquisition (Catalán, 2003). Gender may influence a second-language learner's success, but it is not always predictable (Bidlake, 2005). Several ideas regarding vocabulary acquisition coalesce around the performance of the sexes. Several studies (Geiger & Litwiller, 2005) attempted to explain why men have superior word and meaning memory. However, according to Gurian and Ballew (2003), most the female brains develop earlier and quicker than male brains, allowing females to acquire complex verbal skills before males. Consequently, some researchers (Anderson, 2001; Lowe et al., 2003) emphasize that females are superior to males in verbal and language tasks. The objective of Salamat and Pourgharib's (2013) study was to enhance EFL students' speaking skills through mobile phones. There were forty male and female EFL enrollees between 18 and 25. Participants who had benefited from mobile-assisted learning performed significantly better on a speaking post-test than participants in the control group, as indicated by the results.

Numerous studies have investigated the impact of extracurricular English activities on language learning. Still, there is a notable dearth of research on the role of such activities in vocabulary development among Iranian EFL learners. Xodabande (2017) found that mobile-assisted learning via platforms such as Telegram effectively teaches Iranian EFL students English pronunciation. In addition, Ataeifar et al. (2019) evaluated the impact of mobile-assisted instruction on the English-speaking skills of Iranian female students. However, research on the vocabulary development of Iranian EFL students through extracurricular English activities is required.

In conclusion, while previous research has highlighted the benefits of incidental and deliberate vocabulary acquisition, extracurricular English activities, such as gaming and media exposure, have emerged as a prospective means of enhancing vocabulary acquisition. To investigate the specific effects of extramural English activities on vocabulary development among Iranian EFL learners, additional research is required. This study seeks to fill this void by examining the effect of extramural English activities on vocabulary improvement in the Iranian EFL context, addressing the limitations and gaps in the existing literature, and providing a rationale for conducting the current research. To address the gaps mentioned, this study was conducted to answer the following postulated questions:

  1. 1.

    Is there a significant relationship between Extramural learning and vocabulary development of Iranian EFL learners in English Institutions?

  2. 2.

    Can gender significantly affect EE learning in an Iranian Extramural context?

  3. 3.

    How do Iranian EFL learners perceive the impact of using Extramural activities outside the classroom walls?

Methodology

Design

Depending on the nature of the questions and the types of data required for answering the questions, the study employed a mixed-method research approach. The overall purpose and central premise of mixed methods studies is that combining quantitative and qualitative approaches provides a better understanding of research problems and complex phenomena than either approach alone (Creswell & Plano Clark, 2007). Better understanding can be obtained by triangulating one set of results with another and thereby enhancing the validity of inferences (Greene et al., 1989).

It also examined the co-relationship between EE and learning vocabulary at language teaching Institutes across Kermanshah. The present study is, therefore, theoretically aligned with the underpinnings of the mixed-method approach (Creswell & Creswell, 2018) with a convergent-parallel perspective. Regarding the design, the study enjoyed convergent mixed methods (Creswell & Creswell, 2018) with a Data Transformation (Edmonds & Kennedy, 2017) way of analyzing the data. The reason behind selecting convergent mixed methods design was that in such a design, the researchers converge or blend both quantitative and qualitative data to provide a more comprehensive analysis of the research problem. Both quantitative and qualitative data were collected and analyzed concurrently. The qualitatively-gained themes or categories are transformed into quantitative counts, interrelated with quantitative data, while comparisons are made in the interpretation phase of the study.

Participants

The present study was conducted among 60 Intermediate EFL learners already learning English in language teaching institutes in Iran. Of the 60 participants, 30 were males and 30 females, with 15 of each sex devoted to controlling and the other 15 to the experimental group. The age range of participants was from 12 to 16. The participant’s native language was either Persian or Kurdish. All of the participants were learning English as their foreign language. The majority of randomly selected participants were language students of one of the researchers. Based on the participants’ informed consent, some participated in a semi-structured interview that probed into the learners’ perception of EE-based activities. Out of 60 participants, five males and five females were selected to conduct the qualitative phase of the study. Since data saturation was achieved through the data collected from the ten interviewees, no further participant was interviewed.

Instruments

Based on the nature of the study and the type of data required, two types of instruments were employed to collect the necessary data: semi-structured interviews to explore the perception of participants regarding the usefulness of EE for vocabulary development and the Nation’s online vocabulary test to measure the intermediate EFL students’ knowledge of matching vocabulary test. The function of the test was prognostic, aiming at specifying the proficiency level of the participants before the study. The test comprises just one section, namely vocabulary, 33 matching items in bundles of three. For the qualitative phase of the study, five questions in the form of semi-structured interviews were devised (see “Appendix 1”). The questions aimed at eliciting the perception of the experimental learners, a convenient number of them, regarding the nature and convenience of extramural vocabulary learning.

Procedure

Following the pretest, the specified-for-the-study participants participated in the required activities for eight weeks. During experimentation, the EG group received the EE activities prepared through WhatsApp. For eight weeks, they learned vocabulary through electronic media twice weekly (16 sessions). Students were taught five new words in each session by contextualizing the targeted words by different means, i.e., via films, games, subtitles, music, etc. All learners in the EG were members of the same WhatsApp chat group. Different videos and cartoons available on YouTube were also sent to them, conveying the meaning of the new vocabulary items.

The participants in the (CG) group were taught the same vocabulary as the EG group in traditional ways. The CG group was taught vocabulary in the classroom, repeated, practiced, and completed the exercises, which included fill-in-the-blank exercises and sentence completions based on the targeted words. In other instances, they were required to assemble meaningful sentences from groups of related words. Although the students did not interact, they had common assignments. Different exams containing common words were administered. To convey the meaning of the new vocabulary items to the EG group, YouTube videos and animations were also sent to them.

After eight weeks of using EE activities, the participants were given a similar version of the pretest with different item arrangements (counterbalancing) used as a posttest. After two months, the study participants were also given a delayed posttest to ensure the results' reliability. For the qualitative phase of the study, a semi-structured interview was carried out after the participants had taken the vocabulary test, aiming to answer the research question related to the student’s attitudes toward EE-based activities. The interview comprised five questions about the learners' perception and personal opinion of English language learning with extracurricular activities. (“Appendix 2”). Ten experimental group members (EG) willing to be interviewed were interviewed for this purpose (see “Appendix 2”). It took each participant ten minutes and was conducted by the researcher herself. Their responses were captured and transcribed for analysis purposes. The interviews were conducted in Persian, but the transcript was translated into English.

Data collection and analysis procedure

An independent sample T-test was run to compare the experimental versus control group performance at the pretest, posttest, and delayed posttest phases. Regarding the qualitative phase of the study and to explore the perception of the experimental participants on EE-based activities, Boyatzis's (1998) thematic analysis was conducted. The study's analytical framework has been schematized below for ease of interpretation. This study used Boyatzis's (1998) thematic analysis to analyze the qualitative data obtained from interviews. Thematic analysis is a frequently employed technique for identifying and interpreting patterns, themes, and categories within qualitative data. It entails a methodical process of coding and organizing the data to extricate valuable insights. The interviews were transcribed as the initial stage in the analysis procedure. The process of converting audio recordings of interviews into written text is transcription. This step ensures the data is readable, making analysis and interpretation simpler. After the interviews were transcribed, the researchers thoroughly examined and familiarized themselves with the data. This step involved multiple readings of the transcripts to immerse themselves in the content and gain a comprehensive comprehension of the perspectives and experiences of the participants regarding extramural English activities and vocabulary learning. The subsequent phase involved identifying initial codes or labels that captured significant concepts, ideas, or patterns within the data. Codes are distinct words or brief phrases that label meaningful data units. Instead of imposing predetermined categories, the researchers used an inductive methodology, allowing the codes to emerge directly from the data. They assigned pertinent codes to text segments that reflected similar concepts or ideas. After coding the data, the researchers organized and classified the codes to identify overarching themes. Themes are broader, more abstract concepts that incorporate multiple codes and provide a framework for conceptualizing the data. The researchers examined the codes and their relationships to identify similarities, differences, and patterns, which aided in the development of meaningful themes. The identified themes were examined and refined to ensure consistency and coherence. To ensure the accuracy and validity of the interpretations, the researchers engaged in a process of constant comparison, moving back and forth between the themes, the coded data, and the original transcripts. The researchers then analyzed the data within each theme to gain meaningful insights and reach conclusions. They analyzed each theme's content, considering the context, nuances, and variations in participant responses. This process entailed synthesizing the findings, distinguishing significant patterns or trends, and establishing links between the themes and the research objectives. Boyatzis's thematic analysis presented a systematic and rigorous method for analyzing the qualitative interview data. It consisted of transcribing the interviews, identifying codes, organizing codes into themes, revising and refining the themes, and analyzing the data within each theme to generate insightful conclusions. This process thoroughly explored participants' perspectives and experiences, contributing to a comprehensive understanding of the role of extramural English activities in Iranian EFL learners' vocabulary learning (Table 1).

Table 1 Schematic representation of analytical tools

Results

Results of the pretest

The results of the pretest are displayed in Table 2. The control and experimental groups' mean scores were 16.30 and 15.95, with a standard deviation of 1.78 and 1.97, respectively. Participants in the control and experimental groups did not have significantly different vocabulary pretest results (p = 0.41, Table 2).

Table 2 Results of the independent samples t-test for comparing the pretest score of the experimental group and the control group

Additionally, the mean scores on the vocabulary pretest for the participants—male and female—were 14.90 and 1.54, respectively, with standard deviations of 1.51 and 1.42. For the vocabulary pretest, there was no statistically significant difference between males' and females' scores (p = 0.31, Table 3).

Table 3 Results of the independent sample t-test for comparing male and female pretest scores in the experimental group

Tables 2 and 3 demonstrate that there is no statistically significant distinction between the experimental and control groups' performance. Additionally, there was no significant difference in performance between male and female participants' language knowledge before the intervention.

First research question

The first research question of the study was, “Is there a significant relationship between Extramural learning and vocabulary development of Iranian EFL learners in English Institutions”? As Table 2 demonstrates, there is no significant difference between EG and CG groups regarding their vocabulary knowledge before intervention. However, the results for the posttest show that the intervention significantly changed the groups' performance regarding their vocabulary knowledge. The results of the analysis are summarized in Table 4 below.

Table 4 Results of the independent sample t-test for comparing the control group and experimental group posttest score

The control and experimental groups' respective mean posttest vocabulary test scores were 19.32 and 23.91, with standard deviations of 1.97 and 3.65. For the students in the control and experimental groups, there was a statistically significant difference between their vocabulary posttest results (p < 0.001, Table 4).

To ensure the longevity of the effect of the intervention, a delayed posttest was also conducted with a 2-month postponement, comparing the performance of the EG and CG group. As the results in Table 5 demonstrate, the difference between EG and CG group regarding their vocabulary retention is still significant following the intervention.

Table 5 Results of the independent sample t-test for comparing the control group and experimental group delayed post -test score

As Table 5 demonstrates, the mean of delayed posttest of the control and experimental group is 18.99 and 22.16, with a standard deviation of 1.99 and 3.66, respectively. There are a statistically significant difference in vocabulary delayed posttest scores for the learners in the control and experimental groups (p < 0.001, Table 6), showing that the effect of teaching vocabulary through different means of EE is significantly long-lasting.

Table 6 Results of the independent sample t-test for comparing male and female posttest scores in the experimental group

Second research question

The study's second research question asked, “Can gender play a significant role regarding EE-based learning in an Iranian Extramural context?”. Before the intervention, there was no significant variance in the male and female groups' language knowledge, as shown in Table 3. However, the results for the posttest show that the intervention does not favor either the male or female group regarding their vocabulary knowledge following the intervention. To put it simply, there is no significant difference between male and female participants in their vocabulary knowledge due to the effect of the intervention. The results of the analysis are summarized in Table 6 below.

In terms of the participants' gender, the mean scores for men and women in control and experimental groups were 25.51 and 24.31, respectively, with standard deviations of 3.54 and 3.37. There is no statistically significant difference in vocabulary post-test scores for males and females (p = 0.34, Table 5). Since the difference between the male and female groups regarding their vocabulary knowledge following the posttest was insignificant, no delayed posttest was for this question.

Third research question

The third research question aimed to delve into the participants' perceptions regarding the impact of EE-based activities outside the classroom walls. The question was postulated:” How do Iranian EFL learners perceive the impact of using Extramural activities outside the classroom walls?” To this end, 10 participants were invited to attend an interview session in which five questions were posed. The interviews were recorded, transcribed, and translated into English, and then the participants' answers were thematically coded. The themes of participants’ answers to each question are presented as follows:

First interview question

The first interview question asked, “What is your idea about out-of-class activities through various social networks?’’ The thematically coded answers to this question are presented in Table 7. It should be noted that in the table below and the upcoming tables reporting the frequency of answers, the cumulative frequency may be above the total number of participants, as some participants may have answered more than one code.

Table 7 Thematic answers to the first interview question

As reported in Table 7, half of the interview participants believe that out-of-class activities can result in more learning. They believe such activities' funny and relaxed atmosphere makes them feel tranquil while learning vocabulary. Also, a significant number of interview candidates, 40%, hold the belief that out-of-class activities can act as stress reducers, which can be conducive to learning. Moreover, one of the participants suggested that activities outside the classroom can be a guide to improvement as they are optional, and this makes the atmosphere ready for the learner to scaffold his/herself based on his present capability.

Second interview question

The second interview question asked “What are the probable advantages and disadvantages of out-of-class activities done through social networks?’’ The thematically coded answers to this question are as follows:

As reported in Table 8, half of the participants agree that out-of-class activities can improve the learners' conversation ability. They attributed this to the extended exposure of the learners to the vast amount of dialogic materials. Also, half of the interview candidates believed that out-of-class activities via social networks could contribute to more vocabulary learning for the participants. Further, 30% of the interview participants supported the idea that via-social-network activities conducted outside of the classroom context can result in deeper learning as there is no presence of stress. Further, one of the candidates believed that the application of social networks outside the classroom context could be a good asset for those who are, due to illness or some other problems, unable to keep pace with the classroom procedure. More to add, 4 of the participating candidates 4 of them complained that out-of-class activities through social networks might not be so rewarding as there is unfortunate disruption and disconnection of the Internet, which in turn results in nervousness and anxiety.

Table 8 Thematic answers to the second interview question

Third interview question

The thematically coded answers to the third interview question are presented in Table 9.

Table 9 Thematic answers to the third interview question

As reported in Table 9, half of the 50% of participants believed that applying social networks outside the classroom could improve vocabulary. Also, 20% of Participant 2 believed attending social networks for out-of-class activities could improve the conversation. Interesting to mention that just one of the participants regarded social networks as contributors to grammar improvement. Last but not least, 30% of the participants believe that using social networks outside the classroom has been conducive to improving their pronunciation.

Fourth interview question

The fourth interview question asked, “Can you, by example, refer to some out-of-class activities conducted via social networks which have been rewarding to your learning?” The thematically coded answers to this question are presented in Table 10.

Table 10 Thematic answers to the fourth interview question

As reported in Table 10, the interview participants had various ideas regarding activities to follow outside the classroom. Eighty percent of the interviewees believed that frequent movie watching is very determined in their language improvement. What comes close to watching the movie is playing various games. Seventy percent of the interviewees believed that playing games accompanied by English speaking improved their English significantly. They attributed this to their strong focus on the game and the speaker's emotional tone while playing it. Listening to music was also considered a positive classroom activity for language improvement. Forty percent of the participants held such a belief. What comes next is watching cartoons. Three participants regarded watching cartoons as a very effective out-of-class activity for their language improvement. Also, 2 of the participants referred to subtitles of the movies as an effective out-of-class activity for better language improvement.

Fifth interview question

The fifth interview question asked, “which type of difference did you witness between out-of-class activities via social networks you experienced and previous traditional language learning activities?’’ The thematically coded answers to this question are presented in Table 11.

Table 11 Thematic answers to the fifth interview question

As reported in Table 11, 40% of the interviewees referred to stress-free learning opportunities created by out-of-class learning through social networks. As noted earlier in previous questions, the participants believed that reducing stress in enjoyable learning opportunities from social networks can result in better learning. Next to stress-free learning opportunities was teacher imposition which caught the attention of the interviewees. Three participants believed that with the employment of social networks, there is no imposition which results in less stress and more learning. Also, 2 of the participants suggested that, when they use networks, there is no force for choosing types of materials or kinds of activities, which can result in their enhanced interest in following learning activities. It is noticeable to mention t the interviewees mentioned the management of time by learners in out-of-class activities performed via social networks.

Discussion and conclusion

The first and second research questions investigated the relationship between extramural English (EE) activities and vocabulary acquisition in English institutions in Iran, as well as the potential role of gender in EE acquisition in an Iranian extramural context. A comparison of pre- and post-test results revealed that participation in EE activities significantly impacted students' vocabulary skills. These results support the findings of Gooniband Shooshtari et al. (2013), who demonstrated the efficacy of mobile device applications in teaching English vocabulary items to semi-literate students. In addition, the results of this study are consistent with those of Basoglu (2010), who found that participants using mobile applications had superior word recall abilities compared to those employing conventional methods. Stickler and Hampel (2010) emphasized incorporating various learning and teaching strategies, such as communicative language use, form-focused instruction, and language practice, in online language programs. These findings are also consistent with the findings of the current study. Moreover, Taki and Khazaei (2011) investigated the presentation of language in written and visual forms on mobile devices, and their findings demonstrated that studying vocabulary with pictorial or textual annotations resulted in superior performance compared to no annotations. These findings are consistent with the research conducted by Thornton and Houser (2005), who investigated the use of mobile devices in language learning and found that students had favorable opinions of educational smartphone applications that enhanced their learning experiences and sparked their interests.

Furthermore, this study found no significant differences in vocabulary knowledge post-test performance between male and female pupils. This result is consistent with the findings of Salamat and Pourgharib's (2013) study, which examined the effectiveness of EE activities in enhancing the speaking abilities of EFL students.

Regarding the third research question, the thematic analysis of the semi-structured interview questions revealed that respondents favored engaging in EE activities outside the classroom. They believed that these activities reduced mental stress, encouraged more learning and improvement, enhanced conversation skills, facilitated vocabulary acquisition (De Wilde & Eyckmans, 20172017; Jensen, 2016; Lindgren & Muñoz, 2013; Nation, 2013), promoted deeper learning, and enhanced pronunciation (Nation, 2013; Sylvén & Sundqvist, 2012). These findings are consistent with the findings of Kuppens (2010), Nation (2013), Schmitt and Schmitt (2020), and Sylvén and Sundqvist (2012), which all unequivocally support the notion that engaging in EE activities outside of the classroom context results in enhanced L2 development. In addition, regarding the preferred form of EE application, the participants mentioned various activities with varying frequencies, such as watching movies, listening to music, playing video games, watching cartoons, and concentrating on movie subtitles. This result is consistent with the findings of Jensen (2016), Ryu (2013), and Zeidan (2020), who found that individual preferences for EE applications vary and that various applications may result in varying levels of L2 development.

Incorporating mobile-assisted language learning into a foreign-language curriculum presents numerous benefits and challenges. Benefits include facilitating collaboration through group project tasks, providing learning opportunities even during pandemic periods when physical classroom interactions are limited, and enhancing students' vocabulary through engaging EE activities. These results are consistent with previous research demonstrating the positive impact of mobile device applications on vocabulary acquisition, recall skills, and overall language development. The use of online language programs and the incorporation of various learning and instructional strategies strengthen the efficacy of mobile-assisted language learning in fostering language skills.

However, some obstacles must be overcome. Cost and the possibility of a digital divide, in which pupils from disadvantaged backgrounds may have limited access to mobile devices and internet connectivity, represent a significant obstacles. This issue must be addressed to assure equitable access to mobile-assisted language learning resources. In addition, creating or locating a suitable mobile platform for language learning can be difficult for educators and institutions. Pedagogical considerations and research-based evidence should guide the selection of mobile applications and platforms. In addition, the recruitment of native speakers or language experts to provide authentic language input and cultural insights can pose difficulties in the implementation of mobile-assisted language learning initiatives.

In conclusion, incorporating mobile-assisted language learning into a foreign curriculum has numerous advantages, such as enhanced vocabulary knowledge, collaboration, and flexible learning opportunities. Cost, the digital divide, platform selection, and the recruitment of native speakers are, however, obstacles. Educators can effectively integrate mobile-assisted language learning to enhance language acquisition and create engaging learning environments by addressing these challenges and capitalizing on the benefits. Additional research and ongoing evaluation of mobile-assisted language learning strategies can contribute to enhancing language education practices.

The findings of this study could be advantageous for field stakeholders in several ways. The study revealed that EE-based activities are more effective than traditional instructional methods in developing vocabulary. To this end, instructors can be more thoughtful in selecting the types of EE applications that can most effectively support them during vocabulary instruction. In addition, teachers must recognize that they must abandon their traditional methods of teaching vocabulary in favor of virtual and online-based practices. The outcomes are also advantageous for TTC instructors, who can recommend and introduce various websites and applications to the mentored and supervised teachers during mentoring and supervising sessions. As for material designers, the outcomes can be beneficial. Material designers should be aware that including new techniques for EE-based activities in their course texts and supplementing EE-based applications can enrich their designed materials.

There were some limitations in this study. First, due to limitations on L2 classes in English institutes during the Covid-19 outbreak, the sample size was small because the researcher had limited access to additional participants with the same characteristics as the target participants. Therefore, a study with a larger sample size may produce a more accurate analysis and more comprehensive findings regarding the effects of EE activities on vocabulary development. Second, the research was conducted with L2 learners of intermediate proficiency. Research conducted on other levels of proficiency may produce different results. Thirdly, this research was conducted in an EFL setting, where exposure to English outside of the classroom differs from ESL settings. Comparatively, conducting the study in an ESL context could yield distinct results. The study's EE-based activities were conducted primarily via WhatsApp. Using other EE-based applications to conduct the research could result in different findings.

Availability of data and materials

Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.

References

  • Alamer, A., & Al Khateeb, A. (2021). Effects of using the WhatsApp application on language learners’ motivation: A controlled investigation using structural equation modelling. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2021.1903042

    Article  Google Scholar 

  • Alshaibani, M. H., & Qusti, E. S. (2021). The role of smartphone app “WhatsApp” on achievement motivation and social intelligence among female undergraduate students. Perspectives in Psychiatric Care, 57(2), 597–603. https://doi.org/10.1111/ppc.12582

    Article  Google Scholar 

  • Anderson, J. (2001). Net effect of memory collaboration: How is collaboration affected by factors such as friendship, gender, and age? Scandinavian Journal of Psychology, 42, 367–375. https://doi.org/10.1111/1467-9450.00248

    Article  Google Scholar 

  • Ankeny, R. (2019). Interweaving technology and language goals to promote academic vocabulary: The role of WhatsApp in an intensive English classroom. TESOL Journal. https://doi.org/10.1002/tesj.457

    Article  Google Scholar 

  • Ataeifar, F., Sadighi, F., Bagheri, M. S., & Behjat, F. (2019). Iranian female students’ perceptions of the impact of mobile-assisted instruction on their English-speaking skill. Cogent Education, 6(1), 1662594.

    Article  Google Scholar 

  • Basoglu, E. B. (2010). Comparison of English vocabulary learning levels of students using mobile phones and vocabulary cards. Master thesis, Zonguldak Karaelmas University, Zonguldak, Turkey.

  • Benson, P. (2011). Language learning and teaching beyond the classroom: An introduction to the field. Palgrave Macmillan.

    Book  Google Scholar 

  • Bidlake, E. (2005). Language learning as a gendered experience. Retrieved April 29, 2012, from the World Wide Web: http://research.ncl.ac.uk/ARECLS/volume_4/BIDLAKE.pdf

  • Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development. Sage.

    Google Scholar 

  • Brevik, L. M. (2016). Gamers, surfers, social media users: Unpacking the role of interest. Journal of Computer Assignment. https://doi.org/10.1111/jcal.12362

    Article  Google Scholar 

  • Bytheway, J. (2015). A taxonomy of vocabulary learning strategies used in massively multiplayer online role-playing games. CALICO Journal, 32(3), 508–527.

    Article  Google Scholar 

  • Catalán, R. (2003). Sex differences in L2 vocabulary learning strategies. International Journal of Applied Linguistics, 13, 54–77. https://doi.org/10.1111/1473-4192.00037

    Article  Google Scholar 

  • Cetinkaya, L., & Sutcu, S. S. (2018). The effects of Facebook and WhatsApp on success in English vocabulary instruction. Journal of Computer Assisted Learning, 34(5), 504–514. https://doi.org/10.1111/jcal.12255

    Article  Google Scholar 

  • Chen, M.-H., Tseng, W.-T., & Hsiao, T.-Y. (2018). The effectiveness of digital game-based vocabulary learning: A framework-based view of meta-analysis. British Journal of Educational Technology, 48(1), 69–77.

    Article  Google Scholar 

  • Chiu, Y.-H. (2013). Computer-assisted second language vocabulary instruction: A meta-analysis. British Journal of Educational Technology, 44(2), E52–E56.

    Article  Google Scholar 

  • Cobb, T. (2007). Computing the vocabulary demands of L2 reading. Language Learning & Technology, 11, 38–63.

    Google Scholar 

  • Creswell, J., & Plano Clark, V. (2007). Designing and conducting mixed methods research. Sage.

    Google Scholar 

  • Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches. Sage.

    Google Scholar 

  • d’Ydewalle, G., & Van de Poel, M. (1999). Incidental foreign-language acquisition by children watching subtitled television programs. Journal of Psycholinguistic Research, 28(3), 227–244.

    Article  Google Scholar 

  • De Wilde, V., & Eyckmans, J. (2017). Game on! Young learners’ incidental language learning. Studies in Second Language Learning and Teaching. https://doi.org/10.14746/ssllt.2017.7.4.6

    Article  Google Scholar 

  • De Wilde, V., Brysbaert, M., & Eyckmans, J. (2020). Learning English through out-of-school exposure. Which levels of language proficiency are attained and which types of input are important? Bilingualism: Language and Cognition, 23(1), 171–85.

    Article  Google Scholar 

  • Dong, Y. R. (2004). Teaching language and content to linguistically and culturally diverse student. Information Age Publishing.

    Google Scholar 

  • Farahian, M., & Parhamnia, F. (2022). Knowledge sharing through WhatsApp: Does it promote EFL teachers’ reflective practice? Journal of Applied Research in Higher Education, 14(1), 332–346. https://doi.org/10.1108/JARHE-12-2020-0456

    Article  Google Scholar 

  • García-Gómez, A. (2022). Learning through WhatsApp: Students’ beliefs, L2 pragmatic development and interpersonal relationships. Computer Assisted Language Learning, 35(5–6), 1310–1328. https://doi.org/10.1080/09588221.2020.1799822

    Article  Google Scholar 

  • Geiger, J. F., & Litwiller, R. M. (2005). Spatial working memory and gender differences in science. Journal of Instructional Psychology, 32(1), 49–57.

    Google Scholar 

  • Gonenc, I. M., Dikmen, H. A., & Golbasi, Z. (2021). The effect of WhatsApp-based and conventional education methods on sexual myths and sexual health knowledge: A comparative intervention study in midwifery students. International Journal of Sexual Health, 33(3), 326–341. https://doi.org/10.1080/19317611.2021.1913688

    Article  Google Scholar 

  • Gooniband Shooshtari, Z., Jalilifar, A., & Khazaie, S. (2013). Mobile, L2 vocabulary learning, and fighting illiteracy: A case study of Iranian semi-illiterates beyond transition level. Applied Research on English Language, 2(2), 65–79.

    Google Scholar 

  • Grant, L., & Rong, X. L. (1999). Gender, immigrant generation, ethnicity, and schooling progress of youth. Journal of Research and Development in Education, 33, 15–26.

    Google Scholar 

  • Greene, J., Caracelli, V., & Graham, W. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11, 255–274.

    Article  Google Scholar 

  • Guler, C. (2016). Use of WhatsApp in higher education: What’s up with assessing peers anonymously? Journal of Educational Computing Research, 55(2), 272–289. https://doi.org/10.1177/0735633116667359

    Article  Google Scholar 

  • Gurian, M., & Ballew, A. (2003). The boys and girls learn differently. JosseyBass.

    Google Scholar 

  • https://www.statista.com/topics/2018/whatsapp/#topicOverview

  • Ismail, X. (2017). The effectiveness of social media network telegram in teaching English language pronunciation to Iranian EFL learners. Cogent Education, 4(1), 1347081. https://doi.org/10.1080/2331186X.2017.1347081

    Article  Google Scholar 

  • Jabbari, N., & Eslami, Z. R. (2019). Second language learning in the context of multiplayer online games: A scoping review. ReCALL, 31(01), 92–113. https://doi.org/10.1017/S0958344018000058

    Article  Google Scholar 

  • Jensen, S. H. (2016). Gaming as an extramural English language learning resource among young children in Denmark. CALICO Journal, 34(1), 1–19.

    Article  Google Scholar 

  • Koolstra, K. M., & Beentjes, J. W. J. (1999). Children’s vocabulary acquisition in a foreign language through watching subtitled television programs at home. Education Technology Research and Development, 47(1), 51–60.

    Article  Google Scholar 

  • Kuppens, A. H. (2010). Incidental foreign language acquisition from media exposure. Learning, Media and Technology, 35(1), 65–85.

    Article  Google Scholar 

  • Lai, C., Zhu, W., & Gong, G. (2015). Understanding the quality of out-of-class English learning. TESOL Quarterly, 49(2), 278–308.

    Article  Google Scholar 

  • Lefever, S. (2010). English skills of young learners in Iceland: “I started talking English when I was 4 years old. It just bang… just fall into me.”. Presented at the Menntakvika Conference, Reijkjavik, Iceland.

  • Lindgren, E., & Muñoz, C. (2013). The influence of exposure, parents, and linguistic distance on young European learners’ foreign language comprehension. International Journal of Multilingualism, 10(1), 105–129.

    Article  Google Scholar 

  • Lowe, A. P., Mayfield, W. J., & Reynolds, R. C. (2003). Gender differences in memory test performance among children and adolescents. Journal of Archives of Clinical Neuropsychology, 18(8), 865–878. https://doi.org/10.1093/arclin/18.8.865

    Article  Google Scholar 

  • Manca, S. (2020). Snapping, pinning, liking or texting: Investigating social media in higher education beyond Facebook. The Internet and Higher Education, 44, 100707. https://doi.org/10.1016/j.iheduc.2019.100707

    Article  Google Scholar 

  • Matsuoka, W., & Hirsh, D. (2010). Vocabulary learning through reading: Does an ELT course book provide good opportunities? Reading in a Foreign Language, 22(1), 56–70.

    Google Scholar 

  • Miramontes, O., Nadeau, A., & Commins, N. (1997). Restructuring schools for linguistic diversity: Linking decision making to effective programs. Teachers College Press.

    Google Scholar 

  • Motteram, G., Dawson, S., & Al-Masri, N. (2020). WhatsApp supported language teacher development: A case study in the Zataari refugee camp. Education and Information Technologies, 25, 5731–5751. https://doi.org/10.1007/s10639-020-10233-0

    Article  Google Scholar 

  • Nation, I. S. P. (2013). Learning vocabulary in another language. Cambridge University Press.

    Book  Google Scholar 

  • Olsson, E. (2012). Everything I read on the Internet is in English-On the impact of extramural English on Swedish 16-year-old pupils’ writing proficiency. Accessed from Gothenburg University Library https://gupea.ub.gu.se/handle/2077/30417

  • Olsson, E. (2016). On the impact of extramural English and CLIL on productive vocabulary. Acta Universitatis Gothoburgensis.

  • Peters, E., & Webb, S. (2018). Incidental vocabulary acquisition through viewing L2 television and factors that affect learning. Studies in Second Language Acquisition, 40, 551–577.

    Article  Google Scholar 

  • Peterson, M. (2010). Massively multiplayer online role-playing games as arenas for second language learning. Computer Assisted Language Learning, 23(5), 429–439.

    Article  Google Scholar 

  • Peterson, M. (2013). Computer games and language learning. Palgrave Macmillan. https://doi.org/10.1057/9781137005175

    Book  Google Scholar 

  • Ryu, D. (2013). Play to learn, learn to play: Language learning through gaming culture. ReCALL, 25(02), 286–301.

    Article  Google Scholar 

  • Salamat, A., & Pourgharib, B. (2013). The effect of using mobile on EFL students speaking. International Research Journal of Applied and Basic Sciences, 4(11), 3526–3530.

    Google Scholar 

  • Saritepeci, M., Duran, A., & Ermiş. (2019). A new trend in preparing for foreign language exam (YDS) in Turkey: Case of WhatsApp in mobile learning. Education and Information Technologies, 24, 2677–2699. https://doi.org/10.1007/s10639-019-09893-4

    Article  Google Scholar 

  • Schmitt, D., & Schmitt, N. (2020). Vocabulary in language teaching (2nd ed.). Cambridge University Press.

    Book  Google Scholar 

  • Smemoe, B. W., Cundick, D. K., Evans, N., Henrichsen, L., & Dewey, D. (2012). The relationship between reported out-of-class English use and proficiency gains in English. Applied Language Learning, 22, 21–47.

    Google Scholar 

  • Stickler, U., & Hampel, R. (2010). CyberDeutsch: Language production and user preferences in a Moodle virtual learning environment. CALICO Journal, 28(1), 49–73.

    Article  Google Scholar 

  • Sundqvist, P. (2019). Commercial-off-the-shelf games in the digital wild and L2 learner vocabulary. Language Learning & Technology, 23(1), 87–113.

    Google Scholar 

  • Sundqvist, P., & Wikström, P. (2015). Out-of-school digital gameplay and in-school L2 English vocabulary outcomes. System, 51, 65–76.

    Article  Google Scholar 

  • Sundqvist, P. (2009). Extramural English matters: Out-of-school English and its impact on Swedish Ninth graders' oral proficiency and vocabulary.

  • Sylvén, L., & Sundqvist, P. (2012). Gaming as extramural English L2 learning and L2 proficiency among young learners. ReCALL, 24(3), 302–321. https://doi.org/10.1017/S095834401200016X

    Article  Google Scholar 

  • Taki, S., & Khazaei, S. (2011). Learning vocabulary via mobile phone: Persian EFL learners in focus. Journal of Language Teaching and Research, 2(6), 1252–1258. https://doi.org/10.4304/jltr.2.6.1252-1258

    Article  Google Scholar 

  • Thornton, P., & Houser, C. (2005). Using mobile phones in English education in Japan. Journal of Computer Assisted Learning, 21, 217–228. https://doi.org/10.1111/j.1365-2729.2005.00129

    Article  Google Scholar 

  • Turgut, Y., & İrgin, P. (2009). Young learners’ language learning via computer games. Procedia - Social and Behavioral Sciences, 1(1), 760–764.

    Article  Google Scholar 

  • Wong, K. C., Lam, Y. R., & Ho, L. M. (2002). The effect of schooling on gender differences. British Educational Research Journal, 28, 827–843.

    Article  Google Scholar 

  • Zeidan, L. (2020). The relationship between extramural English activities and performance on vocabulary tests among Swedish upper secondary school learners. Unpublished BA Thesis, Karlstad University, Sweden. https://www.researchgate.net/publication/341131691

Download references

Acknowledgements

No applicable.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for profit sectors.

Author information

Authors and Affiliations

Authors

Contributions

SE: Designed and conducted the procedures and reviewed the final draft, and made necessary revisions. ZA: Collected the data and wrote the first draft. NG: Made necessary revisions.

Corresponding author

Correspondence to Saman Ebadi.

Ethics declarations

Ethics approval and consent to participate

The participants written consents to participate were obtained. Consent for publication: The consent for publication was obtained.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

figure a
 

Game

Island

Mouth

Movie

Song

Yard

1. Land with water all around it

      

2. Part of your body used for eating and talking

      

3. Piece of music

      
 

Boy

Rent

Report

Size

Station

Thing

4. How big or small something is

      

5. Place buses and trains to

      

6. Young man

      
 

Ear

Gold

Lake

Letter

Office

People

7. Information sent to people

      

8. Men and women

      

9. Place for working

      
 

Fellow

Hat

Ice

Joke

Light

System

10. Funny story

      

11. Man or boy

      

12. Something worn on your head

      
 

Date

Forest

Mistake

News

Record

Shop

13. Latest information

      

14. Place with many trees

      

15. Something that is not right

      
 

Bar

Conversation

Neighbor

Rain

Rubbish

Shirt

16. Person who lives nearby

      

17. Things that are thrown away

      

18. Type of clothing

      
 

Continue

COOK

Phone

Pull

Sail

Share

19. Hold and move something toward yourself

      

20. Keep happening

      

21. Use together with others

      
 

Enter

Finish

Happen

Own

Sing

Worry

22. End

      

23. Go inside

      

24. Have something that is yours

      
 

Arrive

Collect

Consider

Glance

Need

Pack

25. Look quickly at something

      

26. Reach the place you are going

      

27. Think about something

      
 

Affordable

Beautiful

Boring

Dry

Rough

Tall

28. Higher than normal

      

29. Not flat

      

30. Not interesting

      
 

Closed

Dirty

Empty

Musical

Orange

Sad

31. Having nothing

      

32. Not clean

      

33. Unhappy

      

Appendix 2

  1. 1.

    What do you think about extramural activities through various social networks?

  2. 2.

    List and explain the potential advantages or disadvantages of extramural activities through social media.

  3. 3.

    Did doing extramural activities through social media help you learn better? Please explain.

  4. 4.

    Can you cite examples of extramural activities through social media that have contributed to your better learning?

  5. 5.

    What is the difference between extramural activities through social media that you experienced during the course and traditional and previous activities? Please explain.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ebadi, S., Amini, Z. & Gheisari, N. On the relationship between mobile-based extramural activities and vocabulary development of EFL learners: a mixed-method study. Smart Learn. Environ. 10, 33 (2023). https://doi.org/10.1186/s40561-023-00252-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40561-023-00252-y

Keywords