Impact of online self-assessing metacognitive strategies accompanied with written languaging on cultivating Iranian IELTS candidates’ listening comprehension

Abstract

The use of self-assessing metacognitive strategies has been shown to enhance English as a foreign language (EFL) learners’ achievement in face-to-face classes. However, its impact on EFL learners’ achievement in online classes, which have become more prevalent due to the development of new technologies has remained largely unexplored. This study aimed to fill this gap by investigating the effects of online self-assessing metacognitive strategies (OSMLS) accompanied with written languaging on IELTS candidates’ listening comprehension (LC) in Iran. A total of 67 IELTS candidates were selected through a convenience sampling method and were homogenized using an IELTS listening test. The participants (n = 44) whose scores were around the mean score were randomly assigned to either a CG (n = 22) or an EC (n = 22). EG received online OSMLS training while the CG did not. Both groups took a pre-test, a post-test, and a delayed post-test of LC. A one-way ANCOVA was used to analyze the collected data. Results indicated that the experimental group outperformed CG on both the post-test and the delayed post-test of LC. The findings suggest that online OSMLS can improve EFL learners’ LC skills and have implications for EFL instruction in online settings.

Introduction

Contemporary social technologies have significantly influenced various aspects of our lives, including education (Azizi, 2022). In this digital age, the integration of technology in education has become increasingly popular and is playing a vital role in transforming traditional teaching and learning approaches (Hawkridge, 2022). According to Hamid et al. (2015), these technologies are engaging, fun to use, and free of charge. Moreover, when utilized in the educational sphere, they are reported to offer ease of use, great flexibility, high functionality, and free access (Jopp & Cohen, 2020). Outstanding changes in second language (L2) education have been initiated by the adoption of such technologies. Their ongoing developments have made L2 teachers and students opt for online education instead of traditional classrooms.

Listening comprehension (LC) is an essential aspect of L2 acquisition, and L2 learners often encounter challenges in comprehending spoken language (Vandergrift, 2007). In other words, L2 learners may have a good command of grammar and lexis in English, yet they may have difficulties understanding movies, TV shows, and news programs (Nguyen & Abbott, 2016). As underscored by In’nami et al. (2023), L2 learners face difficulties in understanding rapid speech, unfamiliar accents, and complex linguistic structures. These obstacles can hinder effective communication and language acquisition. Hence, addressing the challenges faced by L2 learners in LC is of paramount importance. Unfortunately, due to various reasons, including limited time and curriculum constraints, LC tends to receive less attention in L2 instruction (Aryadoust & Luo, 2023). This neglect can have significant repercussions for L2 learners. Without adequate focus and practice on listening skills, L2 learners may struggle to understand real-life conversations, lectures, and even academic materials. Consequently, their overall language proficiency may be hindered. To address the issue of L2 and enhance learning outcomes, the integration of OSMLS accompanied with WL has emerged as a potential solution.

Online self-assessing metacognitive strategies refer to metacognitive processes involving planning, monitoring, and evaluation that learners engage in while assessing their own learning progress in an online environment (Goh, 2008, 2010). Written languaging, on the other hand, involves the use of written language to facilitate learners’ comprehension of tasks and concepts. By combining these two approaches, learners can enhance their listening efficiency and metacognitive awareness (Suzuki, 2012). Nevertheless, it is important to explicitly define and explain these strategies to provide a clear understanding of their potential benefits. By integrating metacognitive strategies and written languaging, L2 learners can become more aware of their listening difficulties, plan their learning activities effectively, monitor their comprehension in real-time, and evaluate their progress more accurately (Vandergrift, 2004). As Goh and Taib (2006) note, metacognition can act as a guiding process to learning, and L2 learners are guided to employ strategies for planning, monitoring, and evaluating their language learning and language use. These processes include planning, monitoring, and evaluating (Goh, 2014; Vandergrift, 2004). This integrated approach has the potential to improve L2 learners’ listening skills and overall language proficiency.

Given the advantages of online education and the powerful theoretical foundations of OSMLS and WL, it is necessary to study if online OSMLS accompanied with WL lead to any significant improvement in L2 learners’ language competence. However, a quick glance at the literature demonstrates that, to date, no study has explored the effects of OSMLS accompanied with WL on improving LC among Iranian EFL learners. In response to this gap, this research aimed to scrutinize the effects of these strategies on Iranian EFL candidates’ LC. The results of this study may provide evidence in favor of using online OSMLS accompanied with WL in online classes. Furthermore, this study can be of importance to researchers, as it is the first to examine the effects of online OSMLS accompanied with WL on fostering EFL learners’ LC. The findings of this research may also be of great help for materials developers, as they can be used to rethink listening activities in textbooks and devise them in accordance with online OSMLS accompanied with WL.

Theoretical foundations

LC is the most prevalently used language skill in EFL/ESL classrooms (Vandergrift, 2007), as it enables L2 learners to access authentic and meaningful input, interact with native and non-native speakers, and develop their communicative competence (Rost & Wilson, 2013). Similarly, Goh and Hu (2014) emphasize the importance of LC to language learning, as it forms the basis for other language skills, such as speaking, reading, and writing. In appreciation of LC’s vital role, it has been called the Cinderella skill (Goh, 1997), because it has been neglected and undervalued in both teaching and research. Despite its importance, not enough SLA research has been done on LC; Buck (2001) states, “we still do not fully understand what the important sub-skills are; nor are we sure what information educators need to teach listening better” (p. 97). Therefore, it is essential to investigate the factors that affect LC and the strategies that enhance it in different contexts and settings.

Recently, there has been some recognition and significance attached to the systematic teaching of listening (Aryadoust & Luo, 2023). This shift from product- to process-based approaches has been intended to focus on the skills required for successful LC (In’nami et al., 2023). One of the process-based approaches that has gained popularity is metacognitive instruction in listening, which is defined as “pedagogical procedures that enable learners to increase their awareness about the listening process and at the same time develop effective skills for self-appraising and self-regulation listening and the progress of their overall listening development” (Goh, 2010, p. 180–1). In simple terms, metacognitive instruction in listening aims to help L2 learners to understand how they listen and how they can improve their listening by using appropriate strategies and techniques (Goh, 2017; Kobayashi, 2018).

Metacognitive instruction in listening consists of three main components: metacognitive knowledge, metacognitive monitoring, and metacognitive control (Vandergrift & Goh, 2012). Metacognitive knowledge refers to the information that L2 learners have about themselves as listeners, the listening tasks and situations, and the strategies that can be used for listening (Bozorgian & Shamsi, 2023). Metacognitive monitoring refers to the process of checking one’s own listening comprehension and identifying difficulties or gaps (Goh, 2018; Vandergrift & Goh, 2012). Metacognitive control refers to the process of selecting and applying suitable strategies to overcome difficulties or enhance comprehension (Vandergrift & Goh, 2012). Metacognitive instruction can improve LC in several ways. First, it can increase L2 learners’ motivation and confidence by helping them to set realistic goals, plan their learning, and evaluate their progress (Bozorgian & Shamsi, 2023). Second, it can enhance learners’ comprehension by helping them to activate their prior knowledge, make predictions and inferences, and use contextual clues (Bozorgian & Shamsi, 2023). Third, it can foster learners’ autonomy by helping them to become more aware of their strengths and weaknesses, choose the best strategies for different tasks and situations, and regulate their own learning processes (Bozorgian & Shamsi, 2023).

However, metacognitive instruction in listening also faces some challenges and difficulties. One of the challenges is the time-consuming nature of this approach, which requires L2 learners and teachers to spend more time on preparing, implementing, and reflecting on the listening activities than conventional methods (Bozorgian & Fakhri Alamdari, 2018; Goh, 2018). This may cause some L2 learners or teachers to lose interest or patience in the process. Another challenge is the lack of resources for this approach, which requires L2 learners and teachers to have access to suitable materials, tools, and guidance for metacognitive instruction in listening (Bozorgian et al., 2022). This may limit the availability or quality of this approach in some contexts or settings. Therefore, it is important to find alternative ways to incorporate metacognitive aspects into LC. One of the often-cited activities for metacognitive instruction in listening is the metacognitive pedagogical sequence (Vandergrift, 2004), which is based on four essential processes of successful LC: “planning for the activity; motivating comprehension; solving comprehension problems; and evaluating the approach and outcomes” (Goh & Hu, 2014, p. 84–5). The metacognitive pedagogical sequence consists of five phases: pre-listening, first verification, second verification, reflection, and expansion (Vandergrift & Goh, 2012). In each phase, L2 learners are guided by a series of questions or prompts that help them to activate their metacognitive knowledge, monitoring, and control.

The metacognitive pedagogical sequence has been proven to be effective in improving LC and metacognition in various studies (Chou, 2017; Fahim et al., 2014; Maftoon & Fakhri Alamdari, 2020). For example, Chou (2017) found that Taiwanese EFL learners who received metacognitive instruction in listening outperformed those who received conventional instruction in both LC tests and metacognitive awareness questionnaires. Fahim et al. (2014) found that Iranian EFL learners who received metacognitive instruction in listening showed significant improvement in their LC scores and strategy use compared to those who received traditional instruction. Maftoon and Fakhri Alamdari (2020) found that Iranian EFL learners who received metacognitive instruction in listening demonstrated higher levels of motivation, self-efficacy, and autonomy than those who received regular instruction.

However, as mentioned earlier, the metacognitive pedagogical sequence is also time-consuming and labor-intensive for both L2 learners and teachers (Bozorgian, 2014). Therefore, Goh and Hu (2014) suggested involving learners in responding to some written checklists before and after they do listening tasks. This would save time and help guide the learners towards autonomous use of metacognitive listening strategies in a shorter amount of time. Goh and Hu (2014) argued that the checklists can help create the opportunity for learners to self-assess their task performance in pre-listening and post-listening phases. Allocating some time for students to write down their answers to the metacognitive-related questions, and then compare these answers with those of their peers, can help build the knowledge and awareness necessary for successful listening processes and strategies (Cross, 2011). This use of written language to construct knowledge and awareness has been termed WL by Suzuki (2012).

WL is an extension of languaging, defined as “the process of making meaning and shaping knowledge and experience through language” (Suzuki, 2012; p. 98). This is similar to the concept of self-explaining (Cross, 2010), which means that individuals often talk to themselves when faced with a difficult task, helping them to understand the task and formulate a solution. Swain (2006) also argued that “languaging about language is one of the ways we learn a second language to an advanced level” (p. 96) and that “it is part of what constitutes learning” (p. 98). Several studies have found that both collaborative oral activities (Vandergrift et al., 2010) and WL (e.g., Moradian et al., 2017; Suzuki, 2009, 2012) can be beneficial to language development.

WL can be especially useful for LC, as it can help learners to reflect on their listening experiences and outcomes. Reflection is the process of thinking critically and analytically about one’s own learning processes and products. It can help learners to identify their strengths and weaknesses, evaluate their strategies and results, and plan for future improvement. Reflection can also help learners to connect their listening experiences with their prior knowledge, personal feelings, or social contexts. Reflection can enhance learners’ comprehension and language development by helping them to notice, internalize, and apply linguistic features of the target language.

Swain’s concept of languaging is closely related to the reflection process and its function in language learning. Swain (2006) proposed that languaging has three main functions: cognitive, social, and metalinguistic. The cognitive function refers to the role of languaging in enhancing thinking and problem-solving skills. The social function refers to the role of languaging in facilitating interaction and collaboration with others. The metalinguistic function refers to the role of languaging in developing awareness and knowledge about language. These functions can be applied to the reflection process in LC, as learners can use WL to think about their listening problems and solutions, share their listening experiences and opinions with others, and learn about the linguistic features and rules of the target language.

Using a self-assessment checklist is based on the theoretical tenets of applying self-assessment in other domains of Second Language Acquisition (SLA). As noted by Azarnoosh (2013), “the importance of assessment as an integral part of the teaching–learning cycle is apparent to many educationalists” (p. 2). They recognize that assessment can complement the learning process by providing evaluative feedback to students, who can then reflect upon and test out their own performance. Similarly, Topping (2009) highlighted that students should be afforded opportunities to examine their own initial hypotheses about language learning and language itself without fear of constant critical evaluation by the teacher; having opportunities to experiment with language in the classroom without being formally scored can be conducive to learning.

Nevertheless, in spite of the recommendations made by Goh and Hu (2014), the role of self-assessing metacognitive listening strategies has received scant attention. Moreover, since Goh and Hu (2014) underscored that we need to find ways to get our students to reflect upon metacognitive experience so as to benefit more from them, the learners are asked to write down their reflections in response to the checklist items. This use of language in constructing further knowledge and awareness has been referred to as ‘written languaging’ (Suzuki, 2012). Swain (2006) maintained that reflection on output productions is essential for learning, since learners try to amend the lacks in their interlanguage systems, and then, remove these gaps. Swain called this function of language as languaging. It is “the process of making meaning and shaping knowledge and experience through language” (Swain, 2006, p. 98). This is the point where the mediation of output grows. Languaging is also “a form of verbalization used to mediate the solution(s) to complex problems and tasks” (Swain et al., 2009, p. 5). WL has been indicated to be effective in raising learners’ awareness of their linguistic errors and stimulating them to develop a more profound understanding of them, so as to elicit some rules about them and get them fixed (Suzuki, 2012).

Online instruction

Over the last decades, digital technology has rapidly spread around the world, providing multiple avenues for L2 learners to learn and practice English in settings different from face-to-face classes (Imamyartha et al., 2021; Naghdipour, 2017). L2 learners can utilize general search-engine information-seeking, watch films and videos, listen to audio files, participate in online forums, and use a variety of apps (Lamb & Arisandy, 2020; Lee, 2022). According to Sockett (2014), digital technology has generated a unique incentive to learn English, as well as the means to achieve noteworthy results. The term ‘online language learning’ was coined to refer to various learning settings, such as a fully online course, a blended or hybrid course, and a web-facilitated class (Blake, 2011).

Online education offers some outstanding advantages for L2 learners and teachers (Azizi, 2022). One of the advantages is flexibility, which means that students and teachers can conduct classes without the limitation of time and place (Parmaxi, 2023). This allows L2 learners to access learning materials and activities at their own pace and convenience, and to adjust their learning schedules according to their personal needs and preferences. Teachers can also design and deliver courses that suit the learners’ levels, interests, and goals, and provide more individualized feedback and support (Tarrayo et al., 2023). Flexibility can enhance learners’ motivation, autonomy, and satisfaction in online language learning (Davis et al., 2019). Another advantage of online education is cost-effectiveness, which means that students and teachers can save time and money as they are not required to commute to university campuses or pay for transportation, accommodation, or other expenses (Latchem & Jung, 2009). Online education can also reduce the costs of learning materials and resources, as learners can access them for free or at a low price through digital platforms. Moreover, online education can increase the opportunities for learners to access high-quality courses and instructors from different countries and regions, which can enrich their linguistic and cultural exposure and awareness. Cost-effectiveness can enhance learners’ accessibility, diversity, and quality in online language learning (Limperos et al., 2015). An additional advantage of online education is being interesting for teachers and students, which means that working with and learning through modern social technologies is captivating for both teachers and students (Alghamdi, 2021). As noted by Azizi et al. (2022), online education can provide learners with more variety and choice in learning materials and activities, such as multimedia, games, simulations, quizzes, podcasts, blogs, etc. Online education can also foster learners’ social interaction and engagement with their peers and instructors through synchronous or asynchronous tools, such as chats, forums, video calls, emails, etc. (Wilson & Stacey, 2004). Online education can also stimulate learners’ creativity and expression by allowing them to create or share digital content, such as videos, podcasts, blogs, etc. According to Nguyen et al. (2022), Being interesting for teachers and students can enhance their enjoyment, involvement, and achievement in online language learning (Panda & Mishra, 2007; Wasilik & Bolliger, 2009).

However, online education also poses some challenges and difficulties for L2 learners and teachers. One of the challenges is the lack of face-to-face interaction, which means that learners may feel isolated or disconnected from their peers or instructors in the online setting (Wasilik & Bolliger, 2009). This may affect their emotional well-being, social presence, or sense of belonging in the online community. Another challenge is the need for self-regulation, which means that learners need to manage their own learning processes effectively in the online setting (Azizi et al., 2022). This includes setting goals, planning strategies, monitoring progress, evaluating outcomes, and seeking help when needed. Self-regulation is crucial for online language learning success (Kormos & Csizér 2014).

Given the points above, it is essential to explore how OSMLS accompanied with WL can help L2 learners to overcome these challenges and benefit from the advantages of online education. OSMLS and WL can help L2 learners to improve their listening comprehension, as well as their motivation, autonomy, and satisfaction in online language learning (Swain, 2006; Vandergrift & Goh 2012). Thus, this study is an early attempt to disclose the potential of OSMLS accompanied with WL in the development of Iranian EFL learners’ LC.

Related studies in the literature

The effects of self-assessing metacognitive listening strategies and WL on L2 learning have received noticeable attention in face-to-face classes over the last years. However, the existing literature has some limitations and gaps that need to be addressed. Swain et al. (2009) conducted an early attempt to investigate the impact of oral languaging on the learning of French grammatical structures. Their findings showed that with the help of oral languaging, the participants could comprehend the grammatical structures. However, they did not examine the effects of WL, which might have different benefits for learners. Additionally, Suzuki (2012) studied the influence of WL on improving accuracy with immediate revision tasks. It was evident that the learners used more WL episodes on grammatical forms than on lexis, and that they saw significant progress in their revisions. However, they did not measure the long-term effects of written languaging on learners’ accuracy. Ishikawa (2013) additionally examined the role of a type of written languaging known as meta-notes on enhancing EFL learners’ writing skills on a translation task. Analyzing the meta-notes of individual cases, it was discovered that those who took more meta-notes saw the most improvement in their writing skills. However, they did not explore the effects of meta-notes on other skills, such as LC.

In the Iranian EFL context, Moradian et al. (2017) examined the impact of direct written corrective feedback accompanied by WL compared to direct written corrective feedback on Iranian EFL writing skills. They found that direct written corrective feedback accompanied with WL was more effective than direct written corrective feedback in promoting the participants’ writing skills. In a more recent study, Moradian et al. (2020) examined the impact of written corrective feedback followed by languaging on cultivating Iranian EFL learners’ writing in terms of grammatical accuracy. Their findings evidenced that, due to the positive effects of the intervention, the participants’ writing skills with respect to grammatical accuracy significantly improved. However, both studies focused only on writing skills and did not investigate the effects of OSMLS and WL on LC.

The above-reviewed studies demonstrate that they have only focused on the effects of self-assessing metacognitive listening strategies and WL or oral languaging in traditional classes. Given the fact that online education, as an alternative to traditional classes, has received noticeable attention in recent years, it is essential to explore if OSMLS accompanied with WL leads to improving EFL learners’ LC. Online education creates a new context that requires learners to use different strategies and skills to cope with the challenges and opportunities of digital platforms. Therefore, it is important to examine how OSMLS and WL can help learners to enhance their LC in this setting. Thus, this study is an early attempt to disclose the potential of OSMLS accompanied with WL in the development of Iranian EFL learners’ LC. Considering these objectives, the following research questions were investigated:

• RQ1: Does online self-assessing metacognitive listening strategies accompanied with written languaging improve Iranian IELTS candidates’ listening comprehension?

• RQ2: Does online self-assessing metacognitive listening strategies accompanied with written languaging lead to the long-term improvement of Iranian IELTS candidates’ listening comprehension?

Method

Design of the study

The research design adopted for this study was a true-experimental design, which is a statistical method to establish a cause-and-effect relationship between variables (Riazi, 2016). A true-experimental design has three elements: a CG, an independent variable, and random assignment (Riazi, 2016). This design allowed the authors to control and manipulate the independent variable (i.e., OSMLS accompanied with WL) and observe its effects on the dependent variable (e.g., LC) while minimizing confounding factors.

Setting and participants

The present study was conducted in the setting of Milad Language Institute (MLI), one of the established IELTS training centers with different branches in Tehran, Iran. The researchers used a convenience sampling method to select the participants, as they had limited access to the population of IELTS candidates and had to rely on the availability and willingness of the learners at MLI. This methodology was appropriate for the research objectives and context, as the authors aimed to investigate the effects of OSMLS accompanied with WL on LC among intermediate to upper-intermediate IELTS candidates in a naturalistic setting. The auhtors administered an IELTS test to 67 IELTS candidates, including 35 males and 32 females, whose ages ranged from 16 to 46 (M = 31.14, SD = 6.25). The participants also reported their educational backgrounds, which varied from high school diploma to master’s degree, with most of them having a bachelor’s degree. The researchers selected 44 participants whose scores fell around the mean (6.5) and assigned them randomly to two groups: a CG and an EG (n = 22). The participants were studying English to take the IELTS exam, and their use of English was mainly restricted to their classes, as they did not have the opportunity to converse in English outside of MLI. This limited language exposure might have hindered their language acquisition, especially their listening and speaking skills, which require more interaction and practice. The authors recruited an IELTS trainer with over 10 years of teaching experience to run the interventions for the two groups. The trainer was familiar with the study’s objectives and procedures, and had received training from the researchers on how to implement the interventions. The trainer’s teaching experience was crucial for ensuring the quality and consistency of the interventions, as well as for establishing rapport and trust with the participants.

The researchers accessed the participants by contacting the general manager of MLI via email and requesting a meeting with him. In the meeting, which took place in his office, they explained the objectives of the study and asked for his cooperation to run the study in the MLI setting. They also assured him that the study would not interfere with the regular classes and activities of MLI, and that it would benefit both the learners and the teachers by providing them with useful insights into listening strategies and written language production. The general manager agreed and directed them to the IELTS classes that met their criteria. With the permission of the instructors, who were also informed about the study’s objectives and procedures, the authors visited each class and introduced themselves to the learners. They provided information about the study and invited them to participate voluntarily. They also explained that participation in the study would entail taking an IELTS test, completing listening tasks with or without SECs, producing written responses to questions, receiving feedback from an experienced trainer, and taking another IELTS test at the end of the study. They emphasized that participation in the study would not affect their grades or evaluation at MLI, and that their performances would be kept confidential and used only for research purposes. They also promised to share the results of the study with them at the end of the study. Those who agreed to participate signed written consent forms and gave their contact details to the researchers. The researchers thanked them for their cooperation and arranged a time and place for administering the pre-test.

Instruments

The data for this study were collected using some instruments. The first instrument was the IELTS listening test, which consisted of three samples of IELTS listening exams. These samples were selected from the official IELTS practice materials, which are developed by the IELTS test partners and have high reliability and validity indices. The participants took the IELTS listening test as part of their regular course assessment, which ensured the ecological validity of the study. The IELTS listening test had four sections, each with 10 questions, covering a variety of topics and task types. The first section was a conversation between two people in a social context, such as booking a hotel or renting a car. The task type was form completion, where the test-takers had to fill in the gaps in a form with information from the conversation. The second section was a monologue on a general topic, such as a radio broadcast or a tour guide speech. The task type was matching, where the test-takers had to match a list of options to another list of categories or features. The third section was a conversation between up to four people in an academic or training context, such as a group discussion or a seminar. The task type was multiple choice, where the test-takers had to choose the correct answer from four options for each question. The fourth section was a lecture or a talk on an academic subject. The task type was plan/map/diagram labeling, where the test-takers had to label parts of a plan, map or diagram with information from the lecture or talk. The recordings featured different accents of English, such as American, Australian, British, Canadian and New Zealand. The recordings were played only once and the test-takers had to answer the questions as they listened. They had 10 min at the end of the test to transfer their answers to an answer sheet. The total duration of the test was about 30 min.

The second instrument was a self-assessment checklist (SEC) that aimed to help EG evaluate their metacognitive awareness of listening strategies and produce written language. The SEC was designed and developed by the auhtors following these steps: First, they reviewed the main texts on metacognitive awareness of listening, such as those by Cross (2010, 2015), Goh (2010, 2014), Vandergrift and Goh (2012), and Vandergrift et al. (2006). Second, they identified the common themes that emerged from these texts regarding metacognitive awareness of listening strategies. Third, they classified these themes into two categories: one for the pre-listening phase and one for the post-listening phase. Fourth, they formulated questions based on these themes to prompt the participants to reflect on their listening process and generate written language. Fifth, they invited two university professors in Applied Linguistics at Tehran University to evaluate the SEC in terms of language and content, and revised it according to their feedback. Sixth, they hired an experienced translator to translate the SEC into Persian, and verified his credentials and qualifications. Seventh, they checked the translation for accuracy, clarity, and cultural appropriateness, and made necessary adjustments. Eighth, they asked three IELTS candidates to read and comment on the SEC in terms of readability, and confirmed that the content was easy to understand. The SEC consisted of two sections: one for the pre-listening phase and one for the post-listening phase. Each section contained questions that aimed to direct learners’ attention and awareness towards various aspects of metacognition such as task, person, and strategy use. The pre-listening section included questions about the topic, purpose, and difficulty of the listening task, as well as the learners’ prior knowledge, expectations, goals, and plans for listening. The post-listening section included questions about the outcome, challenges, and strategies of the listening task, as well as the learners’ self-evaluation, feedback, and improvement plans for listening. The SEC was designed to be simple and short, so as to avoid boredom and monotony (Goh, 2010). The participants were instructed to answer the questions in writing before and after each listening task.

The last instrument consisted of listening tasks that were selected from the Mindset IELTS Series, a set of books that prepare learners for the IELTS test. The selection criteria for these tasks were as follows: First, they had to match the proficiency level of the participants, which was intermediate to upper-intermediate. Second, they had to cover a variety of topics and genres, such as academic lectures, conversations, interviews, and radio programs. Third, they had to include different types of questions and formats, such as multiple choice, matching, table completion, and note-taking. The reason for choosing these tasks was to motivate the participants to engage with the tasks seriously and attentively, as they were relevant to their goal of taking the IELTS test. This also increased the ecological validity of the study, as the tasks reflected the real-world listening situations that the participants might encounter (Mackey & Gass, 2015). The tasks comprised 12 listening parts of the Mindset IELTS Series, which were divided into 15 sessions. Each session lasted for about 45 min and included one or two listening parts. The participants completed the tasks individually and then checked their answers with the answer key provided by the researchers. The tasks aligned with the study’s objectives of investigating the effects of metacognitive awareness of listening strategies on written language production, as they required the participants to use various listening strategies and produce written responses to the questions.

Data collection procedures

To conduct this research, the researchers followed several steps. First, they designed and developed the SEC and hired a translation expert to render it into Persian. Second, they homogenized the participants using a sample of IELTS listening test, and selected 44 participants with scores close to the mean. They randomly assigned them into a CG (n = 22) and an EC (n = 22). Third, they organized two mini-workshops for the instructor to introduce her to the principles and procedure of SAMS for teaching listening comprehension in online classes. The mini-workshops aimed to train the instructor on how to use SAMS to guide the learners to plan, monitor, and evaluate their listening performance. The instructor learned how to provide online feedback and scaffolding to the learners using SAMS. The instructor also practiced using SAMS with some sample listening texts and tasks before implementing them in the actual classes. Fourth, the auhtors conducted a pre-test to measure the participants’ LC before the interventions. The pre-test consisted of a sample of IELTS listening test, which included four sections with 40 questions in total. The questions assessed the participants’ ability to understand main ideas, specific information, opinions, attitudes, and speakers’ purpose in different listening contexts. The pre-test score was used as a baseline to compare the participants’ LC improvement after the interventions.

Fifth, the interventions were offered which lasted for 15 one-hour sessions, held twice a week. EG received online instruction using SAMS and WL, while CG received face-to-face instruction using conventional methods. SAMS are a set of strategies that help EFL learners plan, monitor, and evaluate their listening performance. WL is the use of written language to express and reflect on one’s listening processes and outcomes. The EG used WhatsApp and Skyroom as online platforms for their instruction. WhatsApp is a mobile application that allows users to make voice and video calls, send text messages, and share files over the internet. Skyroom is a web-based program that enables users to create and join virtual classrooms, where they can interact with the instructor and other learners through audio, video, chat, and screen sharing. These platforms enhanced the learning process by facilitating communication, collaboration, and feedback among the EFL learners and the instructor. Each session for the EG followed this procedure: First, the instructor sent a copy of a self-assessment checklist to each EFL learner via WhatsApp. The checklist contained 10 items that asked the EFL learners to rate their use of different metacognitive listening strategies on a five-point Likert scale. The EFL learners completed the checklist individually in the pre-listening phase and sent it back to the instructor. Then, they paired up with classmates via WhatsApp and compared their answers. The instructor also joined the discussion and provided hints and comments on how to use the strategies effectively. Next, the EFL learners joined the Skyroom virtual classroom, where the instructor presented them with a listening task related to their IELTS preparation. The task included an audio file and some comprehension questions in their book. The EFL learners listened to the audio file and answered the questions individually. Then, they wrote a summary of what they listened to in the post-listening phase and sent it to the instructor via WhatsApp. They also paired up again via WhatsApp and compared their answers and summaries. The instructor gave them feedback and clarification on their performance. Finally, the instructor asked for the EFL learners’ self-assessment checklists again for evaluation at home. On the other hand, the CG received face-to-face instruction in a traditional classroom setting. They did not use any self-assessment checklists or written languaging. They only listened to an audio file and answered comprehension questions in their book. To make the instruction time equal to that of the EG, the instructor asked them additional comprehension questions about the audio file or asked them to repeat the audio file sentence by sentence. These are some of the most common techniques used by EFL teachers.

Fifth, after the interventions, the authors administered a post-test to measure the participants’ LC improvement. The post-test was similar to the pre-test in terms of structure and content, but it used different audio files and questions. The post-test score was compared with the pre-test score to determine the effects of SAMS and WL on LC development. Sixth, one month after the post-test, the researchers administered a delayed post-test to measure the participants’ LC retention. The delayed post-test was identical to the post-test in terms of structure and content, but it used different audio files and questions again. The delayed post-test score was compared with the post-test score to examine whether SAMS and WL had any long-term effects on LC maintenance.

Data analysis procedures

The researchers used SPSS, Version 22 to analyze the collected data. First, he calculated the basic descriptive statistics, such as the mean (M) and standard deviation (SD). Then, he used a one-way ANCOVA to determine if there was a statistically significant difference between the EG and the CG concerning the gains in LC at three different points in time.

Results

As noted above, the research questions explored whether OSMLS accompanied with WL led to improving Iranian IELTS candidates’ LC on the post-test and delayed post-test. To answer these research questions, the researchers used a one-way ANCOVA. The assumptions related to this statistical procedure, including linearity, normality, and homogeneity, were checked. Since there was no curvilinear relationship in the distribution of scores for each group on the scatterplot, it was concluded that the linearity assumption was met. Furthermore, the findings of the Kolmogorov–Smirnov test evidenced that the sig. values (0.41) were larger than the critical value (0.05), thus confirming that the normality assumption was met. Additionally, the results of Levene’s test of the equality of the variances showed that the sig. level (0.07) was greater than the alpha level (0.05), thus confirming that the homogeneity assumption was also met.

After ensuring that the necessary assumptions were not violated, the researchers used a one-way ANCOVA. The results of the descriptive analysis are presented in Table 1. As can be seen, M for the EG was 30.90 and SD was 5.07, while M for the CG was 12.72 and SD was 3.25.

Table 1 Descriptive statistic for comparing the scores of both groups on the post-test

As observed in Table 1, there was a noticeable difference between the mean scores on the post-test between the two groups. Thus, to see if this difference was statistically significant, the researchers considered the results of Tests of Between-Subjects Effects. The results are reported in Table 2.

Table 2 Results of tests of between-subjects effects for comparing the scores of both groups on the post-test

As seen in Table 2, the difference between the mean scores of the two groups was statistically significant (F (1, 43) = 739.52, p < 0.001, partial eta squared = 0.94). These results indicated that around 94% of the difference between the two groups in terms of the gains in LC on the post-test might be associated with the effects of the independent variable. However, the effects of the pre-test scores in the post-test scores were significant (F (1, 26) = 160.79, p < 0.00, partial eta squared = 0.79). This implied that around 79% of the difference on the post-test can be attributed to the difference on the pre-test. Afterwards, the results consulted the results of Estimated Marginal Means test to check the adjusting means on the intervention type for the two groups so as to remove the effects of the covariate on the post-test. The results are presented in Table 3.

Table 3 Results of estimated marginal means

After adjusting for the pre-test scores, as reported in Table 4, there was a significant difference between the two groups concerning the gains in the LC, F (1, 26) = 739.52, p = 0.00, partial eta squared = 0.94.

Table 4 Results of univariate tests

As given in Table 5, the researchers consulted the significance values and found that a statistically significant difference existed between the two groups concerning the gains in LC on the post-test.

Table 5 Results of pairwise comparisons of the two groups on the post-test

As seen in Table 5, the difference between the mean scores of the two groups was statistically significant (F (1, 43) = 739.52, p < 0.001, partial eta squared = 0.94). These results indicated that around 94% of the difference between the two groups in terms of the gains in LC on the post-test might be associated with the effects of the independent variable. However, the effects of the pre-test scores in the post-test scores were significant (F (1, 26) = 160.79, p < 0.00, partial eta squared = 0.79). This implied that around 79% of the difference on the post-test can be attributed to the difference on the pre-test. Afterwards, the results consulted the results of Estimated Marginal Means test to check the adjusting means on the intervention type for the two groups so as to remove the effects of the covariate on the post-test. The results are presented in Table 3.

As presented in Table 6, for the EG, M = 30.63 and SD = 4.93 and for the CG, M = 12.81 and SD = 3.00 were calculated on the delayed post-test, in turn.

Table 6 Results of descriptive statistic for comparing the scores of the two groups on delayed post-test

As observed in Table 6, there was a difference between the mean scores of the two groups. To see if this difference was statistically significant, the researchers ran Tests of Between-Subjects Effects. The results are reported in Table 7.

Table 7 Results of tests of between-subjects effects for comparing the post- and delayed post-test scores of the two groups

As presented in Table 7, there existed a statistically significant difference between the two groups regarding the gains in the LC on the post test, F (1, 43) = 28.07, p < 0.001, partial eta squared = 0.48. This means that around 48 percent of the differences can be linked with the effects of the interventions. However, the results indicated that the effects of the difference in the scores of the post-test on the scores of the delayed post-test was significant F (1, 43) = 278.81, p < 0.001, partial eta squared = 0.87. This implies that around 87 percent of the difference can be illuminated by the differences of the scores in the post-test and the scores in the delayed post-test. Afterward, the researchers considered the Estimated Marginal Means to remove the effects of the covariate on the post-test scores. The results are reported in Table 8.

Table 8 Results of estimated marginal means

As presented in Table 9, the researchers adjusted for the post-test and found that there was a statistical significance between the EG and the CG concerning the gains in LC on the delayed post-test, F (1, 43) = 28.77, p = 0.00, partial eta squared = 0.48.

Table 9 Results of univariate tests

Finally, as given in Table 10, considering the significance values, it could be concluded that there existed a statistically significant difference between the two groups regarding their performance on the delayed post-test.

Table 10 Results of pairwise comparisons

Discussion

The first research question examined whether OSMLS accompanied with WL resulted in enhanced listening comprehension (LC) for Iranian IELTS candidates. OSMLS are techniques that help learners plan, monitor, and evaluate their own listening processes, while WL is the use of language to reflect on linguistic issues (see Sect. “Theoretical foundations” for definitions). The findings showed that EG, who used OSMLS and WL, outperformed CG, who did not use these strategies, in terms of LC gains. The second research question investigated whether OSMLS and WL contributed to the long-term improvement of LC for the EG. The results demonstrated that the EG maintained their LC gains on the delayed post-test, while the CG did not. Therefore, it can be argued that the significant and lasting improvement in LC for the EG was due to the positive effects of OSMLS and WL.

The results agree with Vandergrift and Tafaghodtari (2010), who found that the EG who received instruction through a lesson plan resting upon prediction/planning, monitoring, evaluating, and problem-solving outperformed the CG in language competency. The findings align with Moradian and Kogani Baharvand (2017), who attributed the EG’s language competency improvement to their involvement in languaging and collaborative dialogues. The outcomes match the findings of Wang (2016), who demonstrated that self-assessing metacognitive listening strategies enhanced L2 learners’ competency. The results concur with Mahdavi and Miri (2019), who showed that a task-based approach improved participants’ language competency.

One possible explanation for the findings is that the use of the self-assessment checklist might have stimulated the EFL learners to externalize what they had internalized in terms of metacognitive strategies, such as planning, directing attention, problem-solving, mental translation, and evaluation. Along with Goh (2010), it may be argued that the participants might have used the self-assessment checklists to reflect on the quantity and quality of the strategies they employed. In this regard, the concrete WL and collaborative dialogue could have opened up opportunities for the participants to further their understanding of the metacognitive written strategies (Swain, 2006). Moreover, to offer another explanation for the findings, we can refer to Flavell’s (1979) model of metacognition. Along with this model, it may be argued that as the metacognitive experiences were fleeting and transitory, they might have contributed significantly to the participants’ learning when they were engaged in self-regulation, reflection, and problem-solving processes. These, in turn, might have led to more useful processes and more promising results (Goh & Hu, 2014).

Another reason for the findings may be attributed to the positive role of the self-assessment checklist items in guiding the written and collaborative reflections (Bozorgian et al., 2022). The items of the self-assessment checklist might have caught the participants’ attention to the various aspects of metacognition, including metacognitive knowledge (i.e., person, task, and strategy) and self-regulation strategies (e.g., planning, monitoring, and evaluation) (Bozorgian, 2014) This argument, supported by the findings, is consistent with Goh and Hu’s (2014) perspective that L2 teachers should steer students’ reflections, as broad reflections on listening activities are highly conducive to metacognitive development.

The superiority of EG may be attributed to the fact that, after generating written language, the participants had the chance to collaborate with their peers and collectively develop a deeper understanding of metacognitive strategies (Cross, 2010; Mahdavi & Miri, 2019; Moradian et al., 2020). This line of argument is grounded in the thinking of Vygotsky (1978), who asserts that knowledge is initially constructed in a social context through dialogic discussions, allowing individuals to gradually internalize and autonomously apply the co-constructed knowledge. Reflection and self-explanation are powerful tools that likely facilitated the learners’ internalization process (Swain, 2006). In other words, written languaging, as a form of written verbalized reflection and self-explanation, can be a potent tool for profound internalization (Moradian et al., 2017; Suzuki, 2012). Additionally, it can be argued that the opportunity for the EG to engage in discussions with the teacher regarding metacognitive strategies might have aided the learners in co-constructing more metacognitive knowledge and awareness, benefitting from the expertise of a knowledgeable other (Mahdavi & Miri, 2019).

The results could be attributed to the production of written language, which was followed by collaborative dialogues. This process likely assisted the learners in concretely expressing their metacognitive awareness of listening strategies through written responses, offering them additional opportunities for reflection and internalization. By revisiting their written responses multiple times, the learners may have experienced increased knowledge and awareness. Subsequently, engaging in dialogic reflections with their peers could have further contributed to their language comprehension growth. Suzuki (2012) highlights that the combination of written language and collaborative interaction is particularly conducive to learning, fostering a deeper level of understanding. Furthermore, in CG, the use of written language potentially allowed learners to articulate their thoughts concerning different aspects of the listening process, which subsequently could be collaboratively reflected upon with their partner and teacher. This collaborative reflection may have supported the learners in resolving their challenges with listening strategies.

Conclusion and implications

As noted above, this study explored the effects of OSMLS accompanied with WL on Iranian IELTS candidates’ LC. The findings showed that OSMLS accompanied with written WL contributed to a better performance from EG at the end of the interventions. According to the results of the study, it may be argued that L2 listeners can develop more metalinguistic awareness if they use written language when they engage in processing L2 comprehension.

The findings of the current study have implications for different stakeholders involved in EFL listening instruction and assessment. First, school principals, university officials, and language institute owners are advised to use online education as a beneficial alternative to traditional classes in light of the increasing demand for online learning. For this purpose, they can equip their education centers with new technologies and train their EFL teachers and students to raise the required literacy for benefiting from online education. Second, EFL teacher trainers can benefit from the study’s findings to draw EFL teachers’ attention to the advantages of OSMLS and how they can foster learners’ autonomy and self-regulation in listening. Third, EFL teachers are encouraged to draw upon the findings of the current study and reconsider their approaches to teaching listening. According to the study’s findings, they can engage EFL learners in online SAMS and afford them the chance to collaboratively reflect upon their WL and provide them with constructive feedback on their listening performance and strategy use. Fourth, materials developers are another group who can benefit from the study’s results. As Goh (2012) highlights, most of the listening activities in textbooks are restricted to some pre-listening activities and then a listening stage. That is to say, the listening task lacks adequate inclusion of parts on metacognition. To ameliorate this thorny issue, materials developers are suggested to incorporate simple checklists before and after the listening activities to raise EFL teachers’ and EFL learners’ attention to metacognitive listening strategies and help them monitor and evaluate their listening process and outcomes.

Some suggestions for further research are proposed based on the limitations imposed on this study. First, given that this study was confined to one language institute, future studies are recommended to be conducted in other language institutes across the country to enhance the generalizability of the findings. Second, since the participants of this study were IELTS candidates, future studies could consider participants with different levels of proficiency (e.g., beginning, intermediate, and advanced) to examine the effects of OSMLS accompanied with WL on different groups of learners. Third, considering the fact that this study was conducted in the setting of a language institute, future studies may explore the effects of OSMLS accompanied with WL on students’ listening comprehension (LC) in schools and universities to investigate the applicability of the intervention in different contexts. Lastly, as the present study used a quantitative design, upcoming studies might use qualitative designs such as a microgenetic development approach to reveal how OSMLS accompanied with WL can lead to an improvement in IELTS candidates’ listening comprehension by providing more detailed and nuanced data on the change process.