ELL’s science meaning making in multimodal inquiry: a case-study in a Hong Kong bilingual school

Abstract

This paper reports on a multimodal teaching approach delivered to grade 5 elementary students in a bilingual school in Hong Kong, as part of a larger research study aimed at supporting English Language Learners (ELLs) in science class. As language demands of reading, writing and talking science place additional challenges on ELLs, there is much research interest in exploring the use of multiple modes of communication beyond the dominant use of verbal and written language. Research has shown that students develop a better scientific understanding of natural phenomena by using and alternating between a variety of representations. Yet, questions remain as to what meanings ELLs make during a multimodal discourse and, in turn, how such discourse provides support to ELLs in learning science. Drawing on social semiotics, which theorizes language as a meaning making resource comprising a range of modes (e.g. gestures and diagrams), we used a case-study approach to examine how a multimodal instructional approach provided 10 students with multiple avenues to make sense of science learning. Video recordings (capturing gestures, speech and model manipulation) and student works (drawing and writing) were collected during nine inquiry science lessons, which encompassed biology, physics and chemistry science units. Multimodal transcription allowed discourse to be analysed at a fine-grain level which, together with analysis of student works, indicated that the multimodal instructional approach provided the necessary inquiry opportunities and variety of language experiences for ELLs to build science understandings. Analysis also revealed how the affordances of modes attributed to the meaning making potentials for the ELLs and how they provided alternate communication avenues in which new meanings could be made. The findings from this study have implications for ELLs learning science within the growing multilingual Asia-Pacific region.

Introduction

Communicating scientific concepts is often challenging due to the unique structure of scientific language. Scientific language has certain quirks which makes it harder for the general population to acquire and use. The unique structure of scientific language poses a great challenge to its learners (Fang, 2005; Norris & Phillips, 2003). For English Language Learners (ELLs) who are “in the process of actively acquiring English, and whose primary language is one other than English” (Bardack, 2010. p.7), this poses an additional challenge. This is because in order to understand science, ELLs must learn the language of instruction (LOI) while simultaneously learning language of the subject area, along with vocabulary and content (Gibbons, 2003; Haneda, 2014; Lee & Luykx, 2005). To make matters worse, many ELLs in Hong Kong studying in a bilingual school are presented with further challenges as the LOI in science may alternate between grades or within grades from one science topic to another. For example, Earth science (climate, landforms, water-cycle) may be taught in Chinese while Life science, a topic with considerable links to Earth science (for instance ecosystems), is taught in English. Thus, inconsistencies in the LOI cause the segregation of science topics. Currently there is a need for research on bilingual education in curriculum, including science curriculum, (Goldenberg, 2013) in the context of Hong Kong because some studies have shown concerns for ELLs in science learning (Marsh, Hau & Kong, 2000; Yip, Tsang & Cheung, 2003). These studies further affirmed low proficiency in English had negative impacts on students’ achievement scores in science.

Despite the challenges of learning science for ELLs, English immersion is set to continue as a result of its popularity, particularly in non-English speaking countries (Lo & Lin, 2015). In Hong Kong for example, political, economic and societal factors influence the decision of LOI (Lin & Man, 2009; Perez-Milans, 2014). In fact, parental and socio-economic pressure resulted in a change of LOI, following a decline in achievement scores in subjects that were taught in English, including science. The decline was linked to the LOI reverting to Chinese, following the 1997 handover (transfer of sovereignty) of Hong Kong from the United Kingdom. This action not only caused a decrease in test scores, but also limited students’ university choices (Perez-Milans, 2014) and future aspirations. Consequently, parental and social-economic support for English immersion in Hong Kong has strengthened (Hoare, 2010; Lin & Man, 2009), but problems for ELLs in science are likely to persist.

As English immersion increases, so too does the need to resolve ELLs’ problems in science. For ELLs, the majority of problems in science result from the extensive repertoire of scientific language (Echevarria, Richards-Tutor, Canges, & Francis, 2011; Poza, 2016). This is due to the abundance of content and skill specific vocabulary in science. It is also an outcome of several grammatical aspects such as high lexical density, nominalisation, generalisations, technicality, and authoritativeness (Bruna, Vann, Perales, & Moisés, 2007; Fang, 2005). To further accentuate this problem, elementary science teachers are offered little support and remain unaware of how best to teach science to ELLs (Lee & Luykx, 2005; Lee, Maerten-Rivera, Buxton, Penfield, & Secada, 2008). This includes a lack of professional development for elementary science teachers that would ensure they understood the complexity of language repertoire and register for their subject area (Poza, 2016). This limitation leads many teachers to teach scientific vocabulary only (Bruna, Vann, & Perales Escudero, 2007; Halliday & Martin, 1993) and disregard the scientific grammatical aspects mentioned earlier. Teaching only scientific vocabulary or isolated content means there is a possibility that taxonomic relationships will not be established, which are significant to building scientific knowledge (Bruna, Vann, & Perales Escudero, 2007). Subsequently, ELLs require more support (Goldenberg, 2013) to use and apply scientific academic language, as it remains a key component necessary for their future success in science (Taboada & Rutherford, 2011).

Given the complexity of the issue, more research is warranted, specifically, research that explores the teaching and learning of science in bilingual settings in classrooms where English is the LOI. In in this study a case study approach was used to explore ELLs’ meaning making in science at a bilingual Hong Kong school. A multimodal instructional approach (MIA) was adopted and used to teach nine inquiry science lessons where English was the LOI. During each lesson, video recordings captured the meaning making of ten grade 5 students, the majority of whom were ELLs.

The goal of this study was to gain a deeper understanding of ELLs’ meaning making in science. This is necessary before attempts could be made to improve their science learning using English as the LOI. Only by understanding how something works will facilitate educators and researchers to enhance, improve and extend ELLs’ meaning making process. It will also generate new insights that could direct further research with the potential to result in beneficial teaching and learning implications for ELLs. So, given that the majority of studies of ELLs in science have not examined in what ways meanings are made, this study sought to contribute more information on ELL meaning making.

Theoretical implications for the study

Language-based theory of learning

This study uses a sociocultural lens to explore the nature of the relationship between science learning and language in an ELL science classroom. According to Vygotsky (1978), there is a strong relationship between learning science content and language. Thus, without good language skills, students struggle to learn science or any other content. This is because language is used to internalize thoughts which allows learning to occur (Vygotsky, 1986). The relationship between learning and language presents a new dimension to the challenge faced by ELLs in science. Since their problem stems from difficulties with the language of science (Bruna, Vann, Perales, & Moisés, 2007; Echevarria, Richards-Tutor, Canges, & Francis, 2011; Fang, 2005, Lee et al., 2008; Taboada & Rutherford, 2011), it suggests that there is an imperative need to support ELLs language abilities when they learn science.

The function of language

Therefore, a closer examination of language is necessary to support ELLs in science, specifically, an examination of the function of language, which has been questioned by theorists. For instance, Halliday’s (1978) Systemic Functional Linguistics (SFL) rejects the conventional view that language functions as simply a medium to communicate thoughts. Instead, SFL proposes language as a meaning making function. It states that individuals use language resources accessible to them to make meanings. This suggests that language can synchronously be a resource and an ensemble of resources resembling a system (Wells, 1994). This position is consistent with Vygotsky’s (1986) depiction of language as a sign-based tool. Therefore, language is composed of multiple resources and functions as a source of meaning making. This implies ELLs can make meaning in science in multiple ways using a variety of language resources.

With regard to the implications of language for learning, SFL makes similar inferences to Vygotsky’s learning theory described earlier. While Vygotsky outlines the important role language plays, Halliday (1993) positions language as the “process by which experience becomes knowledge” (pp.93–94). Thereby offering language as the necessary system for the acquisition of knowledge. Furthermore, the fact that language includes multiple resources implies two things. First, language resources differ from each other, and second, the choice to use a language resource is dependent on the participants’ selection. The first point implies that if language resources are different they will have different potentials for meaning making. The second implication suggests that language is constructed by those who use it. With regards to science, these implications indicate that scientific language is formed by the scientists who use it while at the same time influenced by the language of science (Fang, 2005; Fang & Schleppegrell, 2010; Halliday & Martin, 1993; Tippett, 2016) including the meaning making potentials (affordances) of each language resource (Lemke, 1990). For ELLs, this insinuates their involvement in the construction of scientific language by way of language resources.

Systemic functional linguistics influences social semiotics

Social semiotics contributes more details regarding the language resources (meaning making tools) available to ELLs in science. Social semiotic theory was formed on the basis of SFL (Halliday, 1993), and offers a perspective of language as a semiotic (meaning making) system, inclusive of a collection of meaning making tools. For this study it is important to differentiate between an isolated meaning making tool, referred to as a mode (of communication), and something that harbors multiple meaning making tools, referred to as multimodal. For instance, the nature of science is multimodal because the act of doing science involves using a variety of modes such as: speech, gesture, diagrams, models, graphical representations, simulations, and mathematical expressions (Lemke, 1998; Kress, Jewitt, Ogborn, & Tsatsarelis, 2014).

The examples listed are considered modes of communication because all have the potential to make meaning. Furthermore, meanings made using modes are made through the creation of signs. This is a process which is dependent upon a person (producer) constructing the sign, and a person (reproducer) receiving and interpreting the sign (Kress & van Leeuwan, 2006; Kress, 2010). Since signs are created and interpreted by people they are considered to be social and cultural constructions. All signs include three systems of meaning making or metafunctions. These include: ideational, which represents ideas about the world, interpersonal, which refers to relationships and interactions and textual, which refers to the organisation of signs and connections made to other signs in a text (Halliday & Matthiessen, 2014). These metafunctions can be used to infer the meanings of signs that are created. For example, Kress, and van Leeuwen (2006) draw on metafunctions to support their examination of images. In their book Reading Images, they illuminate the meanings behind the way image elements (visual signs) are used. In other words they discuss the grammar of visual design. To achieve this aim, they used the three functions to present information on; visual patterns of representation, patterns of interaction and texts. Similarly, we use metafunctions to support the analysis of the meanings made in this study.

Social semiotics, like sociocultural theories, credits all communication to human experiences and considers interactions to be fundamental to meaning making. Thus, for meaning making to occur collaborative experiences that include multiple modes of communication (such as speech, diagrams, models and writing) are pertinent. Although science classrooms have been adept with multiple communication modes for years, social semiotics has highlighted their value and inspired further educational research. Researchers have been interested in multi-representations (Ainsworth, 2008; Gilbert & Treagust, 2009) and multimodal representations (Kress, 2010; O’Toole, 1994; Márquez, Izquierdo, & Espinet, 2006). For example, Ainsworth (2008) explored the roles multiple representations played in science education and provided recommendations when using them with students. While Gilbert & Treagust (2009) focused on representations related to the learning of Chemistry, specifically regarding the relationship between the three types: macro, sub-micro and the symbolic. Kress (2010) presented a view of communication and meaning making through the lens of multimodality, examining the twenty-first century modes of communication. Whereas, Marquez, Izquierdo, & Espinet (2006) developed a theoretical framework to study the teachers’ use of communicative modes in a science classroom. Despite the variety of research, many researchers have not yet investigated the use of multiple modes with ELLs in science.

Implications for meaning making in science for ELLs

Multimodal learning instigated

The theories described provide insight to how meanings are made in science, suggesting ELLs learn through the social construction of scientific language, considered to be multimodal. On closer inspection though, it is actually during the assembly of modes that meanings are made (Bezemer, Diamantopoulou, Jewitt, Kress and Mavers, 2012), as a whole science concept exists only within the integration of modes (Lemke, 2000). This is because in science one mode is unable to make meaning alone (Lemke, 1998). For instance, focusing solely on writing provides an incomplete depiction of a concept (Jewitt, Kress, Ogborn, Tsatsarelis, 2000) such as force. Whereas, adding mathematical formulas, diagrams and simulations provides much needed information to achieve an in-depth understanding. Thus, meaning making in science is purely contingent on the ability to recognize and represent science concepts in multiple modes (Waldrip, Prain, and Carolan, 2010; Prain and Tytler, 2012), it is also dependent upon the ability to translate one science representation from one mode into another mode (Ainsworth 1999; Russell & McGuigan, 2001), such as writing into a model, or gesture into diagrams. An ability to do so can be considered to be part of a student’s “representational competence” (Tippett, 2016, p.727). This term also encompasses an ability to understand a mode’s form and function as well as create representations in multiple modes (Tippett, 2016). In this study ELLs used and expanded their representational competence during the inquiry lessons.

Scientific inquiry

Since the provision of inquiry learning allows social and collaborative opportunities and is an expectation of science teaching, the lessons within this study aimed to provide inquiry experiences for ELLs. Inquiry learning can be seen when “the learner is challenged to gather and analyze information, review it against existing knowledge, seek connection, notice patterns and gradually build an understanding of a concept” (Murdoch & Claxton, 2015 p.14). Intriguingly, inquiry learning also utilizes multiple communication modes including; verbal, visual, oral, pictorial, graphic and textual (Lee & Buxton, 2013; Lee, Maerten-Rivera, Penfield, LeRoy, & Secada, 2008). What is more, studies show inquiry approaches to be particularly effective with ELLs (Lee & Buxton, 2013; Moore & Smith, 2015). This is because they provide an authentic context for language use (practice) in science as they incorporate: hands-on experiences, student-centered meaning making, investigation of scientific phenomena, observation and experiments (Stoddart, Pinal, Latzke, & Canaday, 2002). In these contexts, ELLs have the ability to use and practice the language of science due to the availability of alternate literacy modes such as, gesture, model manipulation, drawing, and experimenting. Many of them are less dependent on traditional English literacy forms (e.g. reading and writing) and incidentally offer more opportunities for ELLs to partake in discourse regarding scientific knowledge and process with others (Lee & Buxton, 2013). Furthermore, relationships between language and science concepts are better established during inquiry learning. This is because, firstly, language is being used within the context to which it relates, and secondly, scientific inquiry approaches can be paralleled to the use of language functions including; predicting, hypothesizing, describing, reasoning and explaining (Stoddart, Bravo, Solís, Mosqueda, & Rodriguez, 2011). In this study, contexts that provide ELLs with experiences to authentically construct, connect and use language (including those just presented) will be referred to as language experiences.

Additionally, interventions utilizing inquiry learning with ELLs have had positive outcomes (Amaral, Garrison, and Klentschy, 2002; Lee, Llosa, Jiang, Haas, O’Connor, and Van Booven, 2016). In fact, Lee and Buxton (2013), compiled intervention research used to support ELLs in English proficiency as well as science. They revealed that effective teachers communicated ideas through multiple modes, and used a variety of methods to explain concepts, such as: hands on experiences, models, realia (the use of real objects or events), and demonstrations. Furthermore, they found effective teachers employed nonlinguistic modes, such as: data, tables, graphs, diagrams, and pictures, to allow opportunities for language construction and communication. Moreover, effective teachers used activities such as those listed above as contexts in which to model language and encourage communication of ideas and high order thinking (Lee & Buxton, 2013). Since inquiry learning has similar attributes to multimodal methods and promotes beneficial outcomes for ELLs, an assumption can be made that multimodal methods will also likely have positive outcomes for ELLs.

Studies with ELLs

In fact, the few studies focused on using multimodal methods with ELLs, concurred with our assumption and found them to be useful in the learning of science (Adamson, Santau & Lee, 2013; Choi & Yi, 2016). Bravo and Cervetti (2014) concluded that multimodal methods are successful for ELL learners because they “allow multiple entry points into understanding and processing” (p.242) while other researchers found ELLs to have benefited from the use of multimodal methods; including reading, writing, talking and participating in science processes (Adamson, Santau, & Lee, 2013). Other beneficial outcomes for ELL learners include: improving ELL’s self-esteem and sense of accomplishment, strengthening their understanding of texts, making meaning relevant, and giving all students a voice (Choi & Yi, 2016). Despite the advantages that multiple modes of communication offer ELLs in learning science, there appears little research making use of these approaches to support ELLs in science. Currently more research is needed which examines how multimodal approaches actually offer support to ELLs. This study seeks to add new understandings as to how ELLs make meaning in science lessons when teachers use multimodal approaches to teaching and learning.

Research questions

Therefore, to address the gap identified in the research, an investigation into how meanings are made by ELLs within multimodal science lessons was warranted. Informed by our objective to explore how meanings were made within an adopted MIA, the specific research questions were: How are meanings made in a multimodal instructional approach? How does the use of modes within a multimodal instructional approach support ELLs’ meaning making?

Methods

For our methodology, we used a case study as it was appropriate in answering an explanatory how question through an in-depth holistic investigation (Merriam, 1998). Furthermore, a case study allowed for exploration and understanding of a phenomenon (Denzin & Lincoln, 2005; Guba & Lincoln, 1989; Stake, 1994) because it yielded a comprehensive holistic approach to understanding educational phenomena (Merriam, 1998). In this case the phenomenon was the meaning making of ELLs during a MIA. The case study methodology enhanced understanding of the phenomenon by enabling an inquiry into this complex social situation that contained more than one variable of possible significance (Merriam, 1998). Given the variety of ways meanings could be made, multiple variables existed. This meant multiple data collection and analysis methods were necessary to capture the different meaning making tools utilized by the ELLs. Moreover, a case study ensured investigation of the complex social interactions (Merriam, 1998) among ELLs during multimodal discourse. Finally, the case study methodology allowed this study to be comprised of multiple cases. For instance, the meaning making of ELLs in science was investigated by adopting a MIA and integrated it into three grade five science units: biology, physics and chemistry. Each science unit (inclusive of three inquiry lessons) was treated as a separate case and analyzed separately. This enabled the researchers’ credibility of results and allowed triangulation to occur. However, this paper will focus on only one lesson from one representative case to illustrate the common patterns found in other cases.

Research site

The research occurred at an independent kindergarten-to-grade-twelve bilingual school in Hong Kong which prides itself on entertaining a bilingual and bicultural ethos with commitment to integrating Confucian philosophies and Chinese heritage. As a result, the ratio of the LOI shifts from Putonghua (Mandarin) to English, beginning in kindergarten with a percentage of seventy-thirty respectively and progressing to fifty-fifty by grade five. In Hong Kong, Chinese and English are the official languages spoken, although Chinese can refer to Cantonese (from Canton Province) or Putonghua from mainland China. Cantonese is the dominant Chinese dialect spoken in Hong Kong, but increased migration from mainland China is beginning to change this demographic, and the percentage of Putonghua speakers is increasing. As the school site honours China’s heritage, a dialect from mainland China was considered more appropriate. Thus, Putonghua was taught to students complete with traditional Chinese characters.

In science, the LOI not only changes between grade levels but also between subject matter. Additionally, the time allotted to teaching science in English shifts. Science is taught in Putonghua from kindergarten to grade three. In grade three however, three additional forty-minute lessons are provided for science and these lessons are delivered through an English LOI. Instructional time increases once again in grade four, with two additional forty-minute lessons a week, which remain consistent in grade five. With respect to the teaching of science, the responsibility falls on the language teachers of Putonghua and English.

Participants

The teacher-researcher was of European descent and English was her first (and only) language. She had over fourteen years of international experience teaching primary students of diverse ages and backgrounds including ELLs in England, Germany, Singapore and Thailand. She had achieved a Master’s of Science Education from Australia and was currently Head of (English) Science at the site. This position was shared with a Chinese colleague who managed the science units taught in Putonghua. The teacher-researcher also taught English Science to a fifth-grade class comprised of twenty students. Purposeful sampling (Creswell, 2007) was necessary to ensure the research took place inside the teacher-researcher’s classroom which was predominantly comprised of ELLs.

Thus, ten students from the teacher-researcher’s class participated. Each was bilingual and spoke varying degrees of Chinese and English. Since the school was an independent bilingual school steeped in Chinese heritage, it attracted mostly affluent Chinese families from a high socio-economic background who could afford to pay for their child’s education. All of the student participants had Chinese heritage and currently lived in Hong Kong. Most had been born in Hong Kong except three; two had been born in mainland China and one had been born in the United States of America. The vast majority of students did not speak English as their first language, although in bilingual households it can be difficult to distinguish a first language. However, in addition to speaking Putonghua and English, five of the participants also spoke Cantonese fluently. Nevertheless, regardless of their language or science abilities, the students were split into two equal groups of five and remained so for the duration of science lessons within the study.

Multimodal instructional approach

As part of the study, a multimodal approach was adopted to design a series of lessons for the grade 5 students. The MIA (Fig. 1) chosen for the study was adapted from the Thinking Frames Approach (Newberry & Gilbert, 2016). Through this approach, students learned science through an inquiry of a puzzling phenomenon and by using multiple modes of representation to explain the phenomenon. The modes of representation included verbal, drawing and writing.

Fig. 1

The MIA adapted from the Thinking Frames Approach (Newberry & Gilbert, 2016)

To address the research questions, we identified a dominant mode of representation at various stages of the lesson. It is important to acknowledge that multiple representations are always juxtaposed and used concurrently at any moment (e.g. talking while pointing at a diagram, drawing with some annotated words). However, it is possible to identify a dominant mode in which the participants oriented themselves towards most of the time (Tang, 2016). As the dominant mode of representation framed and organized the actions of the participants at any moment, each dominant mode was considered to be an indicator of a separate stage, shown in Fig. 2. The multimodal meaning making was observed from within each stage; Stage 1 with a dominant verbal mode, Stage 2 with drawing and Stage 3 through writing. Meanings were also investigated between the successive stages, between Stage 1 and 2 and Stage 2 and 3.

Fig. 2

Dominant Modes and Expected Alternate Modes within the MIA

The commencement of the inquiry mandated a ‘how’ or ‘why’ question be presented to the students. The question was presented with the scientific phenomenon to excite the students and stimulate curiosity. Consequently, the teacher employed either an experiment or demonstration using a model to launch the question. For example, the lesson described within this paper required a demonstration be performed by pulling a dollar bill quickly from under a stack of coins that was resting on top of a glass bottle. The force applied to the bill caused it to detach, while the other objects remained motionless. Before showing a demonstration, students predicted the result which probed their initial thinking. The inquiry question was posed following the demonstration, for instance; Why did the coins stay on the bottle? Immediately after revealing the phenomena, student groups began to co-construct their explanation and progress through the three stages of the inquiry lesson to answer the inquiry question.

The first collaborative task (Stage 1), required the students to choose science concepts that were relevant to the phenomenon observed in the demonstration. To aid the ELLs in this pursuit, catalogues were provided with key science representations (words, images, mathematical formulas) from each science topic: Life and survival, forces, energy changes and particles (Newberry & Gilbert, 2016) and remained with each group for the entirety of the lesson. The requirement of a group consensus promoted the sharing of ideas and prior experiences, and caused debates over relevant science concepts. The second collaborative task (Stage 2), necessitated the translation of a (predominantly verbal) explanation into a pictorial mode through drawing. The final task (Stage 3), required translation of an ELLs existing explanation into a written mode. In particular, each ELL was expected to create an explanation paragraph. Writing templates promoted the planning and organisation of ideas. Accomplishment of each task in the MIA necessitated that ELLs revisit the inquiry question to provide an appropriate explanation, represented in each dominant mode. A final support offered was teacher support, in the form of probing and questioning to incite ELLs to think more critically. By the end of each lesson, one group drawing and five written paragraphs were produced on paper.

Inquiry lesson: Why did the coins stay on the bottle?

To ensure accurate evaluations of ELLs’ meaning making an explanation of the science concepts within this lesson is required. As this lesson occurred several weeks into an eight-week Forces in Motion unit, the ELLs had already had several lessons on concepts, such as: motion; direction; force; mass; acceleration; friction and gravity, as well as the difference between balanced and unbalanced forces. The purpose of the inquiry lesson described in this paper was to provide ELLs with an opportunity to enhance their understanding of these concepts by having them apply their current knowledge to explain a real-life phenomenon. An example of an accurate explanation is as follows:

The coins stayed on the water bottle because of Newton’s first law of motion known as inertia. Since the objects were not moving, they were at rest or balanced. Objects at rest will remain at rest unless provoked by a force. Therefore, when the bill was pulled in one direction, it caused an unbalanced force allowing it to move in the given direction, but since the strong force was exerted only on the bill all other objects remained stationary. Friction was overcome by the strong pulling force on the bill and was helped by smooth surfaces of the objects touching it.

Data collection

If meaning making occurs within the social and collaborative construction of signs, data collection strategies were necessary that captured each entire inquiry lesson. As the stages and discourses in the MIA equally had the potential (and were most likely) to be multimodal, video was necessary. Video could capture modes such as verbal, gestural, and model manipulation. Of the two cameras used, each was responsible for capturing a participant group. Video recordings of participant groups on average lasted approximately 40 min, although at the beginning of each lesson, one camera was also used to record any initial teacher demonstrations or experiments. These usually only lasted several minutes. Overall, a total of nine inquiry lessons with two cameras equated to the collection of approximately 360 min of recordings.

In addition to the recordings, artefacts made by the ELLs through other modes, such as the drawings and writings were collected. These were just as important as the recordings since meanings had the potential of being made by ELLs in any mode and through multiple modes. This meant each written paragraph, drawing or even doodle was of importance and needed to be analyzed separately as well as concurrently with the recordings. Therefore, the artefacts that were collected included A3 size sheets used for group drawings as well as individual A4 sheets used for the written paragraphs. All artefacts were collected from all participants at the completion of each lesson.

Analysis of meaning making

We adopt the notion of semiotic units in our analysis, to investigate meanings made within the inquiry lesson. Semiotic units are combinations of signs that are used to express a complete meaning (Wright, 2011). In this study semiotic units were considered the integrated signs that combined to represent one concept or idea. They were created through the multimodal discourse by one or more participants. When a concept or idea changed during a multimodal discourse, a new semiotic unit evolved and a new discourse began. Semiotic units and discourses could vary considerably in length. To distinguish discourses from one another we applied a construct similar to textual organisation, where one paragraph presents one idea. Likewise, individual discourses were considered to present an idea. Since discourse evolved through social interactions, boundaries were often blurred or intertwined from one into another, thus researcher checks were used to validate the analysis.

The signs constructed using modes in discourses were predicted to be the means through which ELLs made meanings. Since a variety of modes were available within the MIA, meanings were also predicted to be found within a multimodal discourse. Therefore, we chose to analyse the (signs used to create) meanings within discourses using two of the three metafunctions (described earlier), ideational and textual. Ideational has been referred to as presentation meanings (Lemke, 1998) and is closely aligned to the content matter of the science curriculum, requiring a coarse level analysis. Whereas textual, referred to as organizational meanings (Lemke, 1998) includes the intimate joining of words in a grammatical pattern and thus required a fine-grained level of analysis.

To do this, we examined discourse within semiotic units. This was made possible by discovering the meaning making potentials offered by modes, referred to as affordances (Prain & Tytler, 2012; Kress et al., 2014). Meanings made are dependent upon the modal affordance (Kress, 2010) of each mode which describe its constraints and potentials. For example, in writing, meanings are made through the use of words and grammar; likewise, in drawing, meanings are made through the affordances including line, colour and space. Gestures include types of movement, and speech has the capacity for tone and volume changes. Inspecting the meaning potentials of modes allows a more in-depth conception of the semiotic units found in the multimodal discourses.

Video was used to capture the affordances of several modes, including the expression within an ELL’s speech as well as the detailed movements of a gesture. A two-part transcription was necessary to analyse multimodal discourses. First, a content log (Jordan & Henderson, 1995) helped to sequence events, enabling key frames inclusive of semiotic units warranting further investigation to be highlight. Next, these key frames were transcribed using a multimodal transcript, which provided a fine-grained analysis focused on a shorter timescale (Tang, Delgado & Moje, 2014). Other fine-grained methods employed by the researchers were: thematic analysis (Lemke, 1990), used to examine the semantic relationships among the words that were spoken or written by the ELLs during the inquiry lessons; mode continuum (Gibbons, 1998), which examined the language used in the final written artefact and compared it to language used by ELLs as they moved through each phase; and pictorial analysis of the drawings using Kress & van Leeuwan’s, (2006) visual framework.

In addition to the fine-grained analysis methods described, coarse-level analysis was also employed. This was indicative of a larger compositional grain size (Tang et al. 2014) and examined how representations as a whole (e.g. written paragraphs) were produced and re-represented. This included how science concepts were explained (accurately or inaccurately) when verbal explanations were re-represented into drawings, and vice versa. The collection of communication modes used by students during each of the stages of the MIA (Table 3) was also analysed. During all analysis, trustworthiness resided in the researchers critically reviewing each other’s interpretations (Guba & Lincoln, 1989).

In-depth conceptions were achieved through the multi-level analysis as it allowed comparisons to be made. Furthermore, the analysis and patterns found within each lesson were compared to others within and between the three different science units. This provided enough necessary evidence to allow trustworthy conclusions to be drawn. Finally, we used narrative description encompassing “thick descriptions” to ensure findings were presented clearly and comprehensively.

Findings from the analysis are discussed in the next section. To do this, examples of multimodal discourse from the inquiry lesson are presented. The discussion includes examples from both a fine-grain (thematic analysis) and coarse-grain (writing) analysis.

Findings

The findings will now be discussed in three episodes below. The episodes were elicited from one inquiry lesson and show how the use of modes within the MIA provided both inquiry and language opportunities. Moreover, they illustrate how meanings were made by ELLs as they progressed through the lesson and how the science knowledge constructed accumulated through the stages. Each episode describes findings by including either excerpts of highlighted frames from the multimodal analysis or ELL’s co-constructed representations.

In the following transcripts, which aim to illuminate assertions, individual students are denoted by letters, “K” refers to Student K. Other columns record signs found within modes. Speech is represented within one column and due to the limitation of space, gesture and model manipulation were grouped in another. During the participants’ speech, considerable pauses (more than two seconds) are shown by three dots “…” together. If a student gazed at another student it is identified by “>” where the first student (letter) is looking at the second, a mutual gaze between two students is identified by “><”, all other gestures and model manipulation evidence has been recorded in visual form.

Episode 1: Multiple modes and meaning making

The first and second episodes were taken during Stage 1 directly following the demonstration (described earlier) and delivery of the inquiry question: Why did the coins stay on the bottle? At this point, students were attempting to decide what science concepts and representations related to their explanation for what happened (See Table 1).

Table 1 Identification of Science Concepts in Group 2

Episode 1, illustrates that combinations of modes included in the MIA provided inquiry opportunities for ELLs that allowed meaning making to occur. Specifically, it shows how students participated in multimodal discourses regarding science knowledge and processes, representative of inquiry learning (Lee & Buxton, 2013). This was revealed in the following patterns, first, discourses promoted the connection of prior knowledge to new ideas and concepts, second, the affordances of modes ensured discourses were meaningful to the ELLs who constructed them, third, meaningful discourses disseminated from others, and finally, discourses either ended or evolved following the introduction of new concepts or ideas by ELLs or their teacher. The first three patterns will be discussed in more detail in this episode.

The early (predominantly verbal) discourse was initiated directly following the demonstration designed to challenge students to scientifically explain the phenomenon. Initially, ELLs connected their prior knowledge to the demonstration they just witnessed. To do this they considered the relevance of each concept through the calling out of keywords (Table 1, lines 1–14) while often pointing to the word or matching representations (lines 5, 7, 8 and 10) found on the catalogues provided (described earlier). Initially, the ELLs accurately recognised the concepts linked to the phenomenon, such as force (lines 1–6). Student P related force to “pull” (line 5) and Student A believed the force was so strong it could “make it (the bill) can withstand friction” (line 6). Other accurate concepts linked to the phenomenon that were called out were, “Newtons” by student P (line 7) and “balance” by Student K (line 8). The act of calling out concepts prompted each ELL to consider their current understandings. As the ELLs analysed information by relating it to prior knowledge they participated in inquiry learning (Murdoch & Claxton, 2015).

So, as the concepts were stated (or pointed at), ELLs either agreed if they believed it related to a scientific explanation of the phenomenon, or disagreed if they did not. Agreement was usually shown by verbal restatement. For example, in Group 2 (Table 1) Students A and Y (lines 3 and 4) parroted Student P’s statement “It’s forces” (line 2). Later Student P restated the concept “balance” (line 9) together with an affirming exclamation “yeah” and in line 11, she restated a concept with enthusiasm as indicated by the tone and volume (verbal affordances) in her voice, which were re-represented as an exclamation mark. Agreement did not always require additional discourses.

However, when ELLs disagreed or needed clarification of their friend’s assertions they began seeking more connections, this consequently meant that additional discourses were needed. The connections sought during these discourses were between ELL’s current understandings and either the understandings of others, or the concepts being discussed. The ELLs’ attempts to make connections was another aspect of inquiry learning (Murdoch & Claxton, 2015) that caused meaning making to be deepened. This was because the additional discourses allowed for more in-depth discussion of ELL’s ideas and concepts in question. Thus, further language construction and meaning making experiences occurred.

Furthermore, discourses where students disagreed or clarified their thoughts usually embraced alternate literacy modes (e.g. gesture), and differed from the dominant (verbal) mode. This is illustrated in Group 2 (Table 1) where we see that the suggestion of the concept “speed”, sparked a discourse that inspected how speed related to the phenomenon. For instance, when Student Y queried “balance or speed?” in line 3, she did so using the affordances of tone and speech within her chosen mode of communication: verbal. These actions (within this context) signified a question was being raised, directing others to make a choice. Following this, several students confirmed “speed” was more appropriate by (assertively and/or affirmatively) repeating it in lines 4–7. Actually, the majority of students (from both groups) described the movement of the bill using a familiar everyday word: speed. However, it was the rate of speed (acceleration) and not speed (velocity) that related to this phenomenon. Although the ELLs had previously completed science tasks related to acceleration they were unable to verbally explain and apply the concept to this phenomenon.

However, the affordances of alternate literacy modes allowed ELLs the ability to demonstrate their understandings in other ways. For example, Student P proceeded to use the affordance of movement within gesture, together with a verbal mode of communication (line 22) to rationalise her thoughts. As she did, she gestured an action in which her hand (pretending to pull a bill) accelerated horizontally toward her while she simultaneously described the movement “like quick” (line 22). Thus, even though Student P was unable to verbalise acceleration, she demonstrated an accurate representation of the concept through the affordances of gesture.

Likewise, when Student K (with an original suggestion of balance in line 1) deliberated between the concepts, balance and speed, she used the same modes (as Student P), gesture and verbal to communicate (line 15–18) her ideas on how speed altered the outcome of the phenomenon. She demonstrated using both modes “If you were moving very slowly… it will” (line 15) said Student K as she pretended to grip the note and pull it towards her at a constant speed. Consequently, Student K was able to communicate her ideas because she had the capacity to choose modes that achieved her objective and combine modes to support her sharing of ideas. In turn, this allowed ELLs to receive information in multiple forms and join in the discourse from multiple access points. As a result, one ELL in the group participated through gesture and speech (line 22) while others joined in verbally (line 19–21). Multiplying the different affordances of modes permitted Student P and K the ability to make-meaning by enacting while commentating the original demonstration to communicate ideas. This example showed the affordances of modes ensured discourses were meaningful to the ELLs who constructed them and made information accessible to others.

This fragment of the inquiry lesson depicts ELLs identifying, sharing and confirming the initial concepts they believed were necessary to explain the phenomenon scientifically. When the ELLs connected science concepts and key words with the phenomenon they observed, it was considered to be both a meaning making and language experience. This identification and sharing was necessary in Stage 1 of the MIA, because the students needed to agree on the relevant concepts before using them to co-construct an explanation. Achieving this step promoted the discussion of prior understandings and the ELLs were seen reflecting on past lessons, sharing personal experiences or using phrases such as “I think” to justify their choices. This excerpt has shown that the necessary shared consensus of concepts promoted supplementary discourses where understandings could be explored more acutely.

Additionally, supplementary discourses that enabled concepts to be explored in more depth usually warranted one of two things: either evidence be examined (as was the case explored in Group 2 earlier), or ideas be tested. In both cases ELLs were interested in finding patterns to support their understandings, which was yet another aspect of inquiry learning (Murdoch & Claxton, 2015). If ideas were to be tested they were usually done so through the use of alternate literacy modes, including gesture (shown in Group 2 earlier) or model manipulation. For example, in Group 1 the testing of an idea can be witnessed (Table 2) as student N employed model manipulation as a representational mode to test his own idea. Similar to Student K in Group 2, he was interested in the effects speed had on the phenomenon and claimed a fast movement was necessary to perform the trick. He asked the teacher, “I want to know why when, when the note is fast they (the coins) don’t drop and why the note, when, when the note its slow…” she cut him off to invite him to test his theory (Table 2). During the testing of this idea other ELL’s beliefs within Group 2 surface. In fact, Student H asks Student N to “Wait! Let me hold” (line 2) while she cups her hands around the base of the bottle ready for the coins to fall when the bill is pulled “fast” (Student N, line 1). Thereby, uncovering her expectation that the coins would fall and thus highlighting a misconception. Additionally, although Student N proved his idea, “Ok so when its slow… it (coins) moves, it (coins) moves with the note, but” (line 14 and 16), he proceeded to offer a new idea, “when its fast, the coins don’t have time to react” (line 16), consequently uncovering a new misconception, instead of relating the stillness of the coins to inertia or to overcoming friction he related it to time. Nevertheless, corresponding to Group 2, Group 1 focused on the degree of speed in which to pull the note without reference to the degree of force. This was regardless of the difference in accessible communication modes, since (similarly to the gestures used by Group 2) model manipulation afforded Group 1 the necessary movements in which to demonstrate, communicate and test theories on how to pull the note. In most cases ELLs in Group 1 used more force when they pulled the note quickly and accelerated as they did, however they were unable to verbalise the action using accurate scientific vocabulary. Regardless, model manipulation (and gesture in Group 2) allowed ELLs to re-enact the phenomenon to further explore concepts in more detail.

Table 2 Using Alternative Modes to Test an Idea in Group 1

These additional discourses ensured information was examined at a greater level in an attempt by ELLs to build understandings of the concepts. This is representative of inquiry learning (Murdoch & Claxton, 2015) and subsequently allowed further misconceptions to be uncovered. This is important when meaning making because misconceptions must be first uncovered before they can be amended by another student or teacher. Thereby more accurate and in-depth meanings could be formed. Meanings become more in-depth because not only do students understand why something is the way it is, but they also find out why something is incorrect. In essence, having the right answer is one piece of knowledge, but knowing why something is incorrect is another.

In sum, the accessible modes in Stage 1 (notably speech, gesture and model manipulation) allowed ELLs to experience inquiry learning. They provided an opportunity for ELLs share and connect their prior knowledge (as shown in Tables 1, 2 and 3) to the concepts identified. Furthermore, the use of alternate literacy modes that afforded movement such as, gesture or model manipulation allowed the re-enactment of the phenomenon. This supported ELLs communication and meaning making by allowing them to explore concepts more in-depth. Furthermore, the multimodal nature of the supplementary discourses provided ELLs with the necessary affordances in which to access, communicate and discuss information. Moreover, the multimodal discourse allowed the ELLs to participate and make meaning in science regardless of their English level.

Table 3 Modes Used by Participant’s during Inquiry Lesson 1

Episode 2: Interactions of modes for ELL science learning

The second episode illustrates that interactions of modes provide ELLs with the opportunity to participate in a variety of language experiences. Language experiences, defined earlier, refer to contexts where ELLs can construct, connect and use language in science. Language experiences are pertinent if ELLs are to develop science understandings in English science lessons.

In this episode, ELLs are seen integrating alternate literacy modes (e.g. gesture) with traditional literacy modes (e.g. verbal) when constructing multimodal discourse and by doing so, practice the language of science. Furthermore, it shows how the MIA afforded ELLs the necessary language construction opportunities in science because the modes employed (during each stage) were selected by the participants of each multimodal discourse. They were not determined by the teacher’s instructions or dictated by the dominant mode as previously anticipated. For instance, Tables 1, 2 and 3 shows ELLs self-selecting modes (from those available) during multimodal discourses, such as in Table 1, where Student K (line 15) was seen choosing to add additional mode to the current discourse. Student K decided to gesture when words did not communicate the information she wanted to share. Thus, while speaking “If you were moving very slowly… it will” she simultaneously gestured an action, pretending to grip and pull the paper (horizontally) slowly toward her. The choice to represent movement through gesture illustrated to the group the measure of speed, indicating exactly how slow, slow is. Thus, multiple modes ensured a variety of details of a concept were communicated, revealed and discussed.

Another example in Table 3, outlines the modes utilized by each individual ELL in each stage during the lesson. It shows that all (Student D was absent) participants (including the teacher) made personal choices regarding how to communicate scientific information. For example, Student J and H from Group 1 employed writing in Stage 1 (the dominant verbal mode) and Stage 2 (the dominant drawing mode). Similarly, Student K and Y in Group 2, employed gestures during Stage 3 which was considered the dominant mode of writing. In actuality, the modes were determined by the communication needs of the individual ELLs themselves. In essence, students constructed language by creating signs using modes that best communicated the scientific information they wanted to share. Therefore, the collective ability to choose from and have access to multiple modes, increased language communication opportunities for ELLs during science. This was important as science meanings are made through the construction of modes (Bezemer, Diamantopoulou, Jewitt, Kress and Mavers, 2012).

Moreover, the ability to convey language ensured ELLs had the capacity to participate in multimodal discourse. Since multiple modes are necessary to achieve an understanding of a science concept (Lemke, 1990), multimodal discourse (as seen in this episode) exposed multiple details about the science concepts, thereby enabling more comprehensive meanings to be made.

Participating in multimodal discourses also presented ELLs with a circumstance in which to practice the language of science. This was a consequence of the multiple access points offered to ELLs in multimodal discourse. More specifically, some modes (gesture, drawing, model manipulation) provide affordances that do not require knowledge of English (reading, writing, speaking) allowing discourses to continue regardless of ELLs English ability levels. For example, gesture allowed Student K to make meanings she may have been otherwise unable to do using only a verbal mode. In fact, the gesture (line 15) completed an idea regarding the relationship between speed and friction. So, when she chose an inaccurate vocabulary “like crash” (lines 17–18), her fellow ELLs comprehended what she meant (from her hand gesturing a horizontal motion; see line 18) and corrected her. Consequently, the misused vocabulary “crash” (line 18), referring to what happened to the coins after the note was pulled slowly, was corrected with “fall” (line 20) by Student A and Student Y. Thus, English words were able to be connected to the gestural movements of Student K. Therefore, multimodal discourse allowed ELLs to participate by using alternate literacy modes while simultaneously learning more English vocabulary.

Additionally, Table 3 shows ELLs communicating through modes other than the dominant mode of a respective stage. As discussed, ELLs were seen adopting alternate modes for several reasons; explaining perspectives (see Table 1, lines 15–18), defending choices of science concepts and representations, and testing ideas (see Table 2). Nevertheless, regardless of ELLs choice of mode, multimodal discourses permitted the appropriate opportunity for the practice of the language of science.

The aptitude to self-select an appropriate mode (from multiple) ensured all ELLs had a capacity to convey meanings and communicate ideas, by choosing modes that offered alternate literacies. Furthermore, it provided ELLs with opportunities to practice the language of science. Overall, the discussion of science concepts through multiple modes insured more comprehensive meanings were made of science concepts.

Episode 3: Adding modes to ELLs science discourse

Episode three illustrates how certain science information can only be communicated through a combination of integrated modes (e.g. drawing and speech) because the exclusive affordances found in each illuminate precise features of a science concept. Therefore, ELLs’ understandings of science concepts have the potential to be expanded by introducing modes. In this episode, we explore the effect that the translation of modes has on meaning making. In particular, we inspect how meanings were translated into the modes of drawing and writing, successively. The initial excepts from Stage 2, depict the collaborative translation of Group 2’s verbal explanation into a drawing. To gain a better understanding of the impact drawing had, a comparison between meanings made in the first and second stages of the MIA is presented. In addition, the final excerpts from Stage 3 show Student K’s meaning making after the translation of her ideas into the mode of writing, following the construction of an explanatory paragraph.

The drawing mode promotes more detailed science meanings

Firstly, drawing is an alternate literacy mode and offers ELLs equal access to new information. As a result, ELLs can benefit from science concepts uncovered by the specific affordances that drawing provides, such as sequence. To illustrate this, Group 2’s final pictorial representation (Fig. 3), and excerpts from the multimodal transcription, taken before (Table 4) and during (Table 5) the construction the construction of the drawing have been included.

Fig. 3

The Drawing Co-Constructed by Group 2

Table 4 Group 2 Co-Constructing the Drawing

Table 5 Group 2 Co-Constructing the Drawing

This first excerpt shows how the act of constructing a collaborative drawing stimulated further questions from the ELLs. These questions had to be answered before drawing could begin because like Stage 1 (Table 1, lines 1–6), where there was a need for everyone to agree on the related science concepts, Stage 2 of the MIA also required a consensus regarding the subject matter of the drawing. This prompted the use of verbal and gestural modes to discuss presentational and organizational meanings needed to create the drawing. For instance, Table 4 shows Student Y portraying how the action of the phenomena should be depicted (line 3–4) using verbal and gestural modes, “make it slow”, then Student P intervened (line 5) to present her ideas, indicating how objects should be organised in the space provided. After which, Student P continued to suggest multiple drawings be created, “first” (line 7), and “then this one is when” (line 11) to show contrast. Finally, Student P decided that the sequence should include three drawings, “1, 2, 3” (lines 12–14). In this excerpt the affordances of drawing enabled the ELLs to view the phenomenon as a sequence and break the action into three parts. Separating the action encouraged the ELLs to consider what was happening in each picture in the sequence. This promoted a closer inspection of the science concepts at each step and in doing so allowed a more comprehensive explanation to grow, which will be illustrated in the next excerpts.

For instance, Fig. 3 together with (Table 5) the multimodal transcript (taken during construction of the drawing) illustrates how the affordances of movement, direction and sequence, allowed science concepts to be explored and represented in more detail. For example, the depicted objects (coins, money, bottle) were able to be juxtaposed next to one another to allow comparisons to be made and illustrate the passing of time.

Furthermore, sequencing permitted Group 2 to depict objects from before, during and after the bill was pulled. In fact, the objects at rest in the first image became more apparent when compared to other images in the sequence. This was because movement was depicted in the second drawing, using two small curved lines directly above the hand. For the ELLs, the chance to compare images illuminated similarities and differences in each, making it easy to ‘spot the difference’ or in this case draw the difference. Since the side-by-side drawings (Fig. 3) were mirror images (of the table. Bottle and coins), it highlighted their understanding that the pulling force was directed at the note only, despite the image of the coins and bottle depicting no change, the ELLs wrote “still resting” on the coins to demonstrate their knowledge of this.

Table 5 also shows how drawing led to the application and connection of more specific and detailed information surrounding science concepts. At this point in time, the drawing contained only objects, some of which had been identified using labels, as yet, no representation of force existed. The teacher began to probe using questions seen in lines 1, 5, and 6. As students began to justify their understandings verbally, the teacher directed their focus back to the drawing, “have you shown that in the picture” (line 12). Student P asked if they would write the information, in line 15 and again in line 19, where she demonstrated a small section of paper that could be used for a written description. When the teacher denied the request, she asked the students directly, “What did we use to show force?” (line 21), Student S responded “arrows” (line 22) repeated by Student P (line 23). Following this excerpt, the ELLs were required to identify the type of force, determine the size (strength) of the arrow and direction it travelled, all of which promoted a deeper level of thinking about the concept of force. For example, the identification of forces allowed Student A to add a quantitative amount for each force (as measured in Newtons). Additionally, the sideways movement of the hand pulling the bill was seen to be countering the perpendicular direction of gravity. Therefore, not only did the affordances of drawings support ELL’s awareness and understanding of force within their explanation, but they also gave ELLs a means to make visible their ideas in order to clarify their thinking. Moreover, the drawings supported the integration of modes including the necessary speaking and gesturing before and during the drawing, to the final matching of written scientific vocabulary to the images as the students were prompted to provide labels to show what was occurring in each.

In order to explore the effect the mode drawing had on the ELLs meaning making, closer inspection of meanings made between Stage 1 (without the mode of drawing) and Stage 2 was warranted. Therefore, a fine-grain method of analysis known as thematic analysis (Lemke, 1990) was used to compare the meanings made between semiotic units within different stages. In particular, we examined the taxonomic relationships of the vocabulary used by the ELLs that were associated with the concepts of force and motion. For instance, “friction” and “gravity” are hyponyms (subsets) of force and “fall” and “at rest” are hyponyms of motion. In this excerpt, we focus on the patterns found between Stage 1 and 2 of the MIA. The results in Table 6 show that both groups were able to identify vocabulary associated to the concept of force in Stage 1, believed to be from ELLs prior knowledge of previous lessons or past units. Stage 2, however shows an increase in the identification of vocabulary related to force and motion. Although some inaccuracies were present (in Stage 2), they were seen as positive additions, since it was assumed that more scientific vocabulary meant more concepts had been discussed. This suggests that the collaborative drawing in Stage 2 illuminated additional scientific information regarding the explanation that was untapped in Stage 1.

Table 6 Analysis of Meaning Making within Stages 1 and 2

This finding strengthened the researchers’ assertion that the modes accessible to the ELLs had a direct affiliation to the multimodal discourse and as a result, the meanings that were made. The consequence of the ELLs constructing a drawing facilitated a change to the multimodal integrations of previous discourse. Therefore, modal variance between Stage 1 and 2 was believed to be responsible for difference in information unearthed regarding the forces and motion concepts, thus offering broader and perhaps deeper understandings to develop.

The writing mode allows ELLs to connect and sequence science meanings

In Stage 3, writing is seen to promote the solidification of connections between the meanings made in previous stages. For example, an examination of Student K’s written explanation (Table 7) demonstrates how sentences with the conjunction ‘because’ were used in lines 3, 6 and 8. In fact, the construction of sentences highlighted cause and effect relationships for ELLs. For instance, Student K was able to demonstrate her knowledge of balanced forces through writing of sentences, which is mentioned three times in lines 3, 6 and 7.

Table 7 Student K’s Written Explanation with Lexical Strings; Synonym and Contrast

In addition, the logical progression of a written explanation required the formation of a sequence of events. This can be seen from a thematic pattern (Lemke, 1990) that shows the causal, temporal and logical semantic relationships of the discourse in Stage 3 (see Fig. 4). Within Fig. 4, the sematic relationships can be viewed by the grey lines; these have been identified between the key terms used by the student. For Student K they included subclass and a variety of synonyms, all of which were accurate. However, a teacher is also expected to question what is missing from an explanation, and in Student K’s explanation the semantic relationship contrast of balanced and unbalanced forces is missing. In fact, Student K does not describe the force of the pull of the bill nor why it moved at all. Nevertheless, what was written is accurate and considered an explanation of quality according to the curriculum level expected for these students.

Fig. 4

Student K’s Written Explanation

Furthermore, Student K’s written explanation (Table 7) supports the assertion that scientific information accumulated through the stages. For example, related concepts (e.g. force) were identified and explored (through gesture and model manipulation) in Stage 1, whereas in Stage 2 more details of concepts were uncovered (e.g. direction and location of forces) and in Stage 3 ideas were sequenced. Therefore, we found that each stage of the MIA was responsible for different meaning making outcomes. These outcomes collectively enabled the ELLs to make connections between scientific ideas providing an in-depth formation of science concepts, thus allowing the ELLs to form deeper meanings. For example, line 4 demonstrates meanings, such as the degree of speed, that were formed within Stage 1 during gestural and model manipulation. Whereas, line 3 illustrates concepts, such as balance that were identified in Stage 1 but explored within pictorial form in Stage 2, through verbal and gestural discussion with another ELL. Finally, Line 8 provides evidence of the information identified in Stage 1, connected in Stage 2 and reasoned (during verbal and gestural discourse) in Stage 3. Associations allowed.

Overall the findings of this study provided insights into how multimodal discourse allowed ELLs make meanings in science. Specifically, an important point was the translation of meanings into different modes, such as drawing and writing. Drawing added specificity to ELLs’ meanings, while writing required ELLs to form associations between meanings. The findings also showed how the MIA provided ELLs with the necessary support structures to learn science, including facilitating a context for inquiry opportunities and language experiences. These structures also promoted the use of scientific language and allowed scientific meaning making to occur.

Discussion

This study explored how grade 5 ELLs made meaning of science in a multimodal environment. Our analysis of an inquiry based multimodal instructional approach (MIA) to science showed the value of participating in multimodal discourses to make sense of science learning and the support that inquiry-focused MIA provided to improve science learning experiences of ELLs in a bilingual English school in Hong Kong.

Multimodal discourses for successful science learning

ELLs made meanings by participating in multimodal discourses, which ensured different scientific information was elicited through the variety of affordances of modes. These meanings were deepened as scientific information accumulated during the translation of modes within the MIA stages. This finding substantiated the idea that in order to understand a science concept, the assembly of modes (Bezemer, Diamantopoulou, Jewitt, Kress and Mavers, 2012) and integration of modes (Lemke, 2000) are necessary. Furthermore, this finding corroborated the notion that the translation of concepts from one mode into another is important when making meanings in science (Ainsworth 1999; Russell & McGuigan, 2001), and presents implications that this action leads to a more enriched understanding of concepts. For instance, the translation into drawing and writing caused more details of science concepts to be revealed. This was found to be due to the different affordances of the modes.

In fact, the meanings made during multimodal discourse were determined to be the direct result of the affordances of modes or modal combinations. This was because the affordances of modes had the potential to illustrate certain details of a science concept better than others. Therefore, providing ELLs with access to a variety of modes ensured that the meanings made were diverse. Furthermore, the accumulation of these meanings permitted the ELLs to gain a more comprehensive understanding and likewise, present a more informed scientific explanation of the phenomenon.

Multimodal instructional approach (MIA) to teaching science

The MIA provided the necessary inquiry opportunities and variety of language experiences to support ELLs in making meaning in science. In fact, for ELLs the inquiry opportunities provided by the MIA allowed authentic contexts for: (a) connections between prior knowledge and science concepts to be established, (b) ideas to be shared, refuted and tested, and (c) understandings to be collaboratively constructed through multimodal discourse. This finding is consistent with previous research that found inquiry approaches supported ELLs (e.g. Amaral, Garrison, and Klentschy, 2002; Lee, Llosa, Jiang, Haas, O’Connor, and Van Booven, 2016). For example, this study was similar to others that found the collaborative nature of the inquiry task provided the necessary context for the practice of language in science (e.g. Stoddart, Pinal, Latzke, & Canaday, 2002). Likewise, this study corresponded to others that found inquiry ensured ELLs’ language and scientific knowledge was supported through social discourse and cooperative learning (e.g. Amaral, Garrison, and Klentschy, 2002). Furthermore, this study was consistent with others that found inquiry provided ELLs with multiple access points to information during multimodal discourse (e.g. Bravo and Cervetti, 2014; Choi & Yi, 2016).

However, this study also expanded current research that used inquiry supports to help ELLs in science by offering more details as to how that support was achieved. For instance, the variety of modes made available ensured ELLs had multiple opportunities to communicate. Moreover, the alternate literacy modes (gesture, model manipulation, drawing) included in the inquiry lesson included affordances less dependent on traditional literacy modes (e.g. reading and writing). Thus, when modes were combined (such as in multimodal discourse), the affordances available to ELLs multiplied. This offered the ELLs multiple ways to express their ideas and increased their ability to participate and access information. Subsequently, this provided the ELLs with an increased chance to understand and support each other. Support in this study ranged from: communication support, such as rewording or translating signs, substituting signs and filling in discourse gaps; to meaning making support, such as the collaborative deliberation of ideas.

Since scientists develop the language of science from the modes of science (Lemke, 1990), it is important that ELLs have an equal opportunity to do the same when learning science. In countries and territories with multiple official languages, such as: Hong Kong, Macau, Singapore, India, South Africa and the Philippines; the opportunity to use multiple modes of science is of even greater significance. This study was conducted in Hong Kong where three different languages (Putonghua, Cantonese, and English) are official, so the participants (including the teacher) spoke varying degrees of one or more of these languages. However, despite a difference in the languages spoken by participants, all were able to communicate in the science lessons due to the use of science modes. In fact, findings showed that by participating in multimodal discourse during the MIA, all ELLs had the capacity to develop science language. Therefore, the modes in the MIA supported each ELL’s ability to communicate and share their scientific ideas and understandings with others. This consequently provided the ELLs with an opportunity to practice the language of science. In essence, the capacity to convey the language of science afforded ELLs the ability to learn science. Therefore, the MIA ensured all ELLs were provided the necessary language experiences in which to learn science. Thereby, using an MIA in science presented a more equitable learning environment because, regardless of students’ spoken language/s or fluency in the LOI, students were afforded an equal chance to participate and learn.

Implications for teaching ELLs

Exploring the meaning making of ELLs during multimodal inquiry science lessons highlighted several important factors for teachers to consider. Firstly, as science language learning is synonymous with science learning (Fang, 2005), then multimodal opportunities provide authentic science learning experiences for ELLs within science classrooms where English is the LOI. Not only do multimodal approaches provide a necessary language learning experiences, they are also consistent with the nature of science, and thus allow ELLs to learn through inquiry. In fact, because exploration of the phenomenon included multiple modes and the production of multiple representations, ELLs most likely developed deeper understandings than they would have if they answered the question immediately following the demonstration.

Additionally, corresponding with the findings of Sandoval, Bell, Coleman, Enyedy, and Suthers (2000) where complex ideas were seen to be explained more intelligently by students if they had the support of representations as opposed to not having them, this study found the production of representations allowed ELLs’ understandings, including misconceptions, to be highlighted. In fact, they enable the potential for peers and teachers to deliver guidance because they provide insights into students’ thinking. Thus, multimodal methods permit support to ELLs through the uncovering of scientific knowledge, since ELLs can explain understandings in more ways.

Furthermore, the availability of modes impacted the discourses produced and this subsequently effected the meanings that were made. Therefore, access to modes has implications for an ELL’s science learning. This study highlighted how ELLs selected modes that aided their communication regardless of the instructions given by the teacher. When participating in discourses, ELLs chose modes of communication that best served their individual capabilities, the subject matter, and the modes available in the given situation. Of significance to teachers is the importance of having a variety of modes available which plays a key part in limiting or enhancing student communication.

Finally, this study revealed how scientific understandings were built within a multimodal approach. This opens possibilities to science teachers housed in bilingual schools, where the LOI in science is English. Despite limitations of time, or the restriction to topics taught in English, following the recommendations from this study may help teachers to support ELLs in science. What is more, this in turn may provide additional support in implementing the bilingual school model.

Limitation

While the results seen within this study are encouraging, they have been collected within one case study of a small group of students. Considerations for future studies include the need to broaden the scope of this research by using a MIA with cohorts of ELLs in multiple grade levels and in multiple settings where the LOI is English. Furthermore, comparisons regarding the achievement of ELLs before and after this study could not be made and this would be a recommendation for future research as it would elicit further confirmation of the outcomes of a multimodal approach.