In recent years, theories relating to multimodality have challenged the privileging of language in the education system. Communication has always been multimodal – humans make meaning through various modes, including images and gesture – but schooling has focused primarily on oral and written language. There is a need to attend to other modes in the digital age, in which image and movement, for example, have become prevalent across all kinds of screens, (Kress, 2003; 2010). Flewitt (2008: 122-139) offers the following definition of multimodality:

‘The term ‘multimodality’ describes approaches to representation that assume communication and meaning-making are about more than just language. Multimodality takes into account the many different modes in printed and on-screen texts (such as image, layout, colour and language) and also the different modes that people use as they engage in face-to-face interaction (such as gesture, gaze, artefacts and language), and considers how these modes work together to create meanings in a ‘multimodal ensemble’.

Rather than drawing on modes in an arbitrary manner, children generally all use modes in strategic ways and are purposeful in their intent (Rowe, 2008). Whilst young children’s multimodal texts might appear at times to be an ad-hoc mixture of various materials, children have normally chosen their resources very carefully (Kress, 2003). There are similarities and differences in the reading and writing processes with regard to both monomodal and multimodal texts (Walsh, 2007).

In addition to drawing on a range of modes, children also move across media (forms of disseminating meaning) as they engage with texts. There are pedagogical challenges in this work. Teachers and children need to develop an understanding of the affordances of each mode, that is, an understanding of what each mode can offer in the communication process, and therefore an awareness of which modes should be used for what purposes. There is also a need to develop assessment criteria so that teachers are able to identify stages of development in children’s skills and knowledge in this area. Bearne (2009) proposes a framework for analysing children’s multimodal texts that pays attention to:

• Image: content, size, colour, tone, line, placing/use of space
• Language: syntax and lexis
• Sound/vocalization: content, emphasis, volume, vocal intonation, pause, pace
• Gaze: direction of gaze of communicator or character in representation
• Movement: gesture and posture

These aspects can be analysed across children’s multimodal productions, whether they are on paper, screen, or other form. The children in all of our case studies produced a range of multimodal and multimedia texts and a good example can be seen when children make instructional videos using a range of modes in the meaning making process:

• storyboards on paper – often the starting point for videos, though it is worth remembering that paper-based texts are rarely monomodal – they normally use images, space, colour and so on to convey meaning
• filming and editing – drawing on their understanding of the salience of various modes in order to decide how to juxtapose image with sound. One student (Case Study 3) demonstrated a clear understanding of how to plant a seed showing specific body movements and combined this with a dialogue to explain what was inside the seed and how it would grow.

Design is a term frequently used in relation to the creation of multimodal texts (Kress and Bezemer 2009). The concept reflects the intentional nature of text production as the producer carefully chooses specific modes for particular effects. Multimodal design involves choice and decision-making.

The work on multimodality over the past decade has served to emphasise the move away from the primacy of the written word in contemporary society and illustrates the need for pedagogical approaches that enable pupils to work productively with a range of modes and media. If classrooms fail to embrace these approaches, children will grow up unprepared for the textual landscape of the digital future.

For references see 3.6.6 References / Links to Further Resources