How to use multimodal text in literacy

Gill Matthews explores how the sights and sounds of multimodal texts can enrich your literacy lessons...

Often, when discussing the use of multimodal texts, we are inclined to move straight into talking about films. Films are indeed multimodal texts, in that they use more than one means of communication – pictures and sound. But multimodal texts don’t have to involve moving images.

The haunting illustration to the right, which is taken from the picture book Erika’s Story (Jonathan Cape, 2004 ) generates a huge amount of discussion amongst children and adults when prompted by a very simple question ‘What do you think is happening here?’

If you aren’t familiar with the book, it tells the powerful - and true - story of a baby thrown, by her mother, from a train en route to a concentration camp. The illustrations, by Roberto Innocenti, are poignant and moving. Putting sounds to the image, thereby creating a multimodal text, adds a further dimension. The sounds could be as simple as a passing steam train or a piece of slow music. A search for free sound effects on the internet will bring up numerous websites such as http://www.sounddog.com and www.stonewashed.net Most have a search facility to help you find what you want.

A resource that has taken this idea of coupling images and sounds is Picture the Music (http://www.picturethemusic.com). It contains over 500 original photos and illustrations and over 75 minutes of music. By pairing two or three different pieces of music with an image, you can explore with children how their responses vary according to the style and pace of the music. They could then develop class or personal word banks that they can draw on later in their writing.

Planning lessons around films

The National Strategies’ Y5 narrative unit based on the animated film The Piano must be one of the most popular on the website. If you are unfamiliar with using film to stimulate writing, it is a good starting point as it can act as a model for planning units around other short films.

Aidan Chambers, in his book Tell Me: Children, Reading and Talk (Thimble Press, 1993), suggests exploring children’s responses to texts through their likes, dislikes, puzzles (elements that the reader/viewer finds confusing or difficult to understand) and patterns (repeated phrases, events or behaviour, connections within a text and beyond). Although Aidan was referring to response to written texts, the approach works equally well with film and helps children to make sense of what they are seeing and hearing.

Another way of developing children’s responses to moving images is to explore questions that they would want to ask the film maker, such as:

• What made you choose that actor?
• Why did you choose to film from that angle?
• Why did you decide to include that person/building/landscape?
• Why did you pick that piece of music?

Children can then discuss the film maker’s possible responses, either as a whole class or in small groups.

Using adverts and film trailers

When introducing children to a film for the first time, try playing the first minute of the soundtrack without showing the images. Ask the children to visualise what might be happening and then either discuss it or write it down. Equally, you could show the film without any sound and discuss what kind of soundtrack they would add.

When choosing films to use in the classroom, short films can often be easier to work with than full length features – particularly if children are eventually going to produce a written version of the narrative. You could also consider using adverts and film trailers. The purpose of both of these is, of course, to persuade the viewer to do, buy or watch something. Children could be asked to consider who the target audience is; what information is given; the use of music, sound effects, voice overs and dialogue spoken by actors; the role of the characters; the narrative structure; the use of any graphics; any connections they make with other texts and, the big question, does it work as a persuasive text?

Producing different activities

The outcomes of a unit of work based on a multimodal text can vary and don’t need to be limited to a retelling of the story. You could guide the children in using what they have seen and heard to write character sketches, setting descriptions or a poem. They could write and record the voice over for a short documentary or advert. Using clips from news programmes could lead to children writing and recording news reports. And, of course, the outcome doesn’t have to be written. Studying multimodal texts can result in children using their visual literacy skills to make their own animations, short films, trailers, documentaries or adverts.

Lights, Camera, Action!

Six elements to explore in film…

1. Character

We can see what the main character looks like but what are they like as a person. What tells us?

2. Colour

What colours are used? Does the colour change? If so, when and how? What effect does this create?

3. Camera

How are long, medium and close up shots used? Why has the film maker chosen to use particular camera angles?

4. Setting

When and where is this film set? Does it remind you of anywhere you’ve been? How is atmosphere created?

5. Sound

What sounds are there in the film? What do they add? Are they inside or outside the film? Is there any silence? What effect does this have?

6. Story

What information can we glean at the beginning of the film? What dilemmas or complications do the main characters face? How do they overcome them? How does the story end? Does it remind you of anything you’ve experienced, seen or read?

DVD Extras

Reccomended resources for using film to teach visual literacy…

If you find the thought of using film in the classroom daunting, the British Film Institute publishes a very useful resource called Look Again. This guide to using film with primary children is available as a free download from the BFI website (bfi.org.uk) or can be purchased from its online store. The BFI has also produced four DVDs - two for KS1 and two for KS2 - with accompanying teachers’ guides. Each DVD contains a selection of relatively short films that can be used to develop children’s literacy and media literacy skills.

Another useful source of materials and guidance is Film Education. There are various films and supporting teaching notes available to download from its website (filmeducation.org).