Defining ‘multimodal’

Reading the Draft Australian Curriculum for English (‘DACE’…?) I can see that confusion over the meaning of ‘multimodal’ text is about to cause English teachers some major problems.

My understanding is that when we say a text is ‘multimodal’, we mean that the audience participates in the text’s creation.  This is the definition I would say that academics and practitioners in the field of English curriculum would use; consider this explanation by Anastopoulou, Baber & Sharples:

Multimodality is based on the use of sensory modalities by which humans receive information. These modalities could be tactile, visual, auditory, etc. It also requests the use of at least two response modalities to present information (e.g. verbal, manual activity). So, for example, in a multimodal interaction a user may receive information by vision and sound and respond by voice and touch. Multimodality could be compared with ‘unimodality’, which would be based on the use of one modality only to receive or present information (e.g. watching a multimedia presentation and responding by pressing keys).

…but that’s not the definition that ACARA are going with.

The definitional confusion between terms like multimodal, multimedia and media has been around for a while, and speaks to the significant changes in what is considered core content in English brought about by the rise in visual and especially digital texts.  We are very familiar with the concept that language can be spoken, written or heard…but when it comes to texts that combine these modes, things are still a little muddled.

Please take a moment to check out, for example, the preface for the Year 7 section of the DACE (click the image below and get ready for your head to spin):

Year 7 English Content Preface

See what I mean?

In this Preface to the curriculum content descriptors multimodal texts seem to be pitted against texts that are ‘literary’ (which creates even more confusion as the definition of literary appears to change with each new use).  I can appreciate that the ACARA curriculum writers have had to avoid using the word ‘text’ because of the political beat up the term has received in recent years from certain op-ed writers in certain newspapers.  That is why this new curriculum has reverted to the more traditional term Literature – and it is because of this change that we are now supposed to say, it seems, ‘literary text’.

But now check out the etymological shenanigans that take place in the content descriptors of the Literature strand:

Year 7 - Literature

Oh brother.  The constant reference to ‘literary texts’ is supposed to be a nod to the strand content being described as ‘Literature’.  But this is ultimately VERY confusing, as ‘literary’ texts are separated from ‘non-literary’, digital’ and ‘multimodal’ texts in the Preface.  There result is that there is no sense in this strand of multimodal texts being included.

The term ‘literary’ is also conflated with ‘fiction’, and what are really language elements are referred to as literary elements.  In ‘Discussing and responding’ the term ‘text’ makes it in unscathed – which just goes to show that the word does make sense and can be used.  The term ‘text’ is highly appropriate for collectively describing all works of language art, and recognises that the works we study can be written, spoken, aural, or a combination of these.  The term ‘literary texts’ is stupidly redundant, but I’d be happy to get on with using it to placate the punters, if only it were used consistently and provided scope for the study of a broad range of texts!  Which brings me back to multimodality…

In the NSW English syllabus, students engage in what we call a range of language modes.  These are: speaking, writing, representing, listening, reading and viewing.  So ‘multimodal’ could reasonably be taken to mean ‘using more than one language mode’.  This would make film, picture books and digital stories (which use a combination of visual and written language) and many other forms of text multimodal.  OK, I can work with that.

But another thing we do in NSW English 7-12 is differentiate between the activities of composing (which involves text ‘making’ or ‘creation’, not just ‘writing’) and responding (a broader term than ‘reading’ which encompasses the ‘reception’ of all kinds of text).  These activities are viewed as always interrelated in some way, but I would say that it is only when text explicitly invites the audience to participate in the text (e.g. in video games, virtual reality, and participatory narratives such as Inanimate Alice) that the term multimodal should really be applied.  If I’m going to give up the term ‘multimodal’ to the meaning of ‘using more than one language mode’, then I’m going to need a NEW WORD that I can use when I mean ‘texts that the audience helps to construct’.

Currently this recognition of interactivity, and of the interplay between responding and composing, is severely lacking in the DACE.

[ED: Angela Thomas has helped me to clarify my thinking around this, and suggests that students could refer to the ‘cline of interactivity‘ for texts that invite participation.  My thoughts on multimodality have been developed here.  June 2010]

If you are an English teacher and haven’t yet responded to the consultation on the Draft Australian Curriculum, I implore you to log on to the ACARA site and say something about these contradictory and frankly bizarre definitions.  I can’t be the only one who feels like the curriculum writers just didn’t use a glossary!

Faced with the prospect of a shiny new curriculum that is supposed to be clarifying professional meanings and terminology for all teachers, students and parents across the nation, these definitional conflicts are something that must be sorted out before we go any further.  Agreed?

, , , ,

  1. #1 by Katy on May 13, 2010 - 10:12 am

    I was taught at university (and we were taught by the same people, so I imagine you’ve heard something similar) that multimodal refers to texts which use a range of language modes – films, for example, are the combination of the visual, the verbal, and other modes of telling. The reader participates actively in the process of interpreting these modes, just as the reader’s response shapes their reading of any text.

    Personally, I feel that the National Curriculum is based upon political correctness and expediency rather than the needs of teachers. I’m displeased with NAPLAN and I wish the Teachers’ Federation had stuck to their guns (though I don’t work in a public school). I also think that practising teachers, despite the best intentions of ACARA and others, are so busy that they find it hard to participate in the process of curriculum construction and then feel alienated from the document that is eventually produced.

    Sorry….my rant.

    • #2 by kmcg2375 on May 13, 2010 - 6:35 pm

      Actually, I didn’t learn anything about multimodality at Uni…the new literacies area of the faculty really started booming just as I graduated I think. But your definition “that multimodal refers to texts which use a range of language modes” is not actually what I thought (though this is the def used in the DACE).

      If this is the case, then very few texts would fall into the category of ‘unimodal’. Pretty much just novels. Even poetry and drama are written to be read aloud or performed. So does the term become redundant?

      I also like using the term ‘modes of production’. So Hamlet for example can be realised in several ‘modes of production’: Hamlet the playscript, Hamlet as enacted by a particular group, or Hamlet in film.

      Perhaps we should just come clean and say that the distinctions in the English curriculumare between:

      Print based texts (really the only ‘unimodality’ that we engage in – we don’t really do texts that are aural-only, and certainly are criticised for colonising visual arts when we explore something visual-only);
      Dramatic/enacted texts (theatre productions, films, speeches; words made rich through physical representation – movement, sound, lighting etc);
      Visual texts (print advertisements, picture books, visual poems etc)

  2. #3 by eva gold on May 13, 2010 - 6:32 pm

    Hi Kelli,
    Great post – and it is really important to provide an example of reading the curriculum doc in this critical way. I would suggest that your definition of multimodal would actually apply to interactive texts. Texts can be created using several modes without allowing for joint constructions.

  3. #4 by Anya on May 13, 2010 - 6:39 pm

    When I am teaching my students about this I draw from semiotic theory to define multimodal as being multiple sign systems at play – anything from a picture book (words, images, and even the interstices between) to virtual worlds (written words, images, sounds, movement, space etc…). ‘Mode’ also (in SF Linguistic theory) explicitly means the channel (written, oral, visual, gestural, spatial etc…) for the sign system of communication.
    In speaking about composing and responding to texts I usually talk about a cline of interactivity – where you can participate in a novel in certain ways (identifying with the protagonist, creating images in your head etc) but these are different to the ways you might interact when creating fan fiction, or becoming involved in a roleplay where you become part of the narrative experience.
    I think you almost have to know the authors of each part of the National Curriculum to understand the theoretical frameworks that are reflected in it. So far the draft doesn’t really come with the kind of supporting theory to explain itself that we’d like!

  4. #5 by kmcg2375 on May 13, 2010 - 7:34 pm

    Thank you so much Anya! I can see your point, and Eva’s, that the multimodal is different to interactive.

    ‘The cline of interactivity’. That is exactly the phrase I was after re the responding/composing interface.

    I wonder if the answer for the National Curriculum is as simple as saying that there must be a balance of print-based texts (they have made it clear that these should be separated to ‘ensure’ close study of written language and ‘protect’ our textual heritage) and ‘multimodal’ texts.

    The meanings across the document are unclear, depending on who authored them, but every year group has one thing in common: the separation in the Preface of ‘literary’ and ‘non-literary’ from ‘multimodal’ and ‘digital’. A change to distinguishing ‘print’ from ‘multimodal’ could set many of the other areas up to head in a better, more honest direction?

  5. #6 by Anya on May 14, 2010 - 9:06 am

    I think some clearer definitions would be very helpful. I agree, almost every text is multimodal. Cope and Kalantzis (from “The New London Group” and who have written extensively about “Multiliteracies”) make the point that although we’ve slowly made the shift from privileging the linguistic mode (written and spoken language) to encompassing a broader multimodal definition of textuality (semiotics) we’ve actually always used multimodality to express ourselves. They say:

    “Multimodal meaning is no more than the other modes of meaning working together, and much more as well.
    The ‘no more’ is based on the fact that all meaning making is in its nature multimodal.
    Multimodal meaning is no more than the other modes of meaning working together. And work together they always do.
    Linguistic meaning in the form of speaking, for instance, is achieved in combination with audio meaning (prosody) as well as gestural meaning, not to mention spatial meaning (the
    words of the lecturer compared to the conversation of two students sitting next to each other).
    And, to give another example, linguistic meaning in the form of writing is linked to visual, from the business of handwriting itself (graphology) all the way through to the heavily designed pages of desktop publishing in which fonts, point sizes, leading, kerning, bolding and italics, are all integral to the grammar of the words—and the organisation of linguistic meaning around headings, subheadings, indents, bullet points, pictures, diagrams and open spaces.
    Yet multimodal meaning is also much more than the sum of linguistic, visual, spatial, gestural and audio modes of meaning. It also involves processes of integration and moving the emphasis backwards and forwards between the various modes.”
    (Cope and Kalantzis, 2009, p.222)
    from: The International Journal of Learning
    Volume 16, Number 2, 2009, http://www.Learning-Journal.com, ISSN 1447-9494

    So…
    Multimodal = a picture book, a museum exhibition, a film, a play, a graphic novel, a piece of digital fiction like IA

    Multimodal can be printed or spatial or digital or performed etc…
    Multimodal can be literary or non-literary

    The Victorian syllabus defines a literary text as:

    literary texts: Literature, which is fundamental to the English curriculum, uses language to represent, re-create, shape and explore human experience. Literary texts can be based on fiction or fact and includes written and spoken texts. Examples include picture storybooks, traditional stories, speeches, novels, short stories, plays, poetry, translated works, non-print texts and non-fiction works such as biographies. Through reading, writing, listening to and talking about literature, students extend their understanding of the world and of themselves, and they see how cultural beliefs and values are formed.

    So…

    Again I think the main problem is that the authors wanted to emphasise that broader definition of textuality so they have tossed in as many “multimodal” “digital” “non-print” terms as they can really just to make a point, and to then be able to say “oh yes, we have accounted for multimodality and we have accounted for digital, see those words are sprinkled throughout the syllabus”. But sprinkling the words through doesn’t really set teachers up for a clear and explicit understanding about all of these terms, and I would push for a stronger theoretical preface to frame the document!

  6. #7 by Bill Boyd on May 14, 2010 - 9:15 pm

    Hi Kelli,
    I recognise absolutely what you guys are wrestling with, in terms of the terminology and the apparent inability of the new curriculum guidelines to cope with the complexities of ‘text’ in 2010. In the new curriculum guidelines in Scotland, Curriculum for Excellence, we have a new definition of ‘text’ as ‘the medium through which ideas, experiences,opinions and information can be communicated.’ This is fine, and would appear to be ‘future-proof’.
    The documentation also uses the term ‘multimodal texts’, which – and I agree with Anya on this – is used as a kind of catch-all phrase to refer to non-print texts, and allows the authors to claim to have provided an inclusive definition of texts. There is no suggestion of interactivity or indeed your concept of the audience participating in the text’s creation. This is compounded by a dogged insistence on providing separate outcomes for ‘reading’ ‘writing’ and ‘listening and talking’ as if they were separate activities rather than inseparable parts of a complex web of development. Some of those involved in the early stages of writing the lieracy guidelines (myself included) argued for a change to ‘Understanding Texts’ and ‘Creating Texts’ or something similar but it was felt – wrongly – that this would alienate too many teachers.
    I’m interested also in your discussion about ‘literature’ and whether it needs to be re-defined. This is something I have written about at length on my own blog (www.theliteracyadviser.co.uk) and no doubt will continue to do so.

  7. #8 by David Chapman on May 16, 2010 - 9:36 pm

    And then we all read the proposed “Literature” course for 11 and 12. My first thought was “only books”? I will keep reading and hope the definitions are obvious.

  8. #9 by Kate Pullinger on May 17, 2010 - 8:27 pm

    Dear Kelli –

    Thanks for the mention of ‘Inanimate Alice’ in this discussion. As a writer of both digital and print fiction – and all the differences and communalities that implies – I don’t have much to add to the debate about the term ‘multimodal’. However, in my capacity as Reader in Creative Writing and New Media at De Montfort University, I’m a co-founder of TRG – the Transliteracy Research Group. ‘Transliteracy’ is a term you might find of use in these discussions – take a look at our blog – http://www.transliteracy.com The term is gaining currency in a number of fields, including libraries and librarians. For a useful look at current talk on the subject, use the hashtag #transliteracy and search on Twitter.

    best – Kate Pullinger

  9. #10 by kmcg2375 on May 25, 2010 - 1:22 pm

    Thanks for the comment Kate, and for the link – the Transliteracy Research Group looks very generative, and I was interested by the ‘working definition’ (Transliteracy is the ability to read, write and interact across a range of platforms, tools and media from signing and orality through handwriting, print, TV, radio and film, to digital social networks).

    My ideas about what ‘English teachers’ should be teaching seem to expand and contract all the time. On one hand, we do seem to have a special project in the teaching of ‘literature’, but with the definition of what that is blurring and multimodality increasingly recognised…well, I wonder what the value is of putting a fence up around English as a school subject to shut out other forms of representation.

    Lots to think about!

  10. #12 by Imelda Judge on June 13, 2011 - 4:16 pm

    I agree its frustrating but you do well to have this discussion. It helps us all to clarify our understandings!

  11. #13 by kmcg2375 on May 13, 2010 - 6:23 pm

  1. Transmedia, transmodal, multimodal…. literary terms aren’t even close to cathing literary technology. « iTeach Inanimate Alice
  2. Thinking about ‘Transmedia’ and ‘Transliteracy’ « Kelli McGraw
  3. 2010 in review « Kelli McGraw
  4. halong bay cruise
  5. re-framing | tolerance for ambiguity

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: