Our latest blog post is by Penny Ur, who discusses the need for materials writers to pay attention to research on language learning.
I’ve just read Katherine Bilsborough’s blog entry ‘ELT materials writing: More on emerging principles’. I really enjoyed it and learned from it. If you haven’t read it yet, do so! I’d like to pick up and take further one interesting point she makes:
a (surprisingly low) number of theorists … have drawn on research findings from ELT teaching and learning. [my italics].
Yes, surprisingly low. And it’s not just the theorists. It’s the writers themselves.
Materials writers, like teachers, don’t read the research very much. Teachers don’t read because they simply don’t have time: their job description in most cases (there are exceptions, but lamentably few) doesn’t include time for any kind of teacher development, including reading the research literature. Materials writers don’t have that excuse: they organise their own time, and one would expect that, as a matter of professionalism, they would make sure they are up to date with reliable research findings that are directly relevant to the content or design of materials writing, and that they use this research to inform their own work. But very few do so.
OK, admittedly there’s a lot of unreliable ‘research’ out there, lots of studies that are inconclusive, and others that contradict one another. We still have no clear answers to questions like ‘Does task-based instruction work, or doesn’t it?’, or ‘Is it better to teach grammar proactively through traditional “formS-focus” or through reactive “form-focus”?’ This is partly because, of course, language teaching is a very complex construct, and it’s difficult or impossible to design research that neutralises all other factors in order to draw conclusions about the one the researcher has chosen to study. It’s also very context-dependent: something that works well in one context may be completely useless or even counter-productive in another.
Having said all that, there has been some research over the last twenty years or so that has produced convincing evidence for ideas which may be unfashionable but which have certainly made me think twice about my own teaching and about the way I design my own materials. And this research is worth reading and paying attention to. Here are some examples.
Use of L1
It’s fairly clear today that it is rather silly to try to eliminate the L1 from courses in English as an Additional Language. There’s research to show that we inevitably map new words in a second language onto our L1 rather than onto their real-world referents (Jiang, 2002), and there are lots of studies on how L1 use can enhance learning. (I won’t even try to list them all here, but write to me if you’re interested in references.) Even in a multilingual class, students can – and should – be encouraged to insert translations to clarify meanings in their vocabulary notebooks, to note L1 glosses on their reading texts if they feel it helps, or to help classmates with the same L1 to understand new language. And in materials designed for a monolingual class, there are lots of ways of using the mother tongue, apart from instructions and explanations. You can use it to present the meaning of new vocabulary, for example, or as a quick and easy way of checking that learners have got the gist of a reading or listening text, or to test a grammatical feature by asking students to translate an L1 sentence into an English one that uses it.
And while I’m on the subject, what happened to translation? Just because it was done to death in old grammar-translation methods doesn’t mean that it should be banned in modern language teaching. If you’re writing materials for a particular language community, then occasional translation exercises, in either direction, are often motivating, learning-rich, and a nice change from the traditional gap-fills, true/false, and so on.
Lexical sets
Lexical sets are groups of words that all mean the same sort of thing and are the same part of speech; examples are colours, animals or parts of the body. They’re typically used in beginner-level books as a basis for new vocabulary. But there is research to show that teaching lexical sets is counter-productive: learners actually learn new items much better if they are disconnected, or connected thematically, i.e. they are likely to co-occur in a given context (e.g. home, love, mother, family, together). The original research was done a while ago (Tinkham, 1993) but has been replicated a number of times more recently, every time giving the same results (again, write to me if you want references). Beginner coursebooks, however, continue to use lexical sets as a basis for vocabulary teaching: mainly, in my opinion, because these are convenient to compile and easy to make up exercises or design graphics for – in other words, beneficial for the materials writer rather than for the learner.
Another problem with the use of lexical sets, incidentally, is that their use as a basis for vocabulary in the early levels leads to the teaching of relatively rare items because they happen to be in a lexical set. The colour purple for example, is relatively rarely used, as is toe. They take up time and space that could be used for teaching more useful and frequent words and multi-word items.
Guessing from context
Guessing from context is a thoroughly unreliable way of accessing meaning. Again, I’ll give one reference (Nassaji, 2003), and if you want more, ask me. This is mainly a feature of teaching rather than materials design, but it’s something that materials designers need to be aware of. It’s an important strategy for learners to use when reading on their own, but it is not a good way to find out the meaning of a new word used in context. More than 50 per cent of the time learners guess wrong, misleading themselves and sometimes others. This is not just because they don’t know the language well enough: it is more often because a natural context simply does not usually provide enough clues to the meaning of the word.
Supposing I give you a paragraph from Katherine’s blog:
I first considered the concept of principles for ELT materials writers when I attended a presentation by Jill Hadfield at IATEFL 2014 called ‘Do writers have principles?’ Before Jill’s talk I knew plenty about principles of language learning and teaching, but I’d never really thought about materials writers having principles. In _________ that’s a daft supposition; after all, principles from language learning theory find their way into our materials in all kinds of ways. It’s just that most materials writers don’t stop to think about them, perhaps because we are far too busy writing!
Could you guess the missing word? Guess, and then check back to see if you were right. Even if you were right, how sure were you?
If you’re reading for information, then it doesn’t really matter if you got it wrong; you can get the gist of the message from the surrounding text. But if you want to learn the meaning of the target word, then there’s a very real problem: inferencing takes a lot longer than just being told the answer, and guessing the wrong meaning may actually interfere with successful learning.
Bottom line: don’t include guessing-from-context tasks in materials if the aim is to access meanings of new words.
In conclusion, then, keep an eye out for research that is relevant to materials design, and set aside time to read. A good place to start is the overview articles from Cambridge University Press’s quarterly Language Teaching: they give excellent summaries of state-of-the-art research on different areas of language teaching and save you trawling through the periodicals to find particular studies.
References
Jiang, N. (2002). Form-meaning mapping in vocabulary acquisition in a second language. Studies in Second Language Acquisition, 24(4), 617–637.
Nassaji, H. (2003). L2 vocabulary learning from context: Strategies, knowledge sources and their relationship with success in L2 lexical inferencing. TESOL Quarterly, 37(4), 645–670.
Tinkham, T. 1993. The effect of semantic clustering on the learning of L2 vocabulary. System, 21(3): 371–380.
Penny Ur has 35 years’ experience as an English teacher in elementary, middle and high schools in Israel. She has also taught BA and MA courses at Oranim Academic College of Education and Haifa University. She has presented papers at IATEFL and other English teachers’ conferences worldwide, and in 2013 was awarded the OBE for services to English language teaching. Penny has written a number of handbooks for teachers, published by Cambridge University Press, the most recent being Penny Ur’s 100 Teaching Tips (2016). She has written or co-written English coursebooks for elementary and middle-school classes in Israel.
Can you recall and share with us a piece of research that you came across that made a difference to your materials-writing or teaching?
The MaWSIG blog features guest posts by members – please get in touch if you would like to write for us.
Excellent blog, Penny. Very useful.
I completely agree with you, Penny, and I do try my best to keep up with research, at least, in the areas where I do the most work. The biggest hurdle though, I think, for most materials writers is access. The majority of academic journal articles are behind paywalls or published in incredibly expensive academic books and if you’re not attached to a university, then you can’t easily access them. Yes, there are occasional write-ups and summaries in more accessible sources (such as the fabulous ELT Research Bites blog), but they often don’t give you the level of detail you need to properly assess the validity of the research. Not being plugged into that academic circle, it can often feel like you’re only getting the selective snippets that leak out, rather than the whole picture.
Absolutely, Julie, that’s the main problem: accessibility. However, more and more articles are available for downloading free via Google Scholar. And even if you can’t access the full article, you can always read the abstracts which give you an idea. Even then, of course, there’s such a huge amount out there, that it’s mission impossible to reach it all. I have a lot of admiration for writers like yourself who do try to keep up, as much as possible. There are quite a lot who, unfortunately, don’t bother.
Penny, I agree. There is so much research out there, and even if we (teachers / writers) could access it all, it’s very easy to become totally overwhelmed with it all!
I suppose this is why many (if not most) teachers rely on teaching guides and books written for teachers which intend to ‘digest’ all the research findings and pull out the most convincing findings. Still, as the discussion with Geoff shows, deciding what the main and most convincing findings are may be somewhat subjective, and so teachers/writers/anyone who reads those books have to depend on the author having been objective and read as much of the literature on that area as possible, since they’re rarely in a position to ‘check’ for themselves. I see this as one of the problems within ELT that needs addressing more comprehensively.
Thanks for mentioning ELT Research Bites, Julie! 🙂
You’re of course right that summaries such as those we post on that site are not the best way to access research, but I’m proud to be part of an initiative that tries to help people ‘peek’ behind the paywalls!
There’s also Research Gate where some authors share full texts of their work, though this is obviously limited by any copyright agreements they may have entered into.
Still, I also want to mention action research. I think there’s a conceptual gap between teachers / writers and researchers, sadly. Some of the most helpful and relevant research for a teacher wanting to develop materials for a class will be action research conducted within that class! Teachers and writers can be researchers, too! This is a different side of ‘research’ that many people, especially teachers, often forget about!
My first reply got mangled. Please delete and use this instead.
You have repeatedly given your own views on TBLT (“there’s no evidence that it works”) and the usefulness of teaching grammar proactively through traditional focus on formS (“it’s effective”), without adequately discussing the evidence from research findings that challenge such opinions (see, for example, Long 2015).
In this article, you mention 2 areas where research can inform ELT while ignoring the elephant in the room, i.e., the 60 years of research findings on interlanguage development. This research (see Han and Tarone, 2017 for a review) poses a serious challenge to the use of materials such as coursebooks, which chop the target language into bits, and then present and practice the bits in a pre-determined sequence on the assumption that learners learn what they’re taught it this way.
Pienemann’s ( e.g. 1987) work showed that all the children and adult learners of German as a second language in a very big study adhered to a five-stage developmental sequence. Later work by his group and others in the 1990s established an acquisition order for morphemes, negation, questions, word order, embedded clauses and pronouns (see Han and Tarone, 2017, for a review). The conclusion from the research findings is that there are various kinds of developmental sequences and stages in interlanguage development which are impervious to instruction, in the sense that stage order can’t be altered, or stages skipped: acquisition sequences do not reflect instructional sequences, and thus teachability is constrained by learnability.
The implication is that a lot of the materials you recommend, including coursebooks that implement a grammar-based syllaubus based on a PPP methodology, fly in the face of robust findings in SLA research.
References
Han, Z and Tarone, E. (eds.) (2017) Interlanguage Forty years later. Amsterdam, Benjamins.
Long, M. (2015) SLA and Task-based Language Teaching. Oxford, Wiley.
Pienemann, M. (1989). Is language teachable? Psycholinguistic experiments and hypotheses. Applied Linguistics, 10, 52-79.
Thanks for your challenging response, Geoff! I’ll try to respond!
I don’t think I did, actually, in my piece, advocate coursebooks based on a grammatical syllabus? All I said was that the research on grammar teaching or about TBLT is inconclusive. You produced references against explicit grammar teaching and for TBLT: these could easily be countered with evidence such as that produced by Norris and Ortega (2002) in the first case or arguments put forward by Michael Swan (2006) in the second. And a lot of doubt has been cast on the practical implications for teaching of the Pienemann’s teachability hypothesis: see for example Spada and Lightbown, 1999. But my point in this case was not that materials should or should not be grammar based or that TBLT is or is not a good idea: but simply that we have no conclusive proof either way, and a lot of conflicting evidence. On the other hand where we DO have substantial and reliable evidence to support a conclusion that affects materials writing, and we have access to it, I think we have a moral obligation to take it into account in our own composition.
Norris, J. M. & Ortega, L.. (2001). Does type of instruction make a difference? Substantive findings from a meta-analytic review. Language Learning, 51, Supplement 1, 157-213.
Spada, N. & P. M. Lightbown. (1999). Instruction, first language influence, and developmental readiness in second language acquisition. Modern Language Journal, 83 (1), 1-22.
Swan, M. (2005). Legislation by hypothesis: the case of task-based instruction. Applied Linguistics, 26(3), 376-401.
Dear Penny,
Thanks for your reply. I wasn’t referring only to your piece here, but rather to what you’ve said in recent conference talks and in your book “A Course in Language Teaching”. If we take all these into account, I think it’s fair to say that you have criticised, and indeed, dismissed, TBLT without properly discussing different versions of it, and commended courseboooks which implement a grammar-based syllabus through PPP, without properly discussing the evidence from research findings. My general point is that while you accept the role of mediator between academics who carry out emprical research into (instructed) SLA and teachers, you use this role to argue for a very partisan view of ELT, which is often at odds with research findings.
The works I cited were in support of findings in interlanguage development, and all four of the academics you cite – Spada, Lightbown, Norris and Ortega – support the consensus view among scholars of SLA that instruction can’t affect the route of interlanguage development. They also support the commonly-held view that basing ELT on the presenting and practice of pre-selected formal elements of the grammar in a pre-determined order, a methodology which you recommend, flies in the face of robust research findings. It’s surely your duty to discuss these matters with the teachers you council and to explain why you disagree with these views.
You cite the work of Norris and Ortega (2002) as evidence of the value of explicit grammar teaching. Nowhere do these scholars recommend the kind of presentation and practice of successive bits of grammar as you do in your book “A Course in Language Teaching”.
You cite the work of Swan against TBLT. Nowhere does Swan deal with Long’s particular form of TBLT as described in his 2015 book.
You say “a lot of doubt has been cast on the practical implications for teaching of the Pienemann’s teachability hypothesis: see for example Spada and Lightbown, 1999”. One practical implication of Pienemann’s teachability hypothesis has already been mentioned: teaching should respect the learners’ own internal syllabus, and this is an implication that Spada and Lightbown accept. Pienemann’s hypothesis doesn’t ihave clear implications for how to teach, but it does have very clear implications for how not to. You choose to ignore these implications when you encourage teachers to carry on using oursebooks.
Of course we don’t have conclusive proof about the efficacy of grammar-based materials or TBLT. But we do have a great deal of evidence to suggest that you misguide teachers when you tell them that using coursebooks and other materials to support a gramar-based PPP methodology is a perfectly fine way to go about ELT. On the one hand you insist on the need for ELT teachers to be more critical and to pay more attention to research findings, while on the other hand, you don’t deal critically with research findings that flag up the false assumptions on which your own appraoch to ELT are based.
Well said Geoff. Once attending myself one of Penny’s talks, I can assure you that this contradictory training to ELT educators in different countries has been happening for years. I still recall Penny’s answer to my naive question about the lack of evidence on TBLT to improve learning…”I will come back to that later”… meaning shut up and listen. That’s how I felt. Unfortunately.
Jose – thanks for this message. I wish more members of audiences would come back with feedback like this to make us speakers sit up and pay attention to what we are (or are not) saying.
If I said ‘I will come back to that later’, then that is what, at the time, I really meant. The article that I intended to mention later in my talk (but apparently, if you remember rightly, did not) about evidence (or lack of) for the effectiveness of TBLT was Swan, M. (2005). Legislation by hypothesis: the case of task-based instruction. Applied Linguistics, 26(3), 376-401. I don’t know which talk you are referring to so cannot check the ppt; but If I indeed did not talk about this article, then your conclusion as to the subtext of ‘I will come back to that later’ is understandable, and I can only apologize.
Here’s a reply from Mike Long on my post about Ur’s criticisms of TBLT
Penny Ur, and others demand lots of evidence that TBLT “works”, while providing none that traditional grammar-based PPP does. And if some of the evidence is from laboratory studies, they dismiss it as ivory tower, etc., etc., etc.
Here’s the abstract from a forthcoming statistical meta-analysis of 52 evaluations of program-level implementations of TBLT in real classroom settings, including parts of the Middle-East and East Asia, where arm-chair pundits have decided it could never work for “cultural” reasons, and “three-hours-a-week” primary and secondary foreign language settings, where they have also decided it could never work:
“Task-based language teaching (TBLT) is an empirically investigated pedagogy that has garnered attention from language programs across the globe. TBLT provides an alternative to traditional grammar translation or present-practice-produce pedagogies by emphasizing interaction during authentic tasks. Despite several previous meta-analyses investigating the effect of individual tasks or short-term task-based treatments on second language (L2) development, no studies to date have synthesized the effects of long-term implementation of TBLT in authentic language
classrooms. The present study uses meta-analytic techniques to investigate the effectiveness of TBLT programs on L2 learning. Findings based on a sample of 52 studies revealed an overall positive and strong effect (d = 0.93) for TBLT implementation on a variety of learning outcomes.
The study further examined a range of programmatic and methodological features that moderated these main-effects (program region, institution type, needs analysis, and cycles of implementation). Additionally, synthesizing across both quantitative and qualitative data, results also showed positive stakeholder perceptions towards TBLT programs. The study concludes with implications for the domain of TBLT implementation, language program evaluation, and future research in this domain.”
The full article is already published in the On-line First section of the LTR web-site:
Bryfonski, L., & McKay, T. H. (2017). TBLT implementation and evaluation: A meta-analysis. Language Teaching Research
Of course, it’s only 52 studies, some with methodological weaknesses
While I absolutely agree with everything here, I would like to defend the use of lexical sets. One problem learners have, especially younger learners, is organising their notes in such a way that they can easily refer back to what they have learnt in the classroom. This is essential in terms of revision and taking control of their own learning.
If we only teach vocabulary on a thematic basis, it can be difficult for students to remember where a lexical item was first encountered. For example, if we teach the word ‘together’ in an otherwise disconnected group of words, it’s likely that many students will have to flick back through the book for ages to find it again. However, if they want to find the word ‘purple’ they will know that it will be in the lexical set relating to colours. Thus the use of lexical sets empowers the students.
A similar argument exists in various blogs to the effect that a lesson bank is a better way of teaching than a coursebook. But giving younger learners a pile of print-outs and photocopies means it’s very hard for them to backtrack and find the things that they have learnt when exam time comes around at the end of the year.
Thanks for this thought, Alastair. I see that having lists in a vocabulary notebook organized into lexical sets will make it easier to refer back. But there are a few problems with this.
For one, a lot of important vocabulary simply doesn’t belong to a lexical set of items conveniently taught together. Look at the vocabulary items in the above short paragraph I’ve just written: how many of them will conveniently fit into lexical sets for teaching? for organizing in a notebook? Another problem is that ease of reference later may be a consideration, but surely ease and effectiveness of learning in the first place is more important?
Perhaps a compromise could be to suggest to students that they organize their vocabulary notebook into general topic headings, and then as they learn new vocabulary, note it down in these sections so they can find them easily later?
As an ELT video producer I am constantly looking for research, especially to do with the use of video for low level learners but it is hard to come by. If anyone knows of reliable research in the area I’d be very interested to read.
I am a little bemused by your bad tempered, disrespectful approach to the exchange of ideas, Geoff Jordan. While some of your points may indeed be valid and worthy of debate, I don’t think you’re much interested in commenting on Penny Ur’s piece on the importance of materials writers being research-aware – the topic here.
You seem much more interested in attacking her for her views on Task Based Learning and for her views on the use of coursebooks that appear to follow a grammar-based syllabus. My own experience, Geoff, is that the vast majority of English teachers in the world don’t work in private language schools with small groups of motivated students and enthusiastic colleagues (with CELTAs and DELTA’s.) They are state school teachers, language or philology graduates, speak English as an L2, put up with poor working conditions – big classrooms, full timetables, hours of admin and stress to the eyeballs. For this reason they love coursebooks, love bite-sized grammar chunks – they are under obligation to test 3 times a semester – and they loathe Task Based Learning almost as much as they loathe pompous academics telling them that they should embrace it and that much of what they do is wrong ( because it is based on false assumptions?)
We need to understand teachers first, before we beat them around the head with the latest theory, don’t you think?
Hi Catherine,
I don’t tell teachers what to do, and I certainly don’t beat them around the head with the latest – or any – theory. I dedicate just a bit of my time taking leading members of the ELT establshment to task for writing books on how to teach English and giving PD teacher training courses which ignore research findings and misguide teachers by telling them that using coursebooks and other materials to support a gramar-based PPP methodology is a perfectly fine way to go about ELT.
Your only defence of coursebook-driven ELT is that it’s convenient. It’s based on false assumptions? It flies in the face of robust research findings? Never mind. The critic is a bad tempered, disrespectful, loathsome, pompous academic, so there.
Well, you certainly come across as bad tempered Geoff, in this forum! And too quick to point the finger. Loathsome is taking things a bit far, however.
I feel that there are ways of debating and exchanging ideas that are respectful, irrespective of how right you’re convinced that you are.
I don’t think, by the way, that you have much of an idea of what I feel regarding coursebooks (I did not offer a defence of coursebooks but rather a defence of teachers who use and want to use coursebooks), and would add that you have even less of an idea of what I feel about the role (and absence) of research in ELT.
My mistake, was to get rattled by your remarks; I should have let them speak for themselves. At least I won’t take the bait this time.
Thank you for Penny!
I guessed that the missing word was ‘retrospect’ How sure was I that that was the missing word? I grant that I wouldn’t have put money on its being ‘retrospect’ specifically, rather than ‘hindsight’, say, but I was very confident about the meaning of the missing word.
Thanks for a really interesting post, Penny. I’d also echo the value of visiting http://www.eltresearchbites.com . You can also find a selection of Cambridge Research Papers (summaries of research on particular themes, written in an accessible style) on the CUP site: http://www.cambridge.org/elt/blog/campaign_page/cambridge-papers-elt/
(though I’m a bit hesitant to mention a publisher in a thread that Geoff Jordan is commenting on!)