The topic this week was:

Materials evaluation: What would be your top tips for effectively evaluating materials for language teaching? What do you look for? What do you avoid? What influences your decisions in using or not using a given material?

 

photo taken by @Marisa_C

When we evaluate materials, we inevitably ask lots of questions. Turns out discussion of materials evaluation generates a great quantity of questions too. Here is the summary I have cobbled together from a particularly challenging transcript:

(NB: I have filled in the missing letters from all tweets containing abbreviations, just to make reading easier!)

@Marisa_C helpfully defined materials for us: “Materials = coursebook, supplementary, own design, downloads, anything” and everybody jumped in with criteria they consider important. Further criteria arose throughout the discussion, but for convenience and clarity, they are all gathered together here:

Needs to provide good mix of skills and be interactive. Lots of heads-up activities. (@MarjorieRosenbe)

Also important is clear structure and engaging topics. (@MarjorieRosenbe)

Level should be appropriate. (@MarjorieRosenbe)

Materials able to be connected to students’ intrerests/background/culture..? (@TomTesol)

We usually look at  whether suited to specific learning situation  + offer valid methodology in relation to course aims. (@AlexandraKouk)

I think materials need to allow us to communicate well with students – that creates rapport. (@MarjorieRosenbe)

For me, the point of a book is to help me present new input to ss — I have to do the rest (communicative stuff) (@TomTesol)

For me, an important criterion is how memorable is the topic/.content going to be so that language can stay ‘glued’ to it. (@Marisa_C)

How does the material lend itself or be adapted) to natural,meaningful,relevant communication? (@CotterHUE)

I don’t think a book should tell you what, but rather present a selection of things to choose from. (@teflgeek)

F=fun R=rapport I=ideology N=needs D=design. …Sorry missed the E=education (as in principles of) (@Marisa_C)

When I look at a page of material I see if I would be interested myself – then I decide. And I ask sts what they think too. (@MarjorieRosenbe)

I like materials which show you something interesting about the world and help learn/practise language/skills (@robertmclarty)

Systematically:  Does it fit age, level, syllabus criteria… (@teflgeek)

I also feel materials need to appeal to variety of learner types…Learner types can be sensory perception (VAK) but also global-analytic cognitive processing types. (@MarjorieRosenbe)

Another issue to consider with mats is that subject matter might interest you, but does it interest sts? (@pjgallantary)

Assess supplementary audio: is it an EIL approach with non-natives holding conversations? Or native English speakers? (@CotterHUE)

Done and dusted? I think not. Materials evaluation is a complex business.

I posed the question “how do you identify the criteria and which are more/less important also?” and @pjgallantary supplied a useful answer: “course books are where we all start – knowing how they work helps us understand what to look for in materials”  Of course, being able to evaluate effectively isn’t the whole story – there are institutional constraints to take into consideration too. @Shaunwilden reminded us that course books  ”are establishment enforced more often than a choice by teacher and students” and @teflgeek told us about a group evaluation process in which nobody agreed, and the resolution? “There were three of us and the DoS got the casting vote”.  Meanwhile, @TomTesol reminded us that materials evaluation is not just about selection prior to the beginning of a course but a continuing process involving “constantly reviewing, getting students’ and faculty feedback…”

The discussion meandered naturally into the question of materials adaptation, which is a common follow-on to evaluation and identification of shortcomings. Why do we adapt the all-singing, all-dancing glorious multi-colour materials on the market these days?

The following reasons emerged:

Books written for a specific demographic with set format from publisher…which doesn’t match your students (@CotterHUE)

Problems with delivery but mostly missing keys and audioscripts which meant I had to copy them for students (@MarjorieRosenbe)

Other reason was that book on ICT was really outdated. Or book for BEC prep didn’t deal with exam (@MarjorieRosenbe)

Main issue with coursebooks is their homogeneity – T needs to be able to make relevant to own students (@pjgallantary)

Ultimately NO book is ever going to be right for your class because it wasn’t designed specifically for your class. (@teflgeek)

The point in the book is that you may HAVE to use it in which case try to salvage what you can & improve (@Marisa_C)

I constantly adapt and update materials based on surprises, feedback, etc. design new materials too. Assessment important (@CotterHUE)

And how do we go about it?

To lessonize: first, look at relevance of content  i.e. what u want it for –  to teach language point, vocab., skills etc (@AlexandraKouk)

As to developing own materials, As ever we must start from sts needs -WHY are we using this text, this video, etc, then HOW (@pjgallantary)

I create materials for PEO using articles – lots of things you can do – vocab, discussion, grammar etc. (@MarjorieRosenbe)

I adapt to suit the SKILLS I want them to develop (@TomTesol)

As Marisa implied, if you can’t figure out a way to adapt materials so the inpurt will stick, your evaluation is finished: They stink. (@TomTesol)

The question of the role of the Teacher’s Book within the evaluation process, and ultimately teaching, was touched on a few times throughout the discussion and opinons were varied:

Never really use the teachers book, except for answers. Looking at teachers book means I’m unresponsive to the class.(@CotterHUE)

Depends. Came across word I didn’t know in ESP book, now have printed teachers notes.(@MarjorieRosenbe)

Most are now online and often very long. 120 pages or so of pdfs to print out.But lots of info. (@Marjorie Rosenbe)

Teacher’s Book Important to most ‘non-natives’ I’ve worked with. (@TomTesol)

Or inexperienced ‘natives’. Or experienced ‘natives’ looking for new ideas or something to bounce off maybe. (@LizziePinard)

For me not very [important] but if I am choosing for a group of teachers it is something I look at closely. (@Shaunwilden)

Well, depending on the market and availability of training – sometimes that’s all a Teacher can get – a good Teacher’s guide. (@Marisa_C)

A few curveballs were thrown near the end – broadening the scope of “materials” but there wasn’t enough time to go into this in any depth as the hour was fast drawing to a close and next thing we knew everyone was being invited to contribute their final five minute words of wisdom. This is what emerged amongst fielding of curveballs:

Constant assessment of materials, be it website, publisher, etc. What works with your style and students? What doesnt work? (@CotterHUE)

I think to evaluate effectively you need more awareness of your own beliefs/principles etc and good awareness of context etc (@LizziePinard)

Test drive stuff before landing yourself with a CB for a year of pain! Use good placement to match students to level especially important first (@oyajimbo)

Finally, here are the links that were thrown up throughout the discussion:

Very old blog post on choosing a CB or materials (@Marisa_C)

Here’s more recent research with v. useful checklist on p.6 (@AlexandraKouk)

Subject matter might interest you but does it interest sts? Wrote about that (@CotterHUE)

A useful research paper  (@AlexandraKouk)

Another useful research paper (@AlexandraKouk)

Have a look here and add sth if you can – been collecting interesting texts/topics for developing lessons. (@Marisa_C)

Pecha Kucha with mnemonic for evaluation (@Marisa_C)

This is a MATERIALS mnemonic from Tanner and Green (back in the day) (borrowed) (@TomTesol)

Phew! That finally brings me to the end of this summary. Thanks all for a great discussion. And, if you have any criteria you want to add to the list, anything you want to add, agree, or indeed disagree with, feel free to do so in the comments section. Nobody will object to the discussion continuing, I am sure!

Thank you to all who participated. 

Postscript:

If you want an overview of all the literature out there on materials development and have access/can wangle access to journal articles, @HeatherBu2011 recommends the following:

“State-of-the-Art Article: Materials development for language learning and teaching” by Brian Tomlinson in Language Teaching (2012), 45.2, 143–179

Thanks to Lizzie Pinard for this summary.