Research in class: Doing it and using it to improve your teaching – an #ELTChat summary (20/03/13)

Thanks to the Secret DoS for the summary.

Officially we were gathered to discuss how research in the classroom was done and how it was then used for the improvement of teaching. I’m not overly confident that we fulfilled this aim, but we didn’t wander too far off topic for the hour allotted to (by?) us. A full list of participants is included as a footnote to this summary.

The first question of substance was raised by the redoubtable @sueannan: “what do you mean by ‘doing research’?” @shaunwilden set out the contexts that often give rise to research: it might be required for formal professional development, it might be because you are writing an article, it might be for less formal CPD. @Marisa_C chimed in to point out that research could be large scale and statistical as well as small scale and classroom-based. @simon_borg (note the underscore) did not join in, but it is worth pointing out that he has highlighted in the past that teachers have cited a  misconception that research has to be large scale and statistical as a reason for not getting involved. With this in mind, @Marisa_C’s reminder that oranges were not the only fruit was welcome. It also unleashed a torrent of questions about what could be categorised as research:

• Student feedback?
• Class polls?
• Lesson transcriptions?
• Interaction patterns?
• Analysing student writing?
• Evaluating new practice?  

Although I didn’t weigh in at this stage, I will now and will say that none of these are classroom research. They are all potentially useful means of carrying out classroom research, but to be considered as research, it’s not just what you do but what you do with what you do.

I put forward my views on what research might look like fairly late into the discussion, stating that I thought that it needed to be methodical, systematic, ethical, transparent and worthwhile. I also put forward the rather contentious statement that most ELT research was execrably designed. It was a question of minutes before @shaunwilden suggested that this was clearly opinionated and not based upon anything empirical. He was quite right, of course, and it was only half-heartedly that I put up the defence that most ELT research was small scale, inconclusive, had little to no random testing and was invariably beyond replication.

There was discussion around the validity of small scale research. The first thing in its favour was that all teachers can do small scale research (@Florentina_T). Nevertheless, help is often needed in order to make the research any good (@shaunwilden). What needs to be done is to prevent the research from becoming no more than whimsical opinions tied to a literature review (@thesecretdos).  Surely, it was asked, small scale localised research was good to  see? What do you mean by “good” was the rejoinder: is it endearing? Yes. Is it admirable? Yes. Is it reliable? Hmmm…

What to research was an area that we talked around. It was clear that there was agreement that we didn’t need to be reinventing the wheel and also that we didn’t need to be looking constantly for alternatives to the wheel either. Classroom based research is often relevant only within the context in which it is carried out. The idea of copying someone else’s research in your context is to test their hypotheses in other environments. If the same sort of results emerge, it lends strength to their tentative observations. If the same sort of results don’t emerge, it highlights areas for consideration that their research did not uncover. Personally, I think this is where some of my frustration at ELT research stems from: there are thousands upon thousands of small scale research projects that seem to stand in isolation. As a result, to paraphrase @Marisa_C, most research ends up being pretty inconclusive. @louisealix68 referred to reading previous research in order to avoid repeating mistakes, but I think it might be worth reframing this ever so slightly: it is worth reading research in order to try and repeat it with fewer mistakes: the difference (to my ears at least) is to highlight that when we try to replicate somebody else’s research, our work is an extension of  theirs – not just another individual effort to try and reach understanding, but and example of how standing upon the shoulders of giants can help us to see further. The individualism of much of western research focuses us away from the collaborative discoveries and also, as was pointed out in the chat, away from the publication of research that fails: it should go without saying that we need scope to publish much more upon failure because we learn from failure more than we learn from success: when our research fails to answer our research question, we need to publish this because we have learnt what doesn’t  work: valuable information for the researchers that come after us.

We discussed the reliability of research. @shaunwilden proposed that research can be reliable if it fixes an issue or helps teachers to teach better. @Florentina_T questioned my predilection for randomised controlled trials (RCTs) and asked whether or not teachers could do them (and indeed, whether or not teachers should do them). Unfortunately, I am not yet swayed. I made the point that just wanting to do research is not enough. It is important to want to do good research. We have to be wary of the temptation to blind ourselves to flawed research because we want to be encouraging of teachers who are motivated enough to explore their practice. There is simply no point in doing research that isn’t up to scratch: the conclusions it leads to will be unsafe and it is not enough to say that it’s valid because it helps the teacher-researcher to improve their practice: we are susceptible to an impressive list of biases as I have highlighted elsewhere in my blog and I have yet to find research that suggests that the teachers were absolutely confounded by their findings. RCTs are worth doing (when possible) because they are the gold standard of scientific enquiry: so, to answer @Florentina_T: this is why they should be done and if the situation is such that they cannot be done by teachers, then we need to investigate to see what is stopping us from using them. What makes research reliable, I suggested, was research that was well-designed and replicable. At this point, I got the feeling that I was beginning to irritate some people with my grumpiness and more than one person put forward the view that I was being “a tad too esoteric”.  I will leave to one side any discussion about the fairness or otherwise of such characterisation, but will take advantage of this summary to highlight my main point which is that research which is worth doing is worth doing well. I am sure that all would agree with this (perhaps because it is an innocuously bland observation!).

So, the chat proceeded, how do teachers get into research? @louisealix68 suggested ongoing reflection and evaluation would soon enough throw up patterns that would spark curiosities. Only once a problem was identified could teachers be expected to start trying to identify ways of solving the problem. @lexicojoules made the observation that it was essential for teachers to feel interested in their research: investigating something worthy that isn’t of much interest is likely to lead to burnout. There was discussion about the use of observation checklists as a tool that could lead to research. Perhaps rather sulkily, I ventured the observation that checklists were limiting and encouraged observers to look for what was on the list (blinding them to whatever was not on the list). It was clear that I was not winning many friends with my gloomy outlook, but in my defence, this is an established criticism of such checklists.  Nevertheless, I am happy to concede @muranava’s point that “anything that helps teachers to systematize their teaching…is helpful” (with some guarded reservations about the “anything” word…). Common ground that emerged was that teachers should engage in reflective practice prior to embarking upon research: this can be done through reading, writing, wondering, observing and check listing. I am going to abuse my authorial position to argue that teachers really need to read a lot about the hows and whys of research before they embark upon it. The seminal Cohen and Manion belongs in every staffroom alongside the no less magisterial (and perhaps eminently more readable) Dornyei title.

The chat threw up some interesting avenues for those who wish to pursue this area of interest:

• Penny Ur’s wonderful talk on teachers and research: http://t.co.h3BSv6V0Wl 
• A teacher’s research toolkit: http://t.co/KnmOf3W6lm
• Simon Borg’s book on teacher research: http://t.co/vOcJhSpGMB
• Ben Goldacre’s contributions to UK Department for Education: http://t.co/9dlaptkAom and http://t.co/A0liDyPFIr

For UK-based teachers (and those who can afford to travel to the UK), it is also worth highlighting @researchEd2013 – an impressive grassroots conference for teachers who are interested in research within education: started by the admirable @tombennett71.

There was an uncontested assertion that this was an area worthy of revisiting in the future. I hope we do so.  

Participants:

@aClilToClimb, @bengoldacre, @bhrbahar, @bnleez, @evoks036, @Florentina__T, @idc74, @jobethsteel, @JoJoUppark, @lauraahaha, @lexicojules, @louisealix68, @Marisa_C, @MarjorieRosenbe, @MicaelaCarey, @muranava, @naomishema, @prese1, @pysproblem81, @Sharonzspace, @Shaunwilden, @SueAnnan, @TheSecretDoS, @UpparkHG, @web20education