My post on reporting interviews attracted some interest and critique. This one continues the theme and, I hope, ups the ante.

In a response to my post, Caroline Struthers offered an alternative example of reporting qualitative research. You can also find a form resulting from the paper here. What do I think? No problem. If someone wants to have such a list of reminders, good for them. I still don’t know why my title must be a:

concise description of the nature and topic of the study identifying the study as qualitative or indicating the approach

and cannot be, for example,

On being sane in insane places.

It’s hardly a concise description of the nature and topic of the study, identifying the study as qualitative, but is one of the key articles in 20-century psychopathology. So, I would suggest getting out more, but no, I’m not going to throw down my academic gauntlet and prepare to fight for  it. If you really think that a more creative title is such a problem, fine, have it your way, even if I actually don’t know what exactly counts as ‘concise’.

What makes the checklist useful, however, is that it operates at a very different level from the COREQ I described in the earlier post. In fact, you could just about use it for any piece of research. You know, having a title is a good thing in any research, so is having an abstract or a discussion of your research methods. I don’t think there is anything particularly qualitative in those guidelines and perhaps this is what makes the list good. Because it is good, there is really nothing to challenge it on, well, apart from some nit picking.  Is it particularly useful? Sorry, only for saying that I do things by the book. Because now we have ‘the book’.

And here comes the bit in which I hope to up the ante. Having looked at a couple of such checklists and having read and heard about a zillion qualitative papers, I have often wanted to ask the following question:

What don’t you/we disclose our interview questions?

Whether it’s books or articles, researchers make all sorts of claims based on interviews they carried out (or had them carried out) and yet, we, the readers, are none the wiser as to what exactly was asked. No, let me put it differently. We often learn about the topics or themes raised in the interviews, but almost never do we get to read the actual questions. Only PhD student are commonly made to disclose them. Poor souls. Still, they rarely reflect on them, though. After all, an interview is an interview, questions are questions, innit?

Yet, I hope it’s obvious why it is important, but I still would like to tell you anyway. So, it’s story time. When Ulrike Meinhof and I arrived on the Polish-German border, among the things we were interested in was the (experience of) cross-border movement. We visited two sets of towns (Gubin/Guben and Zgorzelec/Goerlitz) which before the WWII were entirely German (Guben and Goerlitz), but after the war the border (river Neisse) ran right through the town. And so what used to be one town, became two. And we wanted to know whether, when, why people we interviewed went….Exactly, where did they go? Did they go (to):

• Germany
• Goerlitz
• across the border
• across the river
• over the bridge
• the other side
• the German side
• over there

and probably a number of other places? The problem with them was that we actually didn’t want to ask any of them. Yes, all of them were perfectly acceptable, all of them were perfectly understandable, but every option actually told more about us, rather than the informants. This is because every option suggested a particular way in which we, the researchers, saw the world. Do we ask about movement between towns or nation states? The difference is huge. And we wanted to know what our informants would say. And in order to achieve that, we actually couldn’t ask the question. The solution was to show photographs, but that’s a different story.

And indeed, in the interviews we heard an informants say:

I quickly popped down to Germany for hair colour.

A most wonderful and bizarre sentence that is just about unsayable. No one ‘pops down to a country’, do they? Except they did. We also found out that there were differences in how generations spoke. If I remember correctly, the oldest generation often went across the river, the middle generation tended to go Germany. See the problem? Of course, you can change the place for, say, depression or the depressive disorder or something yet different.

Now, you can find guides to how to ask questions, from not asking two questions in one (surprisingly frequent in medical practice, it seems) to how to structure interviews. Also, I can tell you how I do it. For example, I structure interviews as two funnels which are joined with their tips. First questions go from large-scope to narrow-scope, then it reverses and I go with narrow-scope and end with large-scope questions (more complex, but it will do). And I actually give it very much thought not only to the scope of questions, but also to their order, and, as you might imagine, to their linguistic form. For example, I tend to use verbs, in contrast to medical literature and, in fact, medics themselves. Is it the right way? I have absolutely no idea. But I can account for it.

And the question that I asked and, so much more often I didn’t ask, is:

Can you, my fellow interview researchers, account for your  interviews? Do you have a rationale for asking your questions in the form and order in which you do?

In any case, I can continue and continue with how important interview questions are. So, let me just state the obvious again. How you ask them, when you ask them, makes the interview yield data which will (or will not) be useful or interesting. And yet, even though they are the absolutely crucial aspect of any interview-based research, ‘we’ don’t disclose them. We get away with saying that we ask our informants about something or even with less.

Moreover, although the point does depend on the research/argument, a lot of us get away without showing interview extracts without the question we asked. And I do remember being challenged on a point I was making at a conference. As I was talking about what the informant said, I was told that the form of response reflected the form of the question. And the questioner was right. If my argument was to make sense, I had to make it differently.

And we still get away with not disclosing the questions we ask! Should we disclose them? Before I answer that question, let me make a comment on checklists.

Why do we have checklists? The obvious answer is that they are an attempt to make qualitative research standardised, uniformed. Apparently, there is a wonderful and absolutely right way of doing and writing about qualitative research and we all should follow it. Bloody hell, how I reject this notion. There isn’t, there can’t be and there shouldn’t be.

I need to disclose a conflict of interest here, though. Yes, I’m afraid there is a self-interest in what I am saying. So, I think that I should publish my research in health-related journals. I want to reach out to people who are unlikely to read discourse analysis. But this means that I often hit the wall of reviewers who already ‘know best’ how to do qualitative research (IPA and TA are in vogue!). So, they don’t like DA, because it’s not IPA or thematic analysis or Foucauldian discourse analysis. I somehow don’t fit into the cool things of qualitative research.

But my own grief is not, of course, relevant here. A general point is. The more checklists we introduce, the more researchers will orient themselves to them. Indeed, the popularity of TA and IPA means that I need to change an account of my research methodology, as papers are rejected on the basis of the ‘new standard’ (no, I will not tell you what it is, not in a million years). Checklists will make it even worse. What is meant, I do fully accept it, as a helpful aide-memoire, will become a yardstick with which to beat researchers over their heads. The measure is bound to become the target (Academia is such a fruitful area in which to see Goodhart’s law).

Interestingly, and I suppose it’s the crux of this post – the checklist doesn’t offer anything that a reviewer cannot. But hey, we have a procedure and we are no longer fluffy. For there is, I think, another point to be made. Why do we, story tellers, have checklists? Isn’t story telling messy enough to escape them? Oh no, it’s not. After all, ‘we’ want to become like the number crunchers. We want to show qualitative research as research equally systematic and procedure ridden that we will do anything to pretend we can be like them. Yes, we do qualitative stuff, wink, wink, but, really, wink, wink, we are (almost) exactly like the real researchers! And soon we’ll do qualitative SPSS, too! But we’re not and we shouldn’t be. And we shouldn’t pretend.

So, what about interview questions? Should we disclose them? Yes, on balance, I think we should, be it to show that we, as groups of researchers, are able to reflect on and learn from bad research. For yes, if you think we see plenty of bad qualitative research, you ain’t seen nothing yet, I reckon. The moment we start disclosing interview questions, we will see oceans of thoughtlessness.

So, fear not, we shall never disclose our interview questions. Just like we shall continue keeping a number of others things comfortably in the shadows. Let’s do checklists – they will soon become perfect display behaviour.