How not to do qualitative research

Qualitative research is not exactly liked, especially in disciplines dominated by positivist, quantitative, ‘real’ science. We, qualitative researchers, object to representations of our research as fluffy and unscientific. And yet, I think, some of the criticisms we get are actually right.

This post started for me when I heard again about an account of qualitative suicidology from the editor of Suicide and Life-Threatening Behavior, Thomas Joiner. In an editorial, he writes that while he accepts that qualitative research is useful for hypothesis generation, the real knowledge comes from number-crunching. I find such comments misguided and lacking understanding of what qualitative research is and what it has to offer.

But to be completely honest,  I also understand the rejection of qualitative research, well, to some extent at least. Moreover, I think, ‘we’, the ‘story-tellers’, researchers who carry such research out, at partly responsible for the rejection of what we do. Over the ending academic year, I have attended more conferences and meetings at which qualitative studies were presented than usual and I have listened to more than my fair share of qualitative research (I’ve just come back from the last one). I have heard some excellent papers, but I have also heard stuff that caused toothache, let alone stuff that quite is fairly criticised by number-crunchers. Very fairly. And so, risking wrath and  scorn (and, probably, some hate, again), I have decided to write about what, for me, are some of qualitative research’s problems.

I want to stress that I do not aim to offer any ultimate guide to (bad) qualitative research, I also realise that my own research position, firmly situated in the ‘objective’ linguistic form, has a bearing on what I write. The following is then just a list of my impressions as I remember them. But I do want to tread very carefully. I do not offer any examples, in fact, I do not wish to speak of any paper/s in particular, I also do not want to take the role of the grand assessor knowing best how to do qualitative research. I don’t. I just want to respond to criticism of the kind of research I also do not by saying that it’s biased and unjustified. I prefer to say, yes, we should do better.

1. Language and thinking. All too often I hear claims that by collecting stories, researchers gain access to what informants think. No, you don’t! By collecting stories, you gain insight into what people say, also how people say things. Thinking? Not really. We must remember that any story, especially told in an interview, is subject to face concerns, power relations, shame and stigma considerations. To say that in an interview our informants simply tell us ‘how it is’, makes no sense at all. It’s also worth remembering that people can lie.

I understand, of course, that informants can construct or co-construct honesty, sincerity, they might even assure us and believe themselves that they tell us the truth. Indeed, I can imagine wonderful action-research based on such stories, yet, you still cannot exclude the possibility that some informants simply go with the flow, say what is ‘required’ of them, for example, because of social pressure. So, no, by collecting stories, you don’t get insight to what I think.

2. Influence. Similarly, just because people say that something influenced them, doesn’t mean it did (for evidence see cognitive psychology, which, no, cannot be dismissed because it’s quantitative). We, qualitative researchers, don’t do influence. We can get stories of influence.

3. Analysis. Repeating what your informants said in your own words does not constitute analysis. It really doesn’t. I sometimes despair when I see some brilliant data which are basically rephrased by the researcher. Of course, I accept that you can do a phenomenological, hermeneutic, big word, big word, analysis, but surely, it must contain more than rephrasing. Recasting the data in terms of some cultural or social categories, understanding the social positions from which informants speak – it all can be done, but it is more than simply describing what your informants said.

And here we come to ‘interpretation’. Just because you think that your informant ‘said something’, let alone meant something, or that you can read it in one way or another, it doesn’t mean that you can. All too often qualitative research ends in the researcher simply asserting something of the data. No argument, no analysis, just an assertion. And I actually would like to see some evidence beyond the researcher’s say-so for the psychological realities ascribed to informants in quite a number of IPA (Interpretative Phenomenological Analysis) studies I have seen. And the argument for what we do is not that questionnaires do not offer much of evidence, either. That’s ‘their’ problem, not ours.

4. Knowledge. In contrast to what Thomas Joiner writes, qualitative research creates knowledge. Moreover, I think it’s valuable knowledge, it uncovers (social) reality around us. But let’s reflect on what exactly we can say.

There are two kinds of studies I have in mind. One is contrasting large social practices (e.g. clinical) with the nuance of individual experience. I think qualitative research is great in showing the complexity of human experience and the difficulty of shoving it into one-fits-all frame attempting to gauge it. But it doesn’t work the other way round. And so, just because you have 10 informants who say something, doesn’t mean that you can postulate that clinical practices should be based on their stories. This is because, my 10 informants might say something different. In fact, I myself can say something different. And no, I really don’t think my experience should be used as a template for the NHS.

5. Thematic analysis. This seems to be very much the in-thing. Quite a lot of qualitative studies are anchored within thematic analysis. Done well, I think thematic analysis can be great and offer interesting insights. The problem for me is that I would like to hear not only what the theme is, but also how thematic analysts come up with it. And no, just because 4 persons agree that ‘there is a theme’ really doesn’t do it for me, no matter how many conversations you have and how structured they are.

You see, almost every paper I review now is based on thematic analysis. And when I read the extracts quoted in the papers, I always come up with different themes. It might be because I am not a psychologist or because, as a linguist, I pay attention to different things. It might be because I am a man, or perhaps it’s because I am an awkward git, I still always come up with different themes.

And this worries me. Because I really don’t know what exactly I am learning about when I look at thematic analyses. Am I learning about a few persons’ impressions, which they agreed as their valid impressions, or am I learning about the themes in what some informants said? And, alas, I tend to think that it’s the former, which, just to stress, is hugely different from the latter.

Now, for me themes are linguistic, that is to say they are related to the linguistic form of what is said. However, I accept that thematic analysis is probably dominated by a host of non-linguistic approaches to themes. But this cannot mean that the theme is not accounted for and, no, by accounting for it, I do not mean ‘emerging in interaction with text’. What do you focus on? What do you look for? What is it that you agree on? At the moment, for all I know, you agree on things because you are engaged in the same project. Even better – you might actually agree not focusing on the same aspects of the texts. And that would be quite funny.

6. Transcripts. I do not use transcriptions like Conversation Analysis. For what I do, it’s way too complex. I still do use a level of transcription which allows me to note for example speech overlaps, repairs or pauses. I think it’s important to know whether a person hesitates (do forgive this word, I realise it’s a problem) when they speak. Yet, time and again, I see quotes from interviews which look as if people were reading! People don’t speak smoothly without any hesitation, stopping, as if they were reproducing a text they had memorised earlier. In fact, it is even unlikely they would have read the text so smoothly!

I believe that if you are engaged in qualitative research on spoken texts, you cannot sanitise transcripts into this idealised speech that actually was never produced. You lose very important aspects of the communicative process.

Now, of course, I understand you may choose a very basic or a very elaborate transcription system and that it will have different functions depending on your study. Thematic analysis, for example, will probably require a very basic transcription system, IPA, on the other hand, a more complex one. Alas, mostly, I can only see this unreal smooth talk that doesn’t exist.

7. Conclusions. I am often surprised with the conclusions offered by qualitative researchers. No, you cannot have your cake and eat it. We cannot offer conclusions at the same level of generalisation as quantitative researchers. And we should not. I don’t know how to write this without examples, so I end here.

In my own conclusion, let me say that I know that I can be disagreed with. Of course I can. That’s fine, disagreement brings reflection. Obviously, I don’t think I am wrong, but I know my account can be challenged with more nuance, with different perspectives, with different ‘politics of research’. That’s all fine. So, let’s engage in a much more serious debate on what we actually do and what we, the story-tellers, can actually say.

Now, why do I do it? Well, because I think qualitative research should not only have a platform from which to criticise quantitative stuff. Qualitative research can and does have a positive agenda both to create knowledge and to change the world with it. But if this is the case, a constant critical self-reflection is crucial. Also, we, the story-tellers should not create more sticks to hit us with. Finally, just because we are continually criticised, qualitative research meetings should not become opportunities to hold hands and sing kumbaya. No, at one of the meetings I attended, I actually suggested a regular annual plenary lecture.  The “Crap Qualitative Research  of the Year” plenary.

 

 

3 Comments
  1. Thanks for the great article, very interesting remarks.

    Could you elaborate a bit more on what you mean when you write: ” at the same level of generalisation as quantitative researchers “?

    Which typ of generalization do you think of? Of course, a inference statistical generalization (sample / population) is not possible. However, to my knowledge, there are at least two types of generalization from qualitative data: 1. Theoretical sampling. If you cannot find more cases that enrich your data, you reached theoretical satturation. Some qualitative researchers argue that this can be understood as a form of generalization; 2. Generalization in form of ‘claims of existence’- it’s possible to say that a certain phenomenon exists (without saying anything about it’s distribution in terms of quantity). For some research questions this might also be a way of generalization.

    Best

    1. Dariusz Galasinski

      Thanks! What I mean is that as a qualitative researcher, I cannot go beyond the corpus of data I have. I did address it some more in the new post on how to do qualitative research!

      That said, I think you make a good point that a ‘claim of existence’ can be seen as a generalisation. I would argue, however, that a qualitative claim of existence is always tentative, it’s a suggestion. At least in my view it should be!

      As for data saturation, I must admit that I have been and I still am sceptical. I am probably not au fait with the current scholarship, but to me ‘saturation’ is more a fiat of the researcher combined with a leap of faith. I can see this, to be honest, in narrative data where certain patterns ’emerge’ very quickly and no matter how many more stories you collect, you still have the patterns you observed initially. I still don’t think that we cannot go beyond what I said above.

Comments are closed.

Loading ...