The Qualitative Debate

The Qualitative-Quantitative Debate

There has probably been more energy expended on debating the differences between and relative advantages of qualitative and quantitative methods than almost any other methodological topic in social research. The “qualitative-quantitative debate” as it is sometimes called is one of those hot-button issues that almost invariably will trigger an intense debate in the hotel bar at any social research convention. I’ve seen friends and colleagues degenerate into academic enemies faster than you can say “last call.”

After years of being involved in such verbal brawling, as an observer and direct participant, the only conclusion I’ve been able to reach is that this debate is “much ado about nothing.” To say that one or the other approach is “better” is, in my view, simply a trivializing of what is a far more complex topic than a dichotomous choice can settle. Both quantitative and qualitative research rest on rich and varied traditions that come from multiple disciplines and both have been employed to address almost any research topic you can think of. In fact, in almost every applied social research project I believe there is value in consciously combining both qualitative and quantitative methods in what is referred to as a “mixed methods” approach.

I find it useful when thinking about this debate to distinguish between the general assumptions involved in undertaking a research project (qualitative, quantitative or mixed) and the data that are collected. At the level of the data, I believe that there is little difference between the qualitative and the quantitative. But at the level of the assumptions that are made, the differences can be profound and irreconcilable (which is why there’s so much fighting that goes on).

Qualitative and Quantitative Data

It may seem odd that I would argue that there is little difference between qualitative and quantitative data. After all, qualitative data typically consists of words while quantitative data consists of numbers. Aren’t these fundamentally different? I don’t think so, for the following reasons:

All qualitative data can be coded quantitatively.

What I mean here is very simple. Anything that is qualitative can be assigned meaningful numerical values. These values can then be manipulated to help us achieve greater insight into the meaning of the data and to help us examine specific hypotheses. Let’s consider a simple example. Many surveys have one or more short open-ended questions that ask the respondent to supply text responses. The simplest example is probably the “Please add any additional comments” question that is often tacked onto a short survey. The immediate responses are text-based and qualitative. But we can always (and usually will) perform some type of simple classification of the text responses. We might sort the responses into simple categories, for instance. Often, we’ll give each category a short label that represents the theme in the response.

What we don’t often recognize is that even the simple act of categorizing can be viewed as a quantitative one as well. For instance, let’s say that we develop five themes that each respondent could express in their open-ended response. Assume that we have ten respondents. We could easily set up a simple coding table like the one in the figure below to represent the coding of the ten responses into the five themes.

This is a simple qualitative thematic coding analysis. But, we can represent exactly the same information quantitatively as in the following table:

Notice that this is the exact same data. The first would probably be called a qualitative coding while the second is clearly quantitative. The quantitative coding gives us additional useful information and makes it possible to do analyses that we couldn’t do with the qualitative coding. For instance, from just the table above we can say that Theme 4 was the most frequently mentioned and that all respondents touched on two or three of the themes. But we can do even more. For instance, we could look at the similarities among the themes based on which respondents addressed them. How? Well, why don’t we do a simple correlation matrix for the table above. Here’s the result:

The analysis shows that Themes 2 and 3 are strongly negatively correlated – People who said Theme 2 seldom said Theme 3 and vice versa (check it for yourself). We can also look at the similarity among respondents as shown below:

We can see immediately that Persons 1 and 3 are perfectly correlated (r = +1.0) as are Persons 4 and 8. There are also a few perfect opposites (r = -1.0) – P1 and P9, P2 and P5, and P3 and P9.

We could do much more. If we had more respondents (and we often would with a survey), we could do some simple multivariate analyses. For instance, we could draw a similarity “map” of the respondents based on their intercorrelations. The map would have one dot per respondent and respondents with more similar responses would cluster closer together.

The point is that the line between qualitative and quantitative is less distinct than we sometimes imagine. All qualitative data can be quantitatively coded in an almost infinite varieties of ways. This doesn’t detract from the qualitative information. We can still do any kinds of judgmental syntheses or analyses we want. But recognizing the similarities between qualitative and quantitative information opens up new possibilities for interpretation that might otherwise go unutilized.

Now to the other side of the coin…

All quantitative data is based on qualitative judgment.

Numbers in and of themselves can’t be interpreted without understanding the assumptions which underlie them. Take, for example, a simple 1-to-5 rating variable:

Here, the respondent answered 2=Disagree. What does this mean? How do we interpret the value “2” here? We can’t really understand this quantitative value unless we dig into some of the judgments and assumptions that underlie it:

• Did the respondent understand the term “capital punishment”?
• Did the respondent understand that a “2” means that they are disagreeing with the statement?
• Does the respondent have any idea about alternatives to capital punishment (otherwise how can they judge what’s “best”)?
• Did the respondent read carefully enough to determine that the statement was limited only to convicted murderers (for instance, rapists were not included)?
• Does the respondent care or were they just circling anything arbitrarily?
• How was this question presented in the context of the survey (e.g., did the questions immediately before this one bias the response in any way)?
• Was the respondent mentally alert (especially if this is late in a long survey or the respondent had other things going on earlier in the day)?
• What was the setting for the survey (e.g., lighting, noise and other distractions)?
• Was the survey anonymous? Was it confidential?
• In the respondent’s mind, is the difference between a “1” and a “2” the same as between a “2” and a “3” (i.e., is this an interval scale?)?

We could go on and on, but my point should be clear. All numerical information involves numerous judgments about what the number means.

The bottom line here is that quantitative and qualitative data are, at some level, virtually inseparable. Neither exists in a vacuum or can be considered totally devoid of the other. To ask which is “better” or more “valid” or has greater “verisimilitude” or whatever ignores the intimate connection between them. To do good research we need to use both the qualitative and the quantitative.

Qualitative and Quantitative Assumptions

To say that qualitative and quantitative data are similar only tells half the story. After all, the intense academic wrangling of the qualitative-quantitative debate must have some basis in reality. My sense is that there are some fundamental differences, but that they lie primarily at the level of assumptions about research (epistemological and ontological assumptions) rather than at the level of the data.

First, let’s do away with the most common myths about the differences between qualitative and quantitative research. Many people believe the following:

• Quantitative research is confirmatory and deductive in nature.
• Qualitative research is exploratory and inductive in nature.

I think that while there’s a shred of truth in each of these statements, they are not exactly correct. In general, a lot of quantitative research tends to be confirmatory and deductive. But there’s lots of quantitative research that can be classified as exploratory as well. And while much qualitative research does tend to be exploratory, it can also be used to confirm very specific deductive hypotheses. The problem I have with these kinds of statements is that they don’t acknowledge the richness of both traditions. They don’t recognize that both qualitative and quantitative research can be used to address almost any kind of research question.

So, if the difference between qualitative and quantitative is not along the exploratory-confirmatory or inductive-deductive dimensions, then where is it?

My belief is that the heart of the quantitative-qualitative debate is philosophical, not methodological. Many qualitative researchers operate under different epistemological assumptions from quantitative researchers. For instance, many qualitative researchers believe that the best way to understand any phenomenon is to view it in its context. They see all quantification as limited in nature, looking only at one small portion of a reality that cannot be split or unitized without losing the importance of the whole phenomenon. For some qualitative researchers, the best way to understand what’s going on is to become immersed in it. Move into the culture or organization you are studying and experience what it is like to be a part of it. Be flexible in your inquiry of people in context. Rather than approaching measurement with the idea of constructing a fixed instrument or set of questions, allow the questions to emerge and change as you become familiar with what you are studying. Many qualitative researchers also operate under different ontological assumptions about the world. They don’t assume that there is a single unitary reality apart from our perceptions. Since each of us experiences from our own point of view, each of us experiences a different reality. Conducting research without taking this into account violates their fundamental view of the individual. Consequently, they may be opposed to methods that attempt to aggregate across individuals on the grounds that each individual is unique. They also argue that the researcher is a unique individual and that all research is essentially biased by each researcher’s individual perceptions. There is no point in trying to establish “validity” in any external or objective sense. All that we can hope to do is interpret our view of the world as researchers.

Let me end this brief excursion into the qualitative-quantitative debate with a few personal observations. Any researcher steeped in the qualitative tradition would certainly take issue with my comments above about the similarities between quantitative and qualitative data. They would argue (with some correctness I fear) that it is not possible to separate your research assumptions from the data. Some would claim that my perspective on data is based on assumptions common to the quantitative tradition. Others would argue that it doesn’t matter if you can code data thematically or quantitatively because they wouldn’t do either – both forms of analysis impose artificial structure on the phenomena and, consequently, introduce distortions and biases. I have to admit that I would see the point in much of this criticism. In fact, I tend to see the point on both sides of the qualitative-quantitative debate.

In the end, people who consider themselves primarily qualitative or primarily quantitative tend to be almost as diverse as those from the opposing camps. There are qualitative researchers who fit comfortably into the post-positivist tradition common to much contemporary quantitative research. And there are quantitative researchers (albeit, probably fewer) who use quantitative information as the basis for exploration, recognizing the inherent limitations and complex assumptions beneath all numbers. In either camp, you’ll find intense and fundamental disagreement about both philosophical assumptions and the nature of data. And, increasingly, we find researchers who are interested in blending the two traditions, attempting to get the advantages of each. I don’t think there’s any resolution to the debate. And, I believe social research is richer for the wider variety of views and methods that the debate generates.