I once heard Carly Fiorina speak, back in 2001, when she was CEO of Hewlett-Packard.  She said something that made it immediately clear to me she was not the right person for the job.  She was touting HP’s new strategy of innovation and she boasted of her early impact on the organization:

"We will continue to invent. We are now the number three generator of patents in the world. We generate five patents a day.”

In 2001, the number of patents filed at HP more than doubled, reaching 5,000 (HP Press Release). Patents are an easy measure of a firm’s innovative capability.  And by easy, I do not mean accurate, useful, or even safe.  Just easy.  And anyone who would pick that measure has little understanding of the process of innovation. 

I was in Philadelphia last week to give a (brief) talk on the virtues of qualitative research to junior faculty in the field of management. In preparing, I rediscovered a variety of passages from various renowned researchers and thought I would share them. Apologies to all who could care less about how theories of business are made but Carly, for one, might have benefited from a better understanding.   

To begin with, qualitative research is typically juxtaposed to quantitative research and so a brief comparison is in order.  (Over)simply put, quantitative research measures things and uses statistics to find relationships between those measures. The methods for doing so allow scientists to predict relationships between changes in inputs and changes in outputs, and are widely applicable and often very useful (e.g., more carbon in the atmosphere correlates with increased global average temperature; more casual touching by the waiter correlates with more tipping by the diner). More importantly, quantitative research allows scientists to test theories about relationships between inputs and outputs across a range of similar situations.  If you had a theory about casual touching and tipping, for example, how would you test it? Across diners and/or across the waitstaff.

On the other hand, qualitative research usually does not attempt to measure the same variables across a range of situation but rather it looks for for how new variables or new relationships can be found within a single situation–variables and relationships that nobody has yet identified and studied. 

As Einstein and so many others have been credited with saying

"Not all that can be measured should be measured and not all that should be measured can be measured"

There is a great deal of value to be had in measuring our world–and a great deal of value in continually questioning the methods and results obtained by our current measures. 

Clifford Geertz, one of my particular heroes in this field, wrote a wonderful piece entitled "Thick Description" which compared the thin descriptions of measurements with the thick description of context and meaning that qualitative research can provide in any given situation.

Geertz’s example remains one of the best.  From a purely physiological perspective, a wink is the contraction of the muscles of a single eye that cause the eyelid to close.  So, of course, is a twitch.  And so is a slow-motion, exaggerated parody of a wink; a fast motion parody of a twitch; and any number of parodies of parodies of twitches and winks that a group 3rd grade boys sitting in the back row might engage in to amuse one another on a spring afternoon.

As Geertz says, "the difference between a twitch and a wink is vast." And any measure of the interactions that include and are driven by these twitches and winks is bound to measure the wrong things and fail to measure the right ones.    

And so the roles of quantitative and qualitative research are complementary. If you are studying biology or chemistry, there might be a diminished role for qualitative research–there is little in the meaning and context of plant interaction that cannot be measured relatively easily.  Or maybe not.  John Steinbeck would argue even here it is critical to continually question the value of what we’re measuring and why:

“The Mexican sierra has “XVII-15-IX” spines in the dorsal fin.  These can easily be counted. But if the sierra strikes hard on the line so that our hands are burned, if the fish sounds and nearly escapes and finally comes in over the rail, his colors pulsing and his tail beating the air, a whole new relational externality has come into being—an entity which is more than the sum of the fish plus the fisherman. The only way to count the spines of the sierra unaffected by this second relational reality is to sit in a laboratory, open an evil-smelling jar, remove the stiff colorless fish from formalin solution, count the spines, write the truth “D. XVII-15-IX.” There you have recorded a reality which cannot be assailed—probably the least important reality concerning either the fish or yourself.”

And when you’re studying people–and people in interactions–the role of qualitative research is more critical. 

Indeed, one of the first quantitative studies of people in organizations attempted to measure the effects of lighting conditions on people’s productivity.  They isolated a control group of workers and turned the lights–and productivity went up.  Then they turned the lights down–and productivity went up. They were clearly affecting, but not measuring, the right something. So the researchers talked to the workers and uncovered a relationship that is much more important the lighting levels: the Hawthorne Effect.

People were responding to the experiment–the attention, the excitement, the changes–in ways that made them more willing to work hard.  This finding led to a great deal of measurable variables–then previously ignored-about worker morale and motivation. 

This is how qualitative research can make a contribution: by identifying and describing the meanings that people have of themselves and of the situation.  These are the meanings that drive their responses to changes in their environment.  To quote Geertz again:

Man is an animal suspended in webs of significance he himself has spun, I take culture to be those webs, and the analysis of it to be therefore not an experimental science in search of law but an interpretive one in search of meaning.

Qualitative research is, at its heart, an attempt to understand how people (or fish) interpret their reality and as a result make it. Anyone who has both looked at manufacturing statistics and wandered the factory floor knows that you can learn a lot by watching and talking to the workers about their work and their lives.

And so, when you decide you want your company to be more innovative–and you decide to reward those who are "innovative"–you need to be very careful how you are measuring innovation. As the WSJ describes:

What [Carly] Fiorina doesn’t mention is why the number of patents skyrocketed. Much of it had to do with a program put in place in 1999 to get HP into the top 10 patent producers. It relied on paying engineers for each new possible filing. At the time, it was $175 for a basic "invention disclosure," $1,750 if it became a patent application, and another chunk of cash and a plaque for an actual patent.  One engineer, Shell Simpson, nearly tripled his salary by working weekends in the first year by filing 120 disclosures and 70 patent applications-at one point taking two weeks off to work on patents full-time.

So when Carly Fiorina boasted  in 2001 about the number of new patent filings at HP, she recorded a reality which cannot be assailed—probably the least important reality concerning either the engineers at HP or herself.