Law of the Instrument
Teams reach for the research method they're most comfortable with — not the one the question actually needs.
Why PMs should care
Teams reach for the method they already know. Data teams run more SQL. UX researchers schedule more interviews. Growth teams A/B test things that really needed a user interview. Analysts build dashboards for questions that are actually qualitative.
The question should decide the method, not the other way around. And most interesting product questions need more than one method, in sequence: qualitative to find the hypothesis, quantitative to size it, qualitative again to understand why the numbers came out the way they did.
Your job is often to name which method is wrong for this particular question. That's socially harder than it sounds, because it quietly says 'your tool isn't the right tool here' — and the person hearing that has been using their tool for years.
Example in product work
A team wants to know why users abandon KYC at the address step. The analytics lead runs a survey to abandoners — it goes out to 8,000 users, 160 respond (a 2% rate), and the top answer is 'other'. The readout: 'users abandon for a variety of reasons, no clear signal.'
The PM pushes for in-person sessions instead. Five users attempt the flow while talking aloud in a usability lab over an afternoon. By user three, the pattern is unmistakable: the address-lookup widget doesn't recognise UK postcodes with no space ('SW1A1AA' vs 'SW1A 1AA'). Users try twice, get a generic error, assume the app doesn't work in their area, and close it.
Cost of finding this: half a day and five coffees. Cost of the survey: three weeks and the wrong conclusion.
What to do when you see it
- Data teams run more SQL. UX researchers schedule more interviews. Each tool shapes which questions feel answerable.
- Most interesting product questions need more than one method, in sequence.
- Qualitative to find the hypothesis, quantitative to size it, qualitative again to understand why the numbers are what they are.
- A PM often has to name which method is wrong for this question — which is socially harder than it sounds.
Sources & further reading
- The Psychology of Science — Abraham Maslow, 1966The book where Maslow articulated the original 'if all you have is a hammer' observation.
- Law of the Instrument — The Decision LabA practical treatment of why over-relying on a familiar tool distorts problem-solving.