Dan Ariely: 7 Problems with Research Groups and Surveys
It’s an oldie but it’s a goodie – an interview by Nex Gen Research with psychologist and behavioural economist Dan Ariey, author of Predictably Irrational (speed summary) and expert on non-rational (non value-maximising) consumer behaviour.
In the interview Dan suggests that market researchers are irrational – and predictably so – in their over reliance on traditional research groups and surveys that generate biased findings. Specifically he identifies 7 problems with research group and survey methodology.
- Traditional survey or research groups that pay people for their opinions biases both who participates and what they say; the desire for cash and to please drives this bias
- Research groups and surveys that solicit opinions from those without expert or extended experience generate stereotypical results that are biased by ignorance – «it’s not useful to ask [these] people their opinion»
- Research groups an surveys that over reach from «what» questions to asking «why» questions create interpretive bias; the research wrongly assumes people have some privileged insight into why they do what they do (and they don’t)
- Survey research (and worse research groups used as mini-surveys of people’s opinions) is but a quick, cheap and inferior substitute for a real experiment or field test that sacrifices validity, reliability and predictive power.
- Focus group verbatims that pepper presentations can provide a dangerous, false, and erroneous sense of confidence in a business direction or decision. It is quite irrational to base business decisions on the uninformed, non-expert opinions of consumer groups ‘It sounds crazy but organizations do it all the time’
- Traditional research methods such as research groups and surveys create an artificial environment that bias results because they are so very different to the natural real-world decision environments that strongly influence our thoughts and behaviours
- Surveys used to segment people into groups create artificial differences – we are all much more similar than we think. Good research should look for similarities not differences between people