When it comes to toothpaste commercials, you've probably heard claims like, nine out of 10 dentists recommend "Brighter, Whiter." Or maybe it's four out of five or 80 percent. But either way, these companies expect you to see a wall of white coats, trust their authority and think no further.
当说到牙膏的广告, 你可能会在广告里听到这样的说法: 十分之九的牙医都会推荐 使牙齿“更亮、更白”的牙膏。 或者可能是五分之四或百分之八十。 但无论哪一种,商家都希望你 看到一排白大褂牙医, 然后相信专家的权威, 别自己思考了。
Now that approach is basically BS, and you probably know it's BS, but the question is: How can you trust your BS radar?
这种方法说的就是废话, 你可能也知道这是废话。 但问题是, 你是否相信自己的“废话”雷达?
[Am I Normal? with Mona Chalabi]
【我正常吗? 和莫娜莎拉比一起】
Well, when I'm faced with suspicious statistics, I tend to ask myself three questions. First, what were people actually asked? Sometimes the question that is put to survey respondents is wildly different than the one that ends up on posters and billboards. For instance, in 2007, one toothpaste ad in the UK claimed that more than 80 percent of dentists recommended their products. What that sounds like is that a majority of dentists prefer their product over all others, that those dentists were asked if this was the best product. But when the Advertising Standards Authority looked into it, they discovered that the dentists were asked to recommend several toothpastes, not one single choice. In fact, another brand was found to be almost as popular. To no one's surprise, the ad was deemed misleading.
当我碰到这些可疑的数据, 我会问自己三个问题。 首先,被问的人究竟是谁? 有时,一些问题的问卷受访者, 与最终出现在 海报和广告牌上的人完全不同。 比如,2007 年英国的一个牙膏广告, 声称百分之八十的牙医 推荐了他们的产品。 这听起来好像, 大部分的牙医喜欢他们的产品 远胜其他产品。 就好像那些牙医被问到的问题是 这是否是最好的产品。 但是当广告标准局调查这个广告时, 他们发现牙医被问的问题是 能否推荐一些牙膏, 并不是单一选项的问题。 事实上,另外一个品牌 几乎同样受欢迎。 不出所料,该广告被认定 具有误导性。
Now, the second question to ask is: What aren’t you telling me? In the 1970s, a sugarless gum company claimed that four out five dentists recommended their product. Now, their slogan was pretty upfront about the fact that these dentists were only recommending the product to people who already chewed gum, but they weren't so forthcoming about the fifth dentist. Decades later, the manufacturer made fun of it in a new ad campaign where they blamed the fifth dentist's different thinking on a freak accident, like a sudden squirrel bite. Now, since I'm all about the deviations in the data, I decided to look into this a little bit further. In fact, it's not that the fifth dentist recommended chewing sugary gum. What they don't say is that most of them recommended that their patients don't chew gum at all.
第二个问题, 你没告诉我的内容是什么? 1970 年代,一家无糖口香糖公司 声称五位牙医里有四位 推荐了他们的产品。 现在,他们的宣传词坦然承认, 这些牙医只是向嚼口香糖的人 推荐了他们的产品。 但他们对第五位没推荐 他们产品的牙医没那么坦然。 几十年后,商家在新的广告中, 打趣指责了这第五位牙医。 说他有不同的想法是因为 一次奇怪的意外,比如被松鼠咬伤。 现在,因为我只关心数据的偏差, 所以我决定进一步研究这个事。 事实上,并不是第五个牙医 推荐有糖口香糖。 他们没说的是,大多数牙医 根本不推荐嚼口香糖。
The last thing to ask is: What was the survey context? Because there's a really big difference between saying "nine out of 10 dentists agree," and "nine out of these 10 dentists agree." Size matters and so does methodology. According to the American Dental Association, there are about 200,000 registered dentists in the US. I’m not going to bore you with the maths here, but to get a statistically significant sample of 200,000 people, you need about 400. So if you're reading in the fine print that only 50 dentists were surveyed, you know that's not statistically significant. It's just a marketing ploy.
最后一个问题, 调查的背景是怎么样的? 因为当你说: “十位牙医里九位都同意” 与“在这里的十位牙医里有九位同意” 意思完全不同。 数据样本大小很重要,方法也是。 根据美国牙医协会的数据, 美国大概有 20 万注册牙医。 这里我不想让数学让你觉得无聊, 但是要获得 20 万人有意义的统计样本, 你大概需要 400 人。 所以如果你在细则中读到, 只有 50 名牙医接受了调查。 你就知道这在统计学角度看, 数据结果并不可靠。 这只是营销策略,
So the next time you see one of these ads, ask yourself: What were people actually asked? What's gone unsaid? And what was the survey context? Hopefully, with those three questions and a little bit of skepticism, you will be able to understand when the data is legit and when it's irrelevant. Nine out of 10 Monas agree. [*No such survey took place]
所以你下次看到这些广告, 问你自己这几个问题: 被问的人到底是谁? 没有告诉的你的内容是什么? 还有调查的背景是什么? 希望有了这三个问题 和一些敢于质疑的精神, 你就会知道何时数据是可靠的, 何时数据是不相关的。 十位莫娜中有九位都赞同我。 【*我可没调查过】