Hi. So, this chap here, he thinks he can tell you the future. His name is Nostradamus, although here the Sun have made him look a little bit like Sean Connery. (Laughter)
嗨各位! 這位仁兄呢 自認可以預知未來 他的名字是諾斯特拉達姆士,但這張圖看起來 反倒有點像西恩康納瑞
And like most of you, I suspect, I don't really believe that people can see into the future. I don't believe in precognition, and every now and then, you hear that somebody has been able to predict something that happened in the future, and that's probably because it was a fluke, and we only hear about the flukes and about the freaks. We don't hear about all the times that people got stuff wrong. Now we expect that to happen with silly stories about precognition, but the problem is, we have exactly the same problem in academia and in medicine, and in this environment, it costs lives.
我呢,跟大部分的人一樣 不太相信人類能預見未來 我不信這套,但你時常會聽到 有些人能夠預測未來 大概都是僥倖猜中的吧 我們也只能聽到這些不尋常的故事 看到人凸槌的機會卻很少 我們以為只有預知未來這種事才會出錯 但實際上, 學術界和醫藥界也有同樣的問題 出錯會賠上人命的
So firstly, thinking just about precognition, as it turns out, just last year a researcher called Daryl Bem conducted a piece of research where he found evidence of precognitive powers in undergraduate students, and this was published in a peer-reviewed academic journal and most of the people who read this just said, "Okay, well, fair enough, but I think that's a fluke, that's a freak, because I know that if I did a study where I found no evidence that undergraduate students had precognitive powers, it probably wouldn't get published in a journal. And in fact, we know that that's true, because several different groups of research scientists tried to replicate the findings of this precognition study, and when they submitted it to the exact same journal, the journal said, "No, we're not interested in publishing replication. We're not interested in your negative data." So this is already evidence of how, in the academic literature, we will see a biased sample of the true picture of all of the scientific studies that have been conducted.
現在我們先把焦點放在預知未來上 去年有位名叫Daryl Bem的研究員 做了一項研究,證明大學生 有預測未來的能力 研究結果收錄在一個同儕審查的學術期刊中 看過的人大部分表示 "好吧算他厲害,但這純粹是僥倖猜中, 因為我如果做了一個無法證明 大學生有預知能力的研究, 應該就沒有期刊會出版了" 事實也真的是如此 有數個研究團隊試圖 重復這項研究的成果 當他們把結果呈給同樣的出版社時 出版社表示,"我們對同樣的研究沒興趣, 你們的結果與先前的相反,所以我們沒興趣" 由此可見,在學術界裡 許多科學研究 都存在一種特定偏見
But it doesn't just happen in the dry academic field of psychology. It also happens in, for example, cancer research. So in March, 2012, just one month ago, some researchers reported in the journal Nature how they had tried to replicate 53 different basic science studies looking at potential treatment targets in cancer, and out of those 53 studies, they were only able to successfully replicate six. Forty-seven out of those 53 were unreplicable. And they say in their discussion that this is very likely because freaks get published. People will do lots and lots and lots of different studies, and the occasions when it works they will publish, and the ones where it doesn't work they won't. And their first recommendation of how to fix this problem, because it is a problem, because it sends us all down blind alleys, their first recommendation of how to fix this problem is to make it easier to publish negative results in science, and to change the incentives so that scientists are encouraged to post more of their negative results in public.
這種導向不只出現在心理學研究 在癌症的研究中也會出現 2012年三月,也就是一個月前 幾位研究員在自然期刊中表示 他們重新做了先前53種治療癌症的基本研究 尋找治療癌症的可行方法 53項研究中, 只有6項成功 有47項是失敗的 討論過程中他們表示,這個結果很可能是因為 不尋常的事物容易備受矚目 人們做了一大堆研究 成功的實驗結果會被公諸於世 失敗的結果不會受到注意 這些研究員對這個問題提出解決辦法 這個問題將大眾導入迷途 他們認為處理這個問題的辦法 即是使失敗的研究結果受到更多重視 改變研究動機,鼓勵科學家勇於 公布失敗的結果
But it doesn't just happen in the very dry world of preclinical basic science cancer research. It also happens in the very real, flesh and blood of academic medicine. So in 1980, some researchers did a study on a drug called lorcainide, and this was an anti-arrhythmic drug, a drug that suppresses abnormal heart rhythms, and the idea was, after people have had a heart attack, they're quite likely to have abnormal heart rhythms, so if we give them a drug that suppresses abnormal heart rhythms, this will increase the chances of them surviving. Early on its development, they did a very small trial, just under a hundred patients. Fifty patients got lorcainide, and of those patients, 10 died. Another 50 patients got a dummy placebo sugar pill with no active ingredient, and only one of them died. So they rightly regarded this drug as a failure, and its commercial development was stopped, and because its commercial development was stopped, this trial was never published.
這不只發生在臨床基本癌症研究 這不只發生在臨床基本癌症研究 這也發生在活生生的學術醫藥中 1980年幾位研究員 對一種叫lorcainide的藥物做研究 這是一種抗心律失常藥 用來抑制異常的心搏 病人心臟病發後 很有可能心律不整 若給病人能夠抑制異常心搏的藥 便能提高存活率 初期試用階段,他們對一百多名病患 做了一個試驗 五十名服用lorcainide的患者中,有十人死亡 另外五十名服用單純糖做的藥丸 只有一個人死亡 他們馬上認定這是個失敗的藥物 廣告活動也停止宣傳 因為如此,試驗結果也從未公布
Unfortunately, over the course of the next five, 10 years, other companies had the same idea about drugs that would prevent arrhythmias in people who have had heart attacks. These drugs were brought to market. They were prescribed very widely because heart attacks are a very common thing, and it took so long for us to find out that these drugs also caused an increased rate of death that before we detected that safety signal, over 100,000 people died unnecessarily in America from the prescription of anti-arrhythmic drugs.
不幸的是,五年、十年過後 其他藥廠開發了同樣能 抑制心臟病患異常心搏的藥 這些藥在市面販售,又因為心臟疾病普遍 這些藥經常變成處方藥物 我們花了很久的時間才發現 這些藥物會提高死亡率 在我們發覺這些危險前 美國有十萬人因服用抗心律失常藥而白白送命 美國有十萬人因服用抗心律失常藥而白白送命
Now actually, in 1993, the researchers who did that 1980 study, that early study, published a mea culpa, an apology to the scientific community, in which they said, "When we carried out our study in 1980, we thought that the increased death rate that occurred in the lorcainide group was an effect of chance." The development of lorcainide was abandoned for commercial reasons, and this study was never published; it's now a good example of publication bias. That's the technical term for the phenomenon where unflattering data gets lost, gets unpublished, is left missing in action, and they say the results described here "might have provided an early warning of trouble ahead."
而在1993年 1980年做過早期試驗的那批研究員 發表了一篇悔過書,向科學界致歉 他們表示 "1980年進行lorcainide試驗時, 我們以為死亡率的提升只是巧合" 我們以為死亡率的提升只是機率問題" 基於商業因素,lorcainide停止研發 這項試驗結果也從未公布 由此可見出版業的偏見 也就是不起眼的資訊常遭到封殺 不是被埋沒就是沒下文 他們表示,這個結果"原本有可能做為警惕" 他們表示,這個結果"原本有可能做為警惕"
Now these are stories from basic science. These are stories from 20, 30 years ago. The academic publishing environment is very different now. There are academic journals like "Trials," the open access journal, which will publish any trial conducted in humans regardless of whether it has a positive or a negative result. But this problem of negative results that go missing in action is still very prevalent. In fact it's so prevalent that it cuts to the core of evidence-based medicine. So this is a drug called reboxetine, and this is a drug that I myself have prescribed. It's an antidepressant. And I'm a very nerdy doctor, so I read all of the studies that I could on this drug. I read the one study that was published that showed that reboxetine was better than placebo, and I read the other three studies that were published that showed that reboxetine was just as good as any other antidepressant, and because this patient hadn't done well on those other antidepressants, I thought, well, reboxetine is just as good. It's one to try. But it turned out that I was misled. In fact, seven trials were conducted comparing reboxetine against a dummy placebo sugar pill. One of them was positive and that was published, but six of them were negative and they were left unpublished. Three trials were published comparing reboxetine against other antidepressants in which reboxetine was just as good, and they were published, but three times as many patients' worth of data was collected which showed that reboxetine was worse than those other treatments, and those trials were not published. I felt misled.
以上只是基層科學方面的故事 而且距今也二、三十年了 現今的學術出版業已截然不同 有一個叫Trials(實驗)的公開期刊 收錄各種與人體有關的實驗結果 結果是好是壞都會被收錄 但失敗的實驗結果仍普遍遭到隱埋 事實上這種情況甚至影響到 講究科學根據為重的藥物研發 這是一種叫rebozetine的藥 我曾把這種藥物開做處方,對抗憂鬱症 身為一個菜鳥醫生,我竭盡所能的查詢 與此藥有關的研究,其中一篇表示 reboxetine比安慰劑(寬心丸)有效 又有三篇表示 reboxetine和其他的抗憂鬱藥效果相當 由於其他藥對我的病人幫助不大 我就嘗試使用藥效類似的reboxetine 結果我被誤導了,事實上 在七項比較reboxetine與安慰劑的實驗中 在七項比較reboxetine與安慰劑的實驗中 只有一個的結果是正面的 其他六項負面的結果都沒被公布 有三項顯示reboxetine 與其他抗憂鬱劑效果相當的實驗結果被公布 與其他抗憂鬱劑效果相當的實驗結果被公布 但有三倍的病例顯示reboxetine的效果 不如其他治療方式 這些結果也並未被公布 我覺得被誤導了
Now you might say, well, that's an extremely unusual example, and I wouldn't want to be guilty of the same kind of cherry-picking and selective referencing that I'm accusing other people of. But it turns out that this phenomenon of publication bias has actually been very, very well studied. So here is one example of how you approach it. The classic model is, you get a bunch of studies where you know that they've been conducted and completed, and then you go and see if they've been published anywhere in the academic literature. So this took all of the trials that had ever been conducted on antidepressants that were approved over a 15-year period by the FDA. They took all of the trials which were submitted to the FDA as part of the approval package. So that's not all of the trials that were ever conducted on these drugs, because we can never know if we have those, but it is the ones that were conducted in order to get the marketing authorization. And then they went to see if these trials had been published in the peer-reviewed academic literature. And this is what they found. It was pretty much a 50-50 split. Half of these trials were positive, half of them were negative, in reality. But when they went to look for these trials in the peer-reviewed academic literature, what they found was a very different picture. Only three of the negative trials were published, but all but one of the positive trials were published. Now if we just flick back and forth between those two, you can see what a staggering difference there was between reality and what doctors, patients, commissioners of health services, and academics were able to see in the peer-reviewed academic literature. We were misled, and this is a systematic flaw in the core of medicine.
你或許認為這只是少數極端的例子 我也不想斷章取義,只挑有利的資料 我也不想斷章取義,只挑有利的資料 藉此指控他人 但後來發現其實有很多人 研究這種偏頗的出版取向 我舉個例子解釋 最常見的方式即是蒐集大量 完整的實驗結果 然後查明他們是否出現在學術文章中 現在這裡是十五年來所有 與抗憂鬱劑有關的實驗 全都經過FDA認可 他們把所有FDA有列入的試驗劃為一組 所以並不是每個試驗都有列入 因為我們無從得知他人的實驗計畫 但所選的實驗結果是為了上市而檢驗 接著我們得看看這些結果是否有公布 在同儕審查的學術文章中,這是我們發現的 一半的結果顯示有效,一半顯示無效 兩者其實差距不大 但我們在同儕審查的期刊中找到的試驗結果 卻是相當不同 只有三個負面結果被公布 正面結果中只有一個沒被公布 如果我們快速切換兩者 你會發現事實與醫生、病患、 醫療行政人員透過同儕審查的期刊 所得知的資訊有著天壤之別 所得知的資訊有著天壤之別 我們都被誤導了,在醫藥界中 這是一個深入核心的體制問題
In fact, there have been so many studies conducted on publication bias now, over a hundred, that they've been collected in a systematic review, published in 2010, that took every single study on publication bias that they could find. Publication bias affects every field of medicine. About half of all trials, on average, go missing in action, and we know that positive findings are around twice as likely to be published as negative findings.
事實上現在有很多研究 旨在研究這種出版偏向 在2010年公布的一項系統性調查中 每一個研究出版取向的研究 加起來總共超過一百多項 整個醫療界會徹底的受影響 平均多達一半的試驗都從而消失了 正面試驗結果被公布的機率 比負面結果高了近兩倍
This is a cancer at the core of evidence-based medicine. If I flipped a coin 100 times but then withheld the results from you from half of those tosses, I could make it look as if I had a coin that always came up heads. But that wouldn't mean that I had a two-headed coin. That would mean that I was a chancer and you were an idiot for letting me get away with it. (Laughter) But this is exactly what we blindly tolerate in the whole of evidence-based medicine. And to me, this is research misconduct. If I conducted one study and I withheld half of the data points from that one study, you would rightly accuse me, essentially, of research fraud. And yet, for some reason, if somebody conducts 10 studies but only publishes the five that give the result that they want, we don't consider that to be research misconduct. And when that responsibility is diffused between a whole network of researchers, academics, industry sponsors, journal editors, for some reason we find it more acceptable, but the effect on patients is damning.
在講究根據為重的醫藥界裡是一個弊端 如果我把一個硬幣拋擲一百次 但只告訴你一半的結果 我就能讓你以為每次投擲都是正面 但並不表是硬幣的兩面都是正面 這只表示我是個投機者 而你是個白癡,因為你讓我得過且過(笑聲) 但在已根據為重的醫藥界裡 我們卻盲目縱容 對我來說,這是研究上的誤導缺失 如果我進行了一個研究 並保留一半的研究結果不公布 你會義正嚴詞地指控我研究詐欺 但出於某種原因,如果有人做了十項研究 卻只公布五個他們想看的結果 我們也不會認為那是誤導研究 而當責任遍及整體研究員、學術界、 贊助人、已及期刊編輯時,基於某些原因 贊助人已及期刊編輯時,基於某些原因 我們會認為一切相當合理 但病人卻得承擔這些嚴重的後果
And this is happening right now, today. This is a drug called Tamiflu. Tamiflu is a drug which governments around the world have spent billions and billions of dollars on stockpiling, and we've stockpiled Tamiflu in panic, in the belief that it will reduce the rate of complications of influenza. Complications is a medical euphemism for pneumonia and death. (Laughter) Now when the Cochrane systematic reviewers were trying to collect together all of the data from all of the trials that had ever been conducted on whether Tamiflu actually did this or not, they found that several of those trials were unpublished. The results were unavailable to them. And when they started obtaining the writeups of those trials through various different means, through Freedom of Information Act requests, through harassing various different organizations, what they found was inconsistent. And when they tried to get a hold of the clinical study reports, the 10,000-page long documents that have the best possible rendition of the information, they were told they weren't allowed to have them. And if you want to read the full correspondence and the excuses and the explanations given by the drug company, you can see that written up in this week's edition of PLOS Medicine.
這些事情正在發生 這是一種叫Tamiflu的藥 世界各地的政府花了好幾十億資金 貯藏這個藥物 我們慌忙貯藏此藥的原因乃因 此藥被認為能夠減少流感的併發症 併發症說穿了就是肺炎 和死路一條(笑聲) 現在Cochrane的系統審查員正試圖 蒐集所有測試Tamiflu用途的實驗 蒐集所有測試Tamiflu用途的實驗 他們發現有幾個試驗並未公布 試驗結果也無從取得 當他們透過不同管道,透過資訊透明法案 並騷擾各個不同單位後 他們發現結果互相矛盾 在他們試圖取得長達一萬多頁 並付有清晰完整的臨床研究報告時 並付有清晰完整的臨床研究報告時 卻無法取得 若你想查看藥廠完整的 信件內容、藉口、和解釋的話 可以去翻翻這星期的PLOS Medicine期刊 可以去翻翻這星期的PLOS Medicine期刊
And the most staggering thing of all of this, to me, is that not only is this a problem, not only do we recognize that this is a problem, but we've had to suffer fake fixes. We've had people pretend that this is a problem that's been fixed. First of all, we had trials registers, and everybody said, oh, it's okay. We'll get everyone to register their trials, they'll post the protocol, they'll say what they're going to do before they do it, and then afterwards we'll be able to check and see if all the trials which have been conducted and completed have been published. But people didn't bother to use those registers. And so then the International Committee of Medical Journal Editors came along, and they said, oh, well, we will hold the line. We won't publish any journals, we won't publish any trials, unless they've been registered before they began. But they didn't hold the line. In 2008, a study was conducted which showed that half of all of trials published by journals edited by members of the ICMJE weren't properly registered, and a quarter of them weren't registered at all. And then finally, the FDA Amendment Act was passed a couple of years ago saying that everybody who conducts a trial must post the results of that trial within one year. And in the BMJ, in the first edition of January, 2012, you can see a study which looks to see if people kept to that ruling, and it turns out that only one in five have done so.
整件事中最令我無法置信的是 這不只是一個單純的問題,我們雖然知道 這是個問題,卻還得裝作問題已解決 我們逼人們假裝這個問題已解決 首先我們讓各方註冊試驗結果 各方會先提出實驗草案 事先告知實驗內容 之後我們就能查明 是否所有完整的實驗結果都有透明化 但人們並沒有乖乖來註冊 國際醫療期刊委員會就上前 表示他們會守住防線 並不會公布任何未經註冊的實驗結果 並不會公布任何未經註冊的實驗結果 但他們並未履行責任,2008年一項研究發現 國際醫療委員會編審的期刊中 一半以上的實驗結果 並未正式註冊,有四分之一甚至未經註冊 直到最後,幾年前FDA修正案終於通過 並規定所有實驗的結果 必須在實驗結束一年內公布 而2012年BMJ中,一月的第一期版本中 有項研究,研究人們是否依然遵循規定 結果顯示五人中只有人有做到 結果顯示五人中只有一人有做到
This is a disaster. We cannot know the true effects of the medicines that we prescribe if we do not have access to all of the information.
這是個災難 若我們無法取得所有資訊 就無從得知處方藥物的實效 就無從得知處方藥物的實效
And this is not a difficult problem to fix. We need to force people to publish all trials conducted in humans, including the older trials, because the FDA Amendment Act only asks that you publish the trials conducted after 2008, and I don't know what world it is in which we're only practicing medicine on the basis of trials that completed in the past two years. We need to publish all trials in humans, including the older trials, for all drugs in current use, and you need to tell everyone you know that this is a problem and that it has not been fixed. Thank you very much. (Applause) (Applause)
而這並不難解決 我們必須強迫人們公布 所有人體實驗,包括舊有的試驗 因為FDA修正案指要求公布2008之後的試驗 我搞不懂為何我們只能 根據兩年內的研究資料進行醫療 所有與人體有關的實驗都得公布 先前的實驗和現用的藥物也是 你必須告訴所有認識的人 這是一個尚未解決的問題 謝謝! (掌聲) (掌聲)