Hej. Så fyren her tror at han kan fortælle dig om fremtiden. Hans navn er Nostradamus, selvom The Sun får ham til at ligne Sean Connery.
Hi. So, this chap here, he thinks he can tell you the future. His name is Nostradamus, although here the Sun have made him look a little bit like Sean Connery. (Laughter)
Og ligesom flertallet af jer, tror jeg, så tror jeg ikke på at folk kan se ind i fremtiden. Jeg tror ikke på forudsigelser og hver gang I hører at nogen har kunne forudsige noget som er sket i fremtiden, så har det sandsynligvis været et lykketræf, og vi hører kun om lykketræf og særtilfælde. Vi hører ikke om alle de gange hvor folk tager fejl. Nu forventer vi at den slags sker med fjollede historier om forudsigelser, men problemet er at vi har præcist det samme problem i den akademiske verden og i lægevidenskaben og i dette miljø koster det liv.
And like most of you, I suspect, I don't really believe that people can see into the future. I don't believe in precognition, and every now and then, you hear that somebody has been able to predict something that happened in the future, and that's probably because it was a fluke, and we only hear about the flukes and about the freaks. We don't hear about all the times that people got stuff wrong. Now we expect that to happen with silly stories about precognition, but the problem is, we have exactly the same problem in academia and in medicine, and in this environment, it costs lives.
Tænk først på forudsigelser, og som det viser sig, så udførte en forsker kaldet Daryl Bem sidste år et stykke forskning, hvor han fandt evidens for synske evner hos studerende og fik det publiceret i et peer-reviewed akademisk tidsskrift og de fleste som læste det sagde bare: "Okay, godt nok jeg tror det er et lykketræf, et særsyn, fordi jeg ved at hvis jeg udførte det studie, ville jeg ikke finde nogen evidens for at studerende har synske evner og det vil sandsynligvis ikke blive publiceret i noget tidsskrift. Og faktisk, vi ved at det er rigtigt, fordi adskillelige grupper af forskere forsøgte at gentage resultaterne af dette forudsigelses-forsøg, og da de sendte det ind til samme tidsskrift, svarede tidsskriftet: "Nej tak, vi er ikke interesseret i at udgive gentagelser. Vi er ikke interesseret i negative data." Så her har vi allerede bevis for hvordan den akademiske litteratur giver en skævvredet billede af hvilke videnskabelige studier der er blevet udført.
So firstly, thinking just about precognition, as it turns out, just last year a researcher called Daryl Bem conducted a piece of research where he found evidence of precognitive powers in undergraduate students, and this was published in a peer-reviewed academic journal and most of the people who read this just said, "Okay, well, fair enough, but I think that's a fluke, that's a freak, because I know that if I did a study where I found no evidence that undergraduate students had precognitive powers, it probably wouldn't get published in a journal. And in fact, we know that that's true, because several different groups of research scientists tried to replicate the findings of this precognition study, and when they submitted it to the exact same journal, the journal said, "No, we're not interested in publishing replication. We're not interested in your negative data." So this is already evidence of how, in the academic literature, we will see a biased sample of the true picture of all of the scientific studies that have been conducted.
Det sker ikke kun indenfor den støvede akademiske psykologi. Det sker også indenfor, for eksempel, kræftforskning. Så i marts 20012, for bare en måned siden, rapporterede nogle forskere i tidsskriftet Nature at de havde forsøgt at gentage 53 forskellige basale videnskabelige forsøg indenfor mulige behandlingsmuligheder for kræft. Og ud af disse 53 forsøg, kunne de kun gentage seks forsøg med succes. I Syvogfyrre tilfælde af disse 53 forsøg kunne resultatet ikke genskabes. Og de sagde i deres diskussion at årsagen muligvis var at kun særtilfælde bliver publiceret. Folk laver masser af forskellige forsøg, og i de tilfælde hvor de virker, vil det blive publiceret og dem som ikke virker, vil ikke blive det. Og deres første anbefaling for at fikse dette problem, fordi det er et problem fordi det sender os alle ned ad blinde veje, deres første anbefaling til at løse dette problem, er at gøre det nemmere at publicere negative resultater indenfor videnskab, og ændre incitamenterne, så videnskabsfolk er opfordret til at indsende flere af deres negative resultater til offentligheden.
But it doesn't just happen in the dry academic field of psychology. It also happens in, for example, cancer research. So in March, 2012, just one month ago, some researchers reported in the journal Nature how they had tried to replicate 53 different basic science studies looking at potential treatment targets in cancer, and out of those 53 studies, they were only able to successfully replicate six. Forty-seven out of those 53 were unreplicable. And they say in their discussion that this is very likely because freaks get published. People will do lots and lots and lots of different studies, and the occasions when it works they will publish, and the ones where it doesn't work they won't. And their first recommendation of how to fix this problem, because it is a problem, because it sends us all down blind alleys, their first recommendation of how to fix this problem is to make it easier to publish negative results in science, and to change the incentives so that scientists are encouraged to post more of their negative results in public.
Men det sker ikke kun i den meget tørre verden af førklinisk basal videnskabelig kræftforskning. Det sker også i virkeligheden, i kød og blod indenfor lægevidenskaben. Så i 1980 udførte nogle forskere et studie om en medicin kaldet lorcainide, som var et middel mod hjerteflimmer, et lægemiddel som undertrykker unormale hjerterytmer, og ideen var at folk som havde haft et hjerteanfald, de har ofte også unormale hjerterytme, så hvis man gav dem et middel som dæmpede unormal hjerterytmer, så ville det forøge deres chancer for at overleve. Tidligt i deres udvikling, lavede de små forsøg, med under hundrede patienter. Halvtres patienter fik lorcainide, og af disse døde 10. Andre 50 patienter fik en placebo sukkerpille uden nogen aktive stoffer og kun en af dem døde. Så de betragtede korrekt dette lægemiddel som en fiasko, og det kommercielle udvikling blev stoppet og fordi den kommercielle udvikling stoppede, blev deres forsøg aldrig publiceret.
But it doesn't just happen in the very dry world of preclinical basic science cancer research. It also happens in the very real, flesh and blood of academic medicine. So in 1980, some researchers did a study on a drug called lorcainide, and this was an anti-arrhythmic drug, a drug that suppresses abnormal heart rhythms, and the idea was, after people have had a heart attack, they're quite likely to have abnormal heart rhythms, so if we give them a drug that suppresses abnormal heart rhythms, this will increase the chances of them surviving. Early on its development, they did a very small trial, just under a hundred patients. Fifty patients got lorcainide, and of those patients, 10 died. Another 50 patients got a dummy placebo sugar pill with no active ingredient, and only one of them died. So they rightly regarded this drug as a failure, and its commercial development was stopped, and because its commercial development was stopped, this trial was never published.
Uheldigvis, i løbet af de næste fem, 10 år fik andre selskaber den samme ide om et lægemiddel mod hjerteflimmer til patienter der havde haft et hjerteanfald. Disse lægemidler kom på markedet. De blev udskrevet bredt fordi hjerteanfald er en meget almindelig ting, og det tog lang tid at finde ud af at disse midler også forøgede dødsraten og før vi opdagede det, døde over 100,000 folk unødvendigt i Amerika på grund af udskrivning af medicin mod hjerteflimmer.
Unfortunately, over the course of the next five, 10 years, other companies had the same idea about drugs that would prevent arrhythmias in people who have had heart attacks. These drugs were brought to market. They were prescribed very widely because heart attacks are a very common thing, and it took so long for us to find out that these drugs also caused an increased rate of death that before we detected that safety signal, over 100,000 people died unnecessarily in America from the prescription of anti-arrhythmic drugs.
Nu faktisk i 1993, forskerne som lavede det oprindelige forsøg i 1980, det tidlige studie, offentliggjorde et mea culpa, en undskyldning til det videnskabelige samfund hvor de skrev: "Da vi udførte vores forsøg i 1980, troede vi at den forøgede dødsrate som skete i lorcainide-gruppen var på grund af et tilfælde" Udviklingen af lorcainide var opgivet af kommercielle årsager, og forsøget blev aldrig publiceret; Det er et godt eksempel på publiceringsbias. Det er den tekniske term for fænomenet hvor uflatterende data forsvinder, forbliver upublicerede, og de siger de resultater, der er beskrevet her "måske havde givet en tidlig varsling af problemer forude."
Now actually, in 1993, the researchers who did that 1980 study, that early study, published a mea culpa, an apology to the scientific community, in which they said, "When we carried out our study in 1980, we thought that the increased death rate that occurred in the lorcainide group was an effect of chance." The development of lorcainide was abandoned for commercial reasons, and this study was never published; it's now a good example of publication bias. That's the technical term for the phenomenon where unflattering data gets lost, gets unpublished, is left missing in action, and they say the results described here "might have provided an early warning of trouble ahead."
Nu er disse historier fra grundvidenskab. Disse er historier fra 20, 30 år siden. Det akademiske publiceringsmiljø er meget forskelligt nu. Der er akademiske tidsskrifter som "Trials" open access tidskrift, som vil offentliggøre ethvert forsøg på mennesker uanset om den har et positivt eller et negativt resultat. Men problemet med negative resultater, som ikke bliver udgivet er stadig meget udbredt. Det er faktisk så fremherskende at det går til kernen af evidensbaseret medicin. Så dette er en medicin, der kaldes reboxetine, og dette er et middel at jeg selv har udskrevet. Det er et antidepressiv. Og jeg er en meget nørdet læge, så jeg læste alle undersøgelserne at jeg kunne om dette middel. Jeg har læst den ene undersøgelse, der blev offentliggjort der viste at reboxetine var bedre end placebo, og jeg har læst de andre tre undersøgelser, der blev offentliggjort der viste at reboxetine var lige så godt som alle andre antidepressiver, og fordi denne patient ikke haft det godt med de andre antidepressiver, Jeg troede at hvis reboxetine er lige så godt, er det et forsøg værd. Men det viste sig, at jeg blev vildledt. Faktisk syv forsøg hvor reboxetine blev sammenlignet med en placebo sukker pille. Et af dem var positivt, og blev offentliggjort, men seks af dem var negative og de blev aldrig udgivet. Tre forsøg blev offentliggjort, som sammenlignede reboxetine med andre antidepressiver, hvor reboxetine var lige så godt, og de blev offentliggjort. Men tre data med gange så mange patienter blev indsamlet som viste at reboxetine var værre end disse andre behandlinger, og disse forsøg blev ikke offentliggjort. Jeg følte mig vildledt.
Now these are stories from basic science. These are stories from 20, 30 years ago. The academic publishing environment is very different now. There are academic journals like "Trials," the open access journal, which will publish any trial conducted in humans regardless of whether it has a positive or a negative result. But this problem of negative results that go missing in action is still very prevalent. In fact it's so prevalent that it cuts to the core of evidence-based medicine. So this is a drug called reboxetine, and this is a drug that I myself have prescribed. It's an antidepressant. And I'm a very nerdy doctor, so I read all of the studies that I could on this drug. I read the one study that was published that showed that reboxetine was better than placebo, and I read the other three studies that were published that showed that reboxetine was just as good as any other antidepressant, and because this patient hadn't done well on those other antidepressants, I thought, well, reboxetine is just as good. It's one to try. But it turned out that I was misled. In fact, seven trials were conducted comparing reboxetine against a dummy placebo sugar pill. One of them was positive and that was published, but six of them were negative and they were left unpublished. Three trials were published comparing reboxetine against other antidepressants in which reboxetine was just as good, and they were published, but three times as many patients' worth of data was collected which showed that reboxetine was worse than those other treatments, and those trials were not published. I felt misled.
Nu du kan sige, godt, det er et yderst usædvanligt eksempel og jeg ønsker ikke at være skyldig i det samme slags udvælgelse og selektive referencer som jeg anklager andre folk for. Men det viser sig, at fænomenet publikationsbias er faktisk blevet meget, meget godt undersøgt. Så er her et eksempel på hvordan du undersøger det. Den klassiske model er, at du får du en masse undersøgelser hvor du ved, at de har gennemført og afsluttet det, og derefter ser om de har været udgivet det i den akademiske litteratur. Så tog her tog man alle forsøgene der nogensinde havde været gennemført på antidepressiver, som er godkendt i en 15-årig periode af FDA. De tog alle de forsøg, som blev indsendt til FDA som en del af godkendelsespakken. Så det er ikke alle de forsøg, der nogensinde blev gennemført på disse medikamenter fordi vi kan aldrig vide, hvis vi har dem alle, men det er dem, der blev gennemført for at få en markedsføringstilladelse. Og derefter undersøgte man om disse forsøg var blevet offentliggjort i peer-reviewed akademisk litteratur. Og resultatet var at det var omtrent 50-50. Halvdelen af disse forsøg var positive, og halvdelen af dem var negativ, i virkeligheden. Men når søgte efter efter disse forsøg i den peer-reviewed akademisk litteratur, fandt de et helt andet billede. Kun tre af de negative forsøg blev offentliggjort, Men alle undtagen én af de positive forsøg blev offentliggjort. Nu, hvis vi bare bladre frem og tilbage mellem disse to, Du kan se hvilken rystende forskel der var mellem virkeligheden og hvad læger, patienter, sundhedsvæsnet og akademikere kunne se i peer-reviewed akademisk litteratur. Vi blev vildledt, og dette er en systematisk fejl i kernen af lægevidenskaben.
Now you might say, well, that's an extremely unusual example, and I wouldn't want to be guilty of the same kind of cherry-picking and selective referencing that I'm accusing other people of. But it turns out that this phenomenon of publication bias has actually been very, very well studied. So here is one example of how you approach it. The classic model is, you get a bunch of studies where you know that they've been conducted and completed, and then you go and see if they've been published anywhere in the academic literature. So this took all of the trials that had ever been conducted on antidepressants that were approved over a 15-year period by the FDA. They took all of the trials which were submitted to the FDA as part of the approval package. So that's not all of the trials that were ever conducted on these drugs, because we can never know if we have those, but it is the ones that were conducted in order to get the marketing authorization. And then they went to see if these trials had been published in the peer-reviewed academic literature. And this is what they found. It was pretty much a 50-50 split. Half of these trials were positive, half of them were negative, in reality. But when they went to look for these trials in the peer-reviewed academic literature, what they found was a very different picture. Only three of the negative trials were published, but all but one of the positive trials were published. Now if we just flick back and forth between those two, you can see what a staggering difference there was between reality and what doctors, patients, commissioners of health services, and academics were able to see in the peer-reviewed academic literature. We were misled, and this is a systematic flaw in the core of medicine.
Faktisk har der været så mange undersøgelser på publikationsbias nu, over hundrede, at de har været samlet i en systematisk gennemgang, offentliggjort i 2010, der tog hver enkelt undersøgelse af publikationsbias at de kunne finde. Publikationsbias påvirker hvert felt af lægevidenskaben. Ca. halvdelen af alle forsøg, i gennemsnit, forbliver upublliceret, og vi ved, at positive resultater har omkring dobbelt så stor chance for at offentliggøres end negative resultater.
In fact, there have been so many studies conducted on publication bias now, over a hundred, that they've been collected in a systematic review, published in 2010, that took every single study on publication bias that they could find. Publication bias affects every field of medicine. About half of all trials, on average, go missing in action, and we know that positive findings are around twice as likely to be published as negative findings.
Dette er en kræftknude i kernen af evidensbaseret medicin. Hvis jeg slog plat eller krone 100 gange men derefter tilbageholdt resultaterne fra halvdelen af disse kast, kunne jeg få det til at se ud som om jeg havde en mønt, der altid viste plat. Men det ville betyde, at jeg havde en tokronet mønt. Det ville betyde, at jeg var en svindler og du var en idiot for lade mig slippe af sted med det. (Latter) Men det er lige præcis hvad vi blindt tolererer i hele den evidensbaserede lægevidenskab. Og for mig, er det et forskningsbedrag. Hvis jeg gennemførte en undersøgelse og jeg tilbageholdt halvdelen af datapunkter fra én undersøgelse, ville I med rette beskylde mig for uredelig forskning. Og alligevel af en eller anden grund, hvis nogen udfører 10 undersøgelser, men kun udgiver de fem, som giver det resultat, at de ønsker, mener vi ikke, at det er uredelighed. Og når dette ansvar er spredt mellem hele netværket af forskere, akademikere, industrisponsorer, tidsskriftsredaktører, finder vi det af en aller anden grund mere acceptabelt, men effekten på patienterne er ødelæggende.
This is a cancer at the core of evidence-based medicine. If I flipped a coin 100 times but then withheld the results from you from half of those tosses, I could make it look as if I had a coin that always came up heads. But that wouldn't mean that I had a two-headed coin. That would mean that I was a chancer and you were an idiot for letting me get away with it. (Laughter) But this is exactly what we blindly tolerate in the whole of evidence-based medicine. And to me, this is research misconduct. If I conducted one study and I withheld half of the data points from that one study, you would rightly accuse me, essentially, of research fraud. And yet, for some reason, if somebody conducts 10 studies but only publishes the five that give the result that they want, we don't consider that to be research misconduct. And when that responsibility is diffused between a whole network of researchers, academics, industry sponsors, journal editors, for some reason we find it more acceptable, but the effect on patients is damning.
Og det sker lige nu, i dag. Dette er et medikament, der kaldes Tamiflu. Tamiflu er en medicin som regeringerne rundt om i verden har brugt milliarder og milliarder af dollars på lagre, og vi har oplagret Tamiflu i panik, i troen på, at det vil reducere graden af komplikationer af influenza. Komplikationer er en medicinsk eufemisme for lungebetændelse og død. (Latter) Nu når Cochranes systematiske reviewere forsøgte at indsamle alle data fra alle forsøgene, som der nogensinde er blevet gennemført, om hvorvidt Tamiflu faktisk gjorde dette eller ikke, fandt de , at flere af disse forsøg var upublicerede. Resultaterne var ikke tilgængelig for dem. Og når de begyndte at få referater af disse forsøg gennem forskellige forskellige kilder, gennem Freedom of Information Act anmodninger, gennem pression på forskellige forskellige organisationer, var det de fandt inkonsistent. Og da de forsøgte at få fat i rapporter vedrørende kliniske undersøgelser 10,000-siders lange dokumenter, der har den bedste muligt gengivelse af resultaterne, blev de fortalt, de ikke kunne få lov at se dem. Og hvis du vil læse den fulde korrespondance og undskyldninger og forklaringer fra medicinal virksomheden Du kan læse det i denne uges udgave af PLOS Medicine.
And this is happening right now, today. This is a drug called Tamiflu. Tamiflu is a drug which governments around the world have spent billions and billions of dollars on stockpiling, and we've stockpiled Tamiflu in panic, in the belief that it will reduce the rate of complications of influenza. Complications is a medical euphemism for pneumonia and death. (Laughter) Now when the Cochrane systematic reviewers were trying to collect together all of the data from all of the trials that had ever been conducted on whether Tamiflu actually did this or not, they found that several of those trials were unpublished. The results were unavailable to them. And when they started obtaining the writeups of those trials through various different means, through Freedom of Information Act requests, through harassing various different organizations, what they found was inconsistent. And when they tried to get a hold of the clinical study reports, the 10,000-page long documents that have the best possible rendition of the information, they were told they weren't allowed to have them. And if you want to read the full correspondence and the excuses and the explanations given by the drug company, you can see that written up in this week's edition of PLOS Medicine.
Og den mest forbløffende ting af alt dette, for mig, er, at ikke alene er dette et problem, som vi ved er et problem, men vi har må lide under snydeløsninger. Folk har foregivet, at dette er et problem, der er blevet løst. For det første havde vi forsøgsregistre, og alle sagde, Åh, er det okay. Vi får alle til at registrere deres forsøg, de vil sende resultaterne, de vil fortælle hvad de vil undersøge før de gør det, og så bagefter kan vi checke og se, hvis alle forsøgene, som er gennemført og afsluttet er blevet offentliggjort. Men folk gider ikke at bruge disse registre. Og så derefter kom den internationale sammenslutning af medicinske tidsskriftsredaktører, og de sagde, Åh, godt, vi vil holde linjen. Vi vil ikke offentliggøre nogen artikler, vi vil ikke offentliggøre eventuelle forsøg medmindre de har været registreret før de begyndte. Men de holde ikke linjen. I 2008 blev der gennemført en undersøgelse som viste, at halvdelen af alle forsøg offentliggjort af tidsskrifter redigeret af medlemmer af ICMJE ikke blev registreret korrekt, og en fjerdedel af dem ikke var registreret overhovedet. Og derefter endelig kom FDA med et krav, for et par år siden, som kræver at alle der udfører et forsøg, skal udgive resultaterne af dette forsøg inden for et år. Og i BMJ, i den første udgave af januar, 2012 kan du se en undersøgelse, som ser på, hvorvidt folk overholdt denne regel, og det viser sig, at kun én ud af fem har gjort det.
And the most staggering thing of all of this, to me, is that not only is this a problem, not only do we recognize that this is a problem, but we've had to suffer fake fixes. We've had people pretend that this is a problem that's been fixed. First of all, we had trials registers, and everybody said, oh, it's okay. We'll get everyone to register their trials, they'll post the protocol, they'll say what they're going to do before they do it, and then afterwards we'll be able to check and see if all the trials which have been conducted and completed have been published. But people didn't bother to use those registers. And so then the International Committee of Medical Journal Editors came along, and they said, oh, well, we will hold the line. We won't publish any journals, we won't publish any trials, unless they've been registered before they began. But they didn't hold the line. In 2008, a study was conducted which showed that half of all of trials published by journals edited by members of the ICMJE weren't properly registered, and a quarter of them weren't registered at all. And then finally, the FDA Amendment Act was passed a couple of years ago saying that everybody who conducts a trial must post the results of that trial within one year. And in the BMJ, in the first edition of January, 2012, you can see a study which looks to see if people kept to that ruling, and it turns out that only one in five have done so.
Dette er en katastrofe. Vi kan ikke vide sande virkningerne af lægemidler, som vi udskriver, hvis vi ikke har adgang til alle oplysninger.
This is a disaster. We cannot know the true effects of the medicines that we prescribe if we do not have access to all of the information.
Og det er ikke et svært problem at løse. Vi har brug at tvinge folk til at udgive alle forsøg udført på mennesker, herunder de ældre forsøg fordi FDA kræver kun, at du udgiver de forsøg, der er gennemført efter 2008, og jeg ved ikke, hvad det er, som vi gør, hvis vi kun praktiserer lægevidenskab på grundlag af forsøg, som fuldført i de seneste to år. Vi har brug at udgive alle forsøg på mennesker, herunder de ældre forsøg for al medicin som bruges, og du skal fortælle alle du kender at dette er et problem, og at det er ikke blevet rettet. Mange tak. (Bifald) (Bifald)
And this is not a difficult problem to fix. We need to force people to publish all trials conducted in humans, including the older trials, because the FDA Amendment Act only asks that you publish the trials conducted after 2008, and I don't know what world it is in which we're only practicing medicine on the basis of trials that completed in the past two years. We need to publish all trials in humans, including the older trials, for all drugs in current use, and you need to tell everyone you know that this is a problem and that it has not been fixed. Thank you very much. (Applause) (Applause)