Hi. So, this chap here, he thinks he can tell you the future. His name is Nostradamus, although here the Sun have made him look a little bit like Sean Connery. (Laughter)
Zdravo. Dakle, ovaj momak misli da može predvideti vašu budućnost. Zove se Nostradamus, a časopis "Sunce" ga je prikazao tako da podseća na Šona Konerija. (Smeh)
And like most of you, I suspect, I don't really believe that people can see into the future. I don't believe in precognition, and every now and then, you hear that somebody has been able to predict something that happened in the future, and that's probably because it was a fluke, and we only hear about the flukes and about the freaks. We don't hear about all the times that people got stuff wrong. Now we expect that to happen with silly stories about precognition, but the problem is, we have exactly the same problem in academia and in medicine, and in this environment, it costs lives.
Pretpostavljam da većina vas, kao i ja ne veruje da ljudi mogu videti budućnost. Ne verujem u vidovitost, ali povremeno čujete da neko može da predvidi događaje u budućnosti, to je verovatno samo slučajnost, a mi samo čujemo o slučajnostima i odstupanjima. Ne čujemo o svim onim situacijama kada su oni pogrešili. Mi očekujemo da se to dogodi u smešnim slučajevima vidovitosti, ali problem je što se sa time nosimo i u nauci i u medicini, a u tim okolnostima, plaća se životima.
So firstly, thinking just about precognition, as it turns out, just last year a researcher called Daryl Bem conducted a piece of research where he found evidence of precognitive powers in undergraduate students, and this was published in a peer-reviewed academic journal and most of the people who read this just said, "Okay, well, fair enough, but I think that's a fluke, that's a freak, because I know that if I did a study where I found no evidence that undergraduate students had precognitive powers, it probably wouldn't get published in a journal. And in fact, we know that that's true, because several different groups of research scientists tried to replicate the findings of this precognition study, and when they submitted it to the exact same journal, the journal said, "No, we're not interested in publishing replication. We're not interested in your negative data." So this is already evidence of how, in the academic literature, we will see a biased sample of the true picture of all of the scientific studies that have been conducted.
Prvo, razmišljajući samo o vidovitosti, prošle godine je istraživač Deril Bem sproveo istraživanje u kom je došao do dokaza o postojanju vidovitosti kod studenata. To je objavljeno u naučnom časopisu, a većina ljudi koja je to pročitala jednostavno je rekla: “OK, ali mislim da je to slučajnost, to je odstupanje jer znam da ako bih ja uradio studiju koja ne dokazuje postojanje vidovitosti studenata, ona verovatno ne bi bila objavljena.” U stvari, znamo da je to tačno jer je nekoliko različitih grupa istraživača pokušalo da ponovi nalaze ove studije o vidovitosti, a kada su ih predali istom tom časopisu, dobili su odgovor: “Ne, nismo zainteresovani za ponovljenu studiju. Nismo zainteresovani za vaš negativan rezultat.” Već ovo je dokaz kako u naučnoj literaturi možemo videti pogrešan uzorak stvarne slike svih naučnih studija koje su sprovedene.
But it doesn't just happen in the dry academic field of psychology. It also happens in, for example, cancer research. So in March, 2012, just one month ago, some researchers reported in the journal Nature how they had tried to replicate 53 different basic science studies looking at potential treatment targets in cancer, and out of those 53 studies, they were only able to successfully replicate six. Forty-seven out of those 53 were unreplicable. And they say in their discussion that this is very likely because freaks get published. People will do lots and lots and lots of different studies, and the occasions when it works they will publish, and the ones where it doesn't work they won't. And their first recommendation of how to fix this problem, because it is a problem, because it sends us all down blind alleys, their first recommendation of how to fix this problem is to make it easier to publish negative results in science, and to change the incentives so that scientists are encouraged to post more of their negative results in public.
Ovo se ne dešava samo u akademskom izučavanju psihologije. Na primer, ovo se takođe dešava u istraživanju karcinoma. Tako su pre samo mesec dana, u martu 2012. neki istraživači objavili u časopisu "Nature" kako su pokušali da ponove 53 različite studije bazičnih nauka koje su proučavale moguće ciljeve terapije karcinoma i od te 53 studije, mogli su uspešno da ponove šest. Četrdeset i sedam od ovih 53 nisu mogli da ponove. U diskusiji su tvrdili da je to verovatno jer se odstupanja objavljuju. Ljudi će uraditi mnogo i mnogo različitih studija, uspešni slučajevi će biti objavljeni, a neuspešni neće. Njihova prva preporuka kako da rešimo ovaj problem, a to jeste problem, jer nas sve šalje niz slepe ulice, njihova prva preporuka kako rešiti ovaj problem je da se olakša objavljivanje negativnih naučnih rezultata i da se promene motivi tako da ohrabe naučnike da objave veći broj negativnih rezultata.
But it doesn't just happen in the very dry world of preclinical basic science cancer research. It also happens in the very real, flesh and blood of academic medicine. So in 1980, some researchers did a study on a drug called lorcainide, and this was an anti-arrhythmic drug, a drug that suppresses abnormal heart rhythms, and the idea was, after people have had a heart attack, they're quite likely to have abnormal heart rhythms, so if we give them a drug that suppresses abnormal heart rhythms, this will increase the chances of them surviving. Early on its development, they did a very small trial, just under a hundred patients. Fifty patients got lorcainide, and of those patients, 10 died. Another 50 patients got a dummy placebo sugar pill with no active ingredient, and only one of them died. So they rightly regarded this drug as a failure, and its commercial development was stopped, and because its commercial development was stopped, this trial was never published.
Ovo se ne događa samo u veoma izolovanoj oblasti prekliničkih bazičnih istraživanja karcinoma. Takođe se događa u veoma stvarnoj, od krvi i mesa oblasti naučne medicine. Tako su 1980. neki istraživači proučavali lek lorkainid, koji je antiaritmik i suzbija nepravilan srčani ritam, a ideja je bila da će ljudi, nakon doživljenog srčanog udara, vrlo verovatno patiti od nepravilnog srčanog ritma, pa ako im se da lek koji sprečava poremećaj srčane ritmike, povećaće se verovatnoća njihovog preživljavanja. Na početku njegovog razvoja, uradili su veoma malu studiju, uključujući manje od sto pacijenata. Pedeset pacijenata je dobilo lorkainid i 10 je umrlo. Drugih 50 je dobilo praznu placebo zašećernu tabletu bez aktivnog sastojka i samo jedan je umro. Dakle, ispravno su zaključili da je lek neuspešan, njegov komercijalni razvoj je zaustavljen, a zbog tog zaustavljanja, ispitivanje nikad nije bilo objavljeno.
Unfortunately, over the course of the next five, 10 years, other companies had the same idea about drugs that would prevent arrhythmias in people who have had heart attacks. These drugs were brought to market. They were prescribed very widely because heart attacks are a very common thing, and it took so long for us to find out that these drugs also caused an increased rate of death that before we detected that safety signal, over 100,000 people died unnecessarily in America from the prescription of anti-arrhythmic drugs.
Nažalost, tokom narednih 5 godina, 10 godina, druge kompanije su došle na istu ideju o lekovima koji bi sprečili aritmije kod ljudi koji su doživeli srčani udar. Ovi lekovi su izašli na tržište. Oni su prepisivani dosta često jer su infarkti srca česti i trebalo nam je dosta vremena da otkrijemo da ovi lekovi takođe povećavaju smrtnost, tako da je, pre nego što smo otkrili znak za uzbunu, preko 100 000 ljudi u Americi nepotrebno umrlo zbog prepisivanja antiaritmika.
Now actually, in 1993, the researchers who did that 1980 study, that early study, published a mea culpa, an apology to the scientific community, in which they said, "When we carried out our study in 1980, we thought that the increased death rate that occurred in the lorcainide group was an effect of chance." The development of lorcainide was abandoned for commercial reasons, and this study was never published; it's now a good example of publication bias. That's the technical term for the phenomenon where unflattering data gets lost, gets unpublished, is left missing in action, and they say the results described here "might have provided an early warning of trouble ahead."
U stvari, 1993. istraživači koji su uradili studiju 1980. - tu početnu studiju, objavili su je kao mea culpa, izvinjenje naučnoj zajednici u kom su izjavili: “Kada smo radili studiju 1980., mislili smo da je povećana smrtnost u lorkainid grupi slučajnost.” Razvoj lorkainida je napušten iz komercijalnih razloga i ova studija nikad nije objavljena; sada je to dobar primer pristrasnosti objavljivanja. To je tehnički izraz za pojavu kada neželjeni podaci budu izgubljeni, ostanu neobjavljeni, izgube se u akciji i oni tvrde da su ovde opisani rezultati “mogli da obezbede rano upozorenje na problem”.
Now these are stories from basic science. These are stories from 20, 30 years ago. The academic publishing environment is very different now. There are academic journals like "Trials," the open access journal, which will publish any trial conducted in humans regardless of whether it has a positive or a negative result. But this problem of negative results that go missing in action is still very prevalent. In fact it's so prevalent that it cuts to the core of evidence-based medicine. So this is a drug called reboxetine, and this is a drug that I myself have prescribed. It's an antidepressant. And I'm a very nerdy doctor, so I read all of the studies that I could on this drug. I read the one study that was published that showed that reboxetine was better than placebo, and I read the other three studies that were published that showed that reboxetine was just as good as any other antidepressant, and because this patient hadn't done well on those other antidepressants, I thought, well, reboxetine is just as good. It's one to try. But it turned out that I was misled. In fact, seven trials were conducted comparing reboxetine against a dummy placebo sugar pill. One of them was positive and that was published, but six of them were negative and they were left unpublished. Three trials were published comparing reboxetine against other antidepressants in which reboxetine was just as good, and they were published, but three times as many patients' worth of data was collected which showed that reboxetine was worse than those other treatments, and those trials were not published. I felt misled.
Ovo su priče iz bazičnih nauka. Ovo su priče od pre 20, 30 godina. Naučno izdavačko okruženje je danas veoma različito. Postoje naučni časopisi kao što je “Trials”, časopis slobodnog pristupa, koji će objaviti svako ispitivanje sprovedeno na ljudima nezavisno od toga da li je ishod pozitivan ili negativan. Ali ovaj problem negativnih rezultata izgubljenih u akciji je i dalje veoma prisutan. U stvari toliko je prisutan, da je u samom centru medicine koja je zasnovana na dokazima. Ovo je lek nazvan reboksetin, taj lek sam i ja prepisivao. On je antidepresiv. Ja sam doktor štreber, tako da sam pročitao sve moguće studije o ovom leku. Pročitao sam jednu objavljenu studiju koja je pokazala da je reboksetin bolji od placeba i još tri studije koje su objavljene, a pokazale su da je dobar koliko i drugi antidepresivi i budući da ovaj pacijent nije dobro odreagovao na druge antidepresive, pomislio sam, reboksetin je jednako dobar. Mogu da pokušam. Ali se ispostavilo da sam bio obmanut. U stvari, sedam ispitivanja je sprovedeno poredeći reboksetin sa lažnom placebo šećernom tabletom. Jedna od njih je bila pozitivna i ta je objavljena, ali šest ostalih su bile negativne i nisu objavljene. Objavljena su tri ispitivanja, koja porede reboksetin sa drugim antidepresivima, u kojima je reboksetin podjednako dobar, ali tri puta više valjanih podataka o pacijentima je sakupljeno, a oni pokazuju da je reboksetin lošiji od tih drugih lekova i ova ispitivanja nisu objavljena. Osećao sam se obmanuto.
Now you might say, well, that's an extremely unusual example, and I wouldn't want to be guilty of the same kind of cherry-picking and selective referencing that I'm accusing other people of. But it turns out that this phenomenon of publication bias has actually been very, very well studied. So here is one example of how you approach it. The classic model is, you get a bunch of studies where you know that they've been conducted and completed, and then you go and see if they've been published anywhere in the academic literature. So this took all of the trials that had ever been conducted on antidepressants that were approved over a 15-year period by the FDA. They took all of the trials which were submitted to the FDA as part of the approval package. So that's not all of the trials that were ever conducted on these drugs, because we can never know if we have those, but it is the ones that were conducted in order to get the marketing authorization. And then they went to see if these trials had been published in the peer-reviewed academic literature. And this is what they found. It was pretty much a 50-50 split. Half of these trials were positive, half of them were negative, in reality. But when they went to look for these trials in the peer-reviewed academic literature, what they found was a very different picture. Only three of the negative trials were published, but all but one of the positive trials were published. Now if we just flick back and forth between those two, you can see what a staggering difference there was between reality and what doctors, patients, commissioners of health services, and academics were able to see in the peer-reviewed academic literature. We were misled, and this is a systematic flaw in the core of medicine.
Sada vi možete reći, pa dobro, to je veoma neobičan primer i ne bih želeo da budem kriv za istu vrstu probiranja i selektivnog obaveštavanja za koje optužujem druge ljude. Ali proizlazi da je ova pojava izdavačke pristrasnosti, zapravo bila veoma, veoma dobro proučavana. Ovde je jedan primer kako da joj pristupite. Klasični model je, dobijete grupu studija za koje znate da su bile sprovedene i završene, a zatim proverite da li su igde objavljene u naučnoj literaturi. Znači, ovim pristupom su uzeta sva ikad sprovedena ispitivanja o antidepresivima, a odobrila ih je FDA tokom 15 godina. Uzeli su sva ispitivanja koja su predata u FDA kao deo paketa za saglasnost. Dakle, to nisu sva ispitivanja o ovim lekovima koja su ikad sprovedena jer nikad ne možemo znati da imamo sva ispitivanja, ali to su ona koja su sprovedena da bi se dobila dozvola za proizvodnju. A zatim su proverili da li su ova ispitivanja objavljena u kritički pregledanoj naučnoj literaturi. Ovo su pronašli. To je podela gotovo pola-pola. Stvarno, polovina ovih ispitivanja je bila pozitivna, polovina negativna. Ali kada su tražili ova ispitivanja u naučnoj literaturi, dobili su veoma različitu sliku. Samo tri negativna ispitivanja su bila objavljena, ali sva pozitivna ispitivanja, izuzev jednog, bila su objavljena. Ako ih sada samo rasvetlimo možete uvideti upadljivu razliku koja postoji između stvarnosti i onoga što lekari, pacijenti, zastupnici zdravstvene službe i naučnici mogu da vide u kritički pregledanoj naučnoj literaturi. Mi smo obmanuti i ovo je sistemska mana u samoj srži medicine.
In fact, there have been so many studies conducted on publication bias now, over a hundred, that they've been collected in a systematic review, published in 2010, that took every single study on publication bias that they could find. Publication bias affects every field of medicine. About half of all trials, on average, go missing in action, and we know that positive findings are around twice as likely to be published as negative findings.
U stvari, postojalo je veoma mnogo studija urađenih o pristrasnosti izdavaštva, preko sto, tako da su one sakupljene u sistematskom prikazu objavljenom 2010. Uzeli su svaku studiju o pristrasnosti izdavaštva koju su mogli da pronađu. Pristrasnost objavljivanja utiče na svaki deo medicine. Prosečno oko polovina svih ispitivanja se izgubi u akciji, a mi znamo da će pozitivni nalazi biti dva puta češće objavljeni nego negativni.
This is a cancer at the core of evidence-based medicine. If I flipped a coin 100 times but then withheld the results from you from half of those tosses, I could make it look as if I had a coin that always came up heads. But that wouldn't mean that I had a two-headed coin. That would mean that I was a chancer and you were an idiot for letting me get away with it. (Laughter) But this is exactly what we blindly tolerate in the whole of evidence-based medicine. And to me, this is research misconduct. If I conducted one study and I withheld half of the data points from that one study, you would rightly accuse me, essentially, of research fraud. And yet, for some reason, if somebody conducts 10 studies but only publishes the five that give the result that they want, we don't consider that to be research misconduct. And when that responsibility is diffused between a whole network of researchers, academics, industry sponsors, journal editors, for some reason we find it more acceptable, but the effect on patients is damning.
Ovo je rak u srži medicine zasnovane na dokazima. Ukoliko 100 puta bacim novčić, ali zatim uskratim razultate polovine ovih bacanja, izgledalo bi da imam novčić koji uvek pokazuje glavu. Ali ovo ne bi značilo da imam novčić sa dve glave. To bi značilo da sam oportunista, i da ste vi glupi što ste mi dozvolili da se sa tim izvučem. (Smeh) Ali ovo je upravo ono što slepo dopuštamo u celokupnoj medicini zasnovanoj na dokazima. Za mene, ovo je istraživački prestup. Ukoliko sam sproveo jednu studiju i uskratio polovinu podataka iz jedne studije, vi biste me ispravno optužili za istraživačku prevaru. Ali iz nekog razloga ako neko sprovede 10 studija, ali objavi samo pet koje su dale rezultat koji su želeli, ne smatramo da je to istraživački prestup. A kada se ta odgovornost raspline u celoj mreži istraživača, naučnika, privrednih sponzora, urednika časopisa, iz nekog razloga mislimo da je prihvatljivija, ali efekat je sudbinski za pacijente.
And this is happening right now, today. This is a drug called Tamiflu. Tamiflu is a drug which governments around the world have spent billions and billions of dollars on stockpiling, and we've stockpiled Tamiflu in panic, in the belief that it will reduce the rate of complications of influenza. Complications is a medical euphemism for pneumonia and death. (Laughter) Now when the Cochrane systematic reviewers were trying to collect together all of the data from all of the trials that had ever been conducted on whether Tamiflu actually did this or not, they found that several of those trials were unpublished. The results were unavailable to them. And when they started obtaining the writeups of those trials through various different means, through Freedom of Information Act requests, through harassing various different organizations, what they found was inconsistent. And when they tried to get a hold of the clinical study reports, the 10,000-page long documents that have the best possible rendition of the information, they were told they weren't allowed to have them. And if you want to read the full correspondence and the excuses and the explanations given by the drug company, you can see that written up in this week's edition of PLOS Medicine.
Ovo se upravo sada događa, danas. Ovo je lek nazvan Tamiflu. Tamiflu je lek na koji su vlade sveta potrošile milijarde i milijarde dolara da bi stvorili zalihe i napravili smo zalihe Tamiflua u panici, verujući da će smanjiti učestalost komplikacija gripa. Komplikacije su medicinski eufemizam za upalu pluća i smrt. (Smeh) Kada su istraživači Kokranove biblioteke pokušali da sakupe sve podatke iz svih istraživanja koja su ikad sprovedena radi provere da li Tamiflu ima taj efekat ili ne, pronašli su da nekoliko ispitivanja nije objavljeno. Ovi rezultati su im bili nedostupni. A kada su počeli da dolaze do zapisa iz ovih ispitivanja, različitim putevima, preko Zakona o slobodi informacija, uznemiravanjem različitih organizacija, našli su protivurečnosti. Kada su pokušali da uzmu izveštaje kliničkih studija, to su dokumenti dugi 10 000 stranica koji sadrže najbolje predstavljene informacije, rečeno im je da nemaju dozvolu za njih. Ukoliko želite da pročitate potpunu prepisku, opravdanja i objašnjenja koja je dala farmaceutska kompanija, možete da ih vidite u ovonedeljnom izdanju časopisa PLOS Medicine.
And the most staggering thing of all of this, to me, is that not only is this a problem, not only do we recognize that this is a problem, but we've had to suffer fake fixes. We've had people pretend that this is a problem that's been fixed. First of all, we had trials registers, and everybody said, oh, it's okay. We'll get everyone to register their trials, they'll post the protocol, they'll say what they're going to do before they do it, and then afterwards we'll be able to check and see if all the trials which have been conducted and completed have been published. But people didn't bother to use those registers. And so then the International Committee of Medical Journal Editors came along, and they said, oh, well, we will hold the line. We won't publish any journals, we won't publish any trials, unless they've been registered before they began. But they didn't hold the line. In 2008, a study was conducted which showed that half of all of trials published by journals edited by members of the ICMJE weren't properly registered, and a quarter of them weren't registered at all. And then finally, the FDA Amendment Act was passed a couple of years ago saying that everybody who conducts a trial must post the results of that trial within one year. And in the BMJ, in the first edition of January, 2012, you can see a study which looks to see if people kept to that ruling, and it turns out that only one in five have done so.
Ono što mene od svega najviše zbunjuje je da ovo nije samo problem koji prepoznajemo nego što moramo da trpimo lažna rešenja. Ljudi su se pretvarali da je ovaj problem rešen. Prvo, imali smo registar ispitivanja i svi su tvrdili oh, to je dobro. Svako će prijaviti svoja ispitivanja, poslaće protokol, reći će šta će šta želi da uradi pre nego što počne, a posle ćemo moći da vidimo i proverimo da li su sva sprovedena i završena ispitivanja objavljena. Ali, ljudi se nisu zamarali upotrebom tih registara. Zatim se pojavio Internacionalni odbor urednika medicinskih časopisa koji je tvrdio da će oni biti čuvari. "Nećemo objavljivati nijedan časopis i nijedno ispitivanje ukoliko nije bilo prijavljeno pre nego što je započeto." Ali nisu izdržali. 2008. sprovedena je studija koja je pokazala da polovina ispitivanja objavljenih u časopisima, koje su uređivali članovi Odbora, nije bila pravilno prijavljena, a četvrtina uopšte nije bila prijavljena. Konačno, pre nekoliko godina usvojen je FDA amandman koji traži da svako ko sprovodi ispitivanje mora da dostavi rezultate u roku od godinu dana. U BMJ, u prvom izdanju iz januara 2012., možete da vidite studiju koja proverava da li su se istraživači pridržavali ove odluke i proizlazi da samo jedan od pet jeste.
This is a disaster. We cannot know the true effects of the medicines that we prescribe if we do not have access to all of the information.
Ovo je katastrofa. Ne možemo da znamo prave efekte lekova koje prepisujemo ako nemamo pristup svim informacijama.
And this is not a difficult problem to fix. We need to force people to publish all trials conducted in humans, including the older trials, because the FDA Amendment Act only asks that you publish the trials conducted after 2008, and I don't know what world it is in which we're only practicing medicine on the basis of trials that completed in the past two years. We need to publish all trials in humans, including the older trials, for all drugs in current use, and you need to tell everyone you know that this is a problem and that it has not been fixed. Thank you very much. (Applause) (Applause)
A ovaj problem nije teško rešiti. Moramo da nateramo istraživače da objavljuju sva ispitivanja sprovedena na ljudima, uključujući i ranija ispitivanja jer FDA amandman zahteva objavljivanje ispitivanja samo posle 2008. Ne znam u kakvom svetu živimo, ako lečimo samo na osnovu ispitivanja završenih u poslednje dve godine. Moramo da objavimo sva ispitivanja na ljudima, uključujući starija ispitivanja, za sve lekove koji se sad koriste i morate da obavestite sve koje poznajete da je ovo problem i da nije rešen. Hvala vam mnogo. (Aplauz) (Aplauz)