Hi. So, this chap here, he thinks he can tell you the future. His name is Nostradamus, although here the Sun have made him look a little bit like Sean Connery. (Laughter)
Ola! Resulta que este tipo cre que pode predicir o futuro. Chámase Nostradamus, aínda que aquí en <i>The Sun</i> paréceselle un pouco ao Sean Connery. (Risas)
And like most of you, I suspect, I don't really believe that people can see into the future. I don't believe in precognition, and every now and then, you hear that somebody has been able to predict something that happened in the future, and that's probably because it was a fluke, and we only hear about the flukes and about the freaks. We don't hear about all the times that people got stuff wrong. Now we expect that to happen with silly stories about precognition, but the problem is, we have exactly the same problem in academia and in medicine, and in this environment, it costs lives.
E como a maioría de vós, sospeito, non creo realmente que a xente poida prever o futuro. Non creo na precognición e, cando ás veces escoitas que alguén foi quen de predicir algo que pasou no futuro, probablemente fose unha casualidade, pero só escoitamos falar das casualidades e da sorte. Non sabemos todas as veces que errou esta xente. Se ben esperamos que isto ocorra coas historias parvas sobre precognición, o caso é que temos o mesmo problema no mundo académico e na medicina, e, nese contexto, custa vidas.
So firstly, thinking just about precognition, as it turns out, just last year a researcher called Daryl Bem conducted a piece of research where he found evidence of precognitive powers in undergraduate students, and this was published in a peer-reviewed academic journal and most of the people who read this just said, "Okay, well, fair enough, but I think that's a fluke, that's a freak, because I know that if I did a study where I found no evidence that undergraduate students had precognitive powers, it probably wouldn't get published in a journal. And in fact, we know that that's true, because several different groups of research scientists tried to replicate the findings of this precognition study, and when they submitted it to the exact same journal, the journal said, "No, we're not interested in publishing replication. We're not interested in your negative data." So this is already evidence of how, in the academic literature, we will see a biased sample of the true picture of all of the scientific studies that have been conducted.
Falando da precognición, resulta que, o pasado ano, o investigador Daryl Bem realizou un estudo no que achou probas de poderes precognitivos en estudantes universitarios, e isto publicouse nunha revista académica revisada por pares. A maioría da xente que o leu dixo: “Vale, pero creo que tiveron sorte, porque sei que se eu fixese un estudo no que non atopase probas de que os estudantes teñen poderes precognitivos, probablemente non se publicaría nunha revista.” De feito, sabemos que é verdade, porque varios grupos diferentes de investigadores intentaron replicar os achados deste estudo de precognición e, ao presentalo á mesma revista, contestáronlles: “Non estamos interesados en publicar copias nin nos vosos datos negativos.” Isto xa evidencia como, na literatura académica, veremos unha mostra parcial da imaxe real de todos os estudos científicos realizados.
But it doesn't just happen in the dry academic field of psychology. It also happens in, for example, cancer research. So in March, 2012, just one month ago, some researchers reported in the journal Nature how they had tried to replicate 53 different basic science studies looking at potential treatment targets in cancer, and out of those 53 studies, they were only able to successfully replicate six. Forty-seven out of those 53 were unreplicable. And they say in their discussion that this is very likely because freaks get published. People will do lots and lots and lots of different studies, and the occasions when it works they will publish, and the ones where it doesn't work they won't. And their first recommendation of how to fix this problem, because it is a problem, because it sends us all down blind alleys, their first recommendation of how to fix this problem is to make it easier to publish negative results in science, and to change the incentives so that scientists are encouraged to post more of their negative results in public.
Pero non só ocorre no árido campo académico da psicoloxía. Tamén acontece, por exemplo, na investigación do cancro. En marzo do 2012, hai un mes, algúns investigadores informaron na revista <i>Nature</i> de como intentaran replicar 53 estudos científicos básicos observando obxectivos potenciais de tratamento do cancro. Destes 53 estudos, só puideron replicar seis con éxito; 47 deses 53 non eran reproducibles. Na discusión, din que isto pode ser porque as casualidades son as que se publican. Fanse moitísimos estudos diferentes. Cando funcionan, saen publicados e, cando non funcionan, non. A súa primeira recomendación para solucionar o problema, porque é un problema que nos envíanos a rúas sen saída, a súa primeira recomendación para solucionalo é facilitar a publicación de resultados negativos na ciencia e cambiar os incentivos para animar os científicos a publicar máis dos seus resultados negativos.
But it doesn't just happen in the very dry world of preclinical basic science cancer research. It also happens in the very real, flesh and blood of academic medicine. So in 1980, some researchers did a study on a drug called lorcainide, and this was an anti-arrhythmic drug, a drug that suppresses abnormal heart rhythms, and the idea was, after people have had a heart attack, they're quite likely to have abnormal heart rhythms, so if we give them a drug that suppresses abnormal heart rhythms, this will increase the chances of them surviving. Early on its development, they did a very small trial, just under a hundred patients. Fifty patients got lorcainide, and of those patients, 10 died. Another 50 patients got a dummy placebo sugar pill with no active ingredient, and only one of them died. So they rightly regarded this drug as a failure, and its commercial development was stopped, and because its commercial development was stopped, this trial was never published.
Pero isto non só ocorre no árido mundo da investigación preclínica do cancro. Tamén acontece na medicina académica real, de carne e óso. No ano 1980, uns investigadores fixeron un estudo sobre a iorcainida, un medicamento antiarrítmico, que suprime os ritmos cardíacos anormais, e a idea era que, como despois dun ataque cardíaco é moi probable ter ritmos cardíacos anormais, se lles damos algo que suprima ese ritmo anormal, aumentarán as posibilidades de supervivencia. Nos seus comezos, fixeron un ensaio pequeno, con apenas 100 pacientes. Cincuenta pacientes recibiron iorcainida, e dez deles morreron. Outros 50 recibiron o placebo, unha pílula de azucre sen ingrediente activo, e só un deles morreu, así que consideraron este fármaco como un fracaso e detívose o seu desenvolvemento, e por isto mesmo o ensaio nunca se publicou.
Unfortunately, over the course of the next five, 10 years, other companies had the same idea about drugs that would prevent arrhythmias in people who have had heart attacks. These drugs were brought to market. They were prescribed very widely because heart attacks are a very common thing, and it took so long for us to find out that these drugs also caused an increased rate of death that before we detected that safety signal, over 100,000 people died unnecessarily in America from the prescription of anti-arrhythmic drugs.
Desafortunadamente, ao longo dos seguintes cinco, dez anos, outras compañías tiveron a mesma idea sobre fármacos que evitarían arritmias en persoas con ataques cardíacos. Estes fármacos leváronse ao mercado. Prescribíronse moito, pois os ataques cardíacos son moi comúns, e tardamos tanto en descubrir que estes fármacos tamén aumentaban a taxa de mortalidade que, antes de detectar ese problema, máis de 100 000 persoas morreron innecesariamente nos EE. UU. por prescrición de medicamentos antiarrítmicos.
Now actually, in 1993, the researchers who did that 1980 study, that early study, published a mea culpa, an apology to the scientific community, in which they said, "When we carried out our study in 1980, we thought that the increased death rate that occurred in the lorcainide group was an effect of chance." The development of lorcainide was abandoned for commercial reasons, and this study was never published; it's now a good example of publication bias. That's the technical term for the phenomenon where unflattering data gets lost, gets unpublished, is left missing in action, and they say the results described here "might have provided an early warning of trouble ahead."
De feito, en 1993, os investigadores que fixeran o estudo de 1980 publicaron un mea culpa, unha desculpa á comunidade científica, na que dixeron: “Cando fixemos o noso estudo en 1980, pensamos que o aumento da mortalidade do grupo da iorcainida fora efecto do azar.” O desenvolvemento detívose por razóns comerciais, e este estudo nunca se publicou; un bo exemplo de nesgo de publicación. Este é o termo técnico para o fenómeno en que os datos desagradables se perden, non se publican, desaparecen e, polo que din, os resultados aquí descritos “poderían proporcionar un aviso de problemas por vir.”
Now these are stories from basic science. These are stories from 20, 30 years ago. The academic publishing environment is very different now. There are academic journals like "Trials," the open access journal, which will publish any trial conducted in humans regardless of whether it has a positive or a negative result. But this problem of negative results that go missing in action is still very prevalent. In fact it's so prevalent that it cuts to the core of evidence-based medicine. So this is a drug called reboxetine, and this is a drug that I myself have prescribed. It's an antidepressant. And I'm a very nerdy doctor, so I read all of the studies that I could on this drug. I read the one study that was published that showed that reboxetine was better than placebo, and I read the other three studies that were published that showed that reboxetine was just as good as any other antidepressant, and because this patient hadn't done well on those other antidepressants, I thought, well, reboxetine is just as good. It's one to try. But it turned out that I was misled. In fact, seven trials were conducted comparing reboxetine against a dummy placebo sugar pill. One of them was positive and that was published, but six of them were negative and they were left unpublished. Three trials were published comparing reboxetine against other antidepressants in which reboxetine was just as good, and they were published, but three times as many patients' worth of data was collected which showed that reboxetine was worse than those other treatments, and those trials were not published. I felt misled.
Estas son historias de ciencia básica, historias de hai 20, 30 anos. O ambiente académico editorial é moi diferente agora. Hai revistas académicas como <i>Trials</i>, a revista de acceso aberto, que publica calquera ensaio feito en humanos, tivese un resultado positivo ou negativo. Pero este problema de resultados negativos que se abandonan segue sendo moi frecuente. É tan frecuente que afecta ás bases da medicina baseada en probas. Este é un fármaco chamado reboxetina; téñoa prescrito eu mesmo. É un antidepresivo. Son un médico moi estudoso, así que lin todos os estudos que puiden dela. Lin un estudo publicado que amosaba que a reboxetina era mellor que o placebo, e tamén outros tres estudos que mostraban que era igual de boa que outros antidepresivos. Como ao paciente non lle foran ben os outros antidepresivos, pensei que podía probar a reboxetina. Pero resulta que me enganaron. De feito, houbo sete ensaios que comparaban a reboxetina cunha pílula de azucre placebo. Un deles foi positivo e publicouse, pero seis foron negativos e quedaron sen publicar. Publicáronse tres ensaios que afirmaban que era tan boa como outros antidepresivos, polo que se publicaron, pero recolléronse tres veces máis datos de pacientes que demostraban que a reboxetina era peor que outros tratamentos, e estes ensaios non se publicaron. Sentinme enganado.
Now you might say, well, that's an extremely unusual example, and I wouldn't want to be guilty of the same kind of cherry-picking and selective referencing that I'm accusing other people of. But it turns out that this phenomenon of publication bias has actually been very, very well studied. So here is one example of how you approach it. The classic model is, you get a bunch of studies where you know that they've been conducted and completed, and then you go and see if they've been published anywhere in the academic literature. So this took all of the trials that had ever been conducted on antidepressants that were approved over a 15-year period by the FDA. They took all of the trials which were submitted to the FDA as part of the approval package. So that's not all of the trials that were ever conducted on these drugs, because we can never know if we have those, but it is the ones that were conducted in order to get the marketing authorization. And then they went to see if these trials had been published in the peer-reviewed academic literature. And this is what they found. It was pretty much a 50-50 split. Half of these trials were positive, half of them were negative, in reality. But when they went to look for these trials in the peer-reviewed academic literature, what they found was a very different picture. Only three of the negative trials were published, but all but one of the positive trials were published. Now if we just flick back and forth between those two, you can see what a staggering difference there was between reality and what doctors, patients, commissioners of health services, and academics were able to see in the peer-reviewed academic literature. We were misled, and this is a systematic flaw in the core of medicine.
Poderiades dicir que é un exemplo moi pouco habitual, e non me gustaría caer no mesmo tipo de escolla e referencia selectiva das que acuso outras persoas, pero o fenómeno do nesgo de publicación está máis que estudado. Aquí temos un exemplo de como abordalo. O modelo clásico é coller uns cantos estudos que sabemos que se fixeron e completaron e logo ver se se publicaron xa na literatura académica. Isto levou a todos os ensaios sobre antidepresivos aprobados pola FDA nun período de 15 anos. Tomaron todos os ensaios presentados á FDA para a súa aprobación. Non son todos os ensaios feitos sobre estes fármacos. Nunca poderemos saber se os temos todos. Son os que se fixeron para obter a autorización de comercialización. Despois, viron se estes ensaios se publicaran na literatura revisada por pares. Isto é o que descubriron. Había unha división de 50-50. A metade destes ensaios foran positivos e a metade negativos. Pero cando buscaron estes ensaios na literatura revisada por pares, o que descubriron foi moi diferente: só tres dos ensaios negativos se publicaran, pero publicáranse todos os ensaios positivos menos un. Agora ben, se facemos unha comparativa, podedes ver a diferenza asombrosa que había entre a realidade e o que os médicos, os pacientes, os membros dos servizos de saúde e os académicos puideron ver na literatura académica revisada por pares. Enganáronnos, e isto é un defecto sistemático na base da medicina.
In fact, there have been so many studies conducted on publication bias now, over a hundred, that they've been collected in a systematic review, published in 2010, that took every single study on publication bias that they could find. Publication bias affects every field of medicine. About half of all trials, on average, go missing in action, and we know that positive findings are around twice as likely to be published as negative findings.
De feito, fixéronse tantos estudos sobre o nesgo na publicación, máis de 100, que se recolleron nunha revisión sistemática publicada no 2010, con todos os estudos que se atoparon sobre o nesgo na publicación. O nesgo na publicación afecta a todos os campos da medicina. Arredor da metade dos ensaios, de media, abandónanse, e sabemos que os resultados positivos teñen o dobre de probabilidades de publicarse que os negativos.
This is a cancer at the core of evidence-based medicine. If I flipped a coin 100 times but then withheld the results from you from half of those tosses, I could make it look as if I had a coin that always came up heads. But that wouldn't mean that I had a two-headed coin. That would mean that I was a chancer and you were an idiot for letting me get away with it. (Laughter) But this is exactly what we blindly tolerate in the whole of evidence-based medicine. And to me, this is research misconduct. If I conducted one study and I withheld half of the data points from that one study, you would rightly accuse me, essentially, of research fraud. And yet, for some reason, if somebody conducts 10 studies but only publishes the five that give the result that they want, we don't consider that to be research misconduct. And when that responsibility is diffused between a whole network of researchers, academics, industry sponsors, journal editors, for some reason we find it more acceptable, but the effect on patients is damning.
Isto é un cancro na base dunha medicina baseada en probas. Se lanzase unha moeda 100 veces, pero agochase os resultados da metade dos lanzamentos, podería facer que parecese que a moeda sempre cae de cara. Pero non é que a moeda teña dúas caras, senón que son un oportunista e vós uns idiotas por deixarmo facer. (Risas) Pero é exactamente isto o que toleramos cegamente na medicina baseada en probas. Para min, isto é mala conduta na investigación. Se eu fixese un estudo e agochase a metade dos datos dese estudo, poderiades acusarme con razón de fraude na investigación. Aínda así, por algunha razón, se alguén fai dez estudos pero só publica os cinco que deron o resultado que quería, non o consideramos mala conduta na investigación. Cando a responsabilidade se difunde entre toda unha rede de investigadores, académicos, patrocinadores do sector e editores de revistas, por algún motivo atopámolo máis aceptable, pero o efecto sobre os pacientes é moi negativo.
And this is happening right now, today. This is a drug called Tamiflu. Tamiflu is a drug which governments around the world have spent billions and billions of dollars on stockpiling, and we've stockpiled Tamiflu in panic, in the belief that it will reduce the rate of complications of influenza. Complications is a medical euphemism for pneumonia and death. (Laughter) Now when the Cochrane systematic reviewers were trying to collect together all of the data from all of the trials that had ever been conducted on whether Tamiflu actually did this or not, they found that several of those trials were unpublished. The results were unavailable to them. And when they started obtaining the writeups of those trials through various different means, through Freedom of Information Act requests, through harassing various different organizations, what they found was inconsistent. And when they tried to get a hold of the clinical study reports, the 10,000-page long documents that have the best possible rendition of the information, they were told they weren't allowed to have them. And if you want to read the full correspondence and the excuses and the explanations given by the drug company, you can see that written up in this week's edition of PLOS Medicine.
E isto está a suceder agora mesmo, hoxe. Este é un fármaco chamado Tamiflu. Os gobernos de todo o mundo gastaron miles de millóns de dólares en acumular este fármaco, e fixérono por pánico, por crer que reduciría a taxa de complicacións da gripe. Complicacións aquí é un eufemismo médico para a pneumonía e a morte. (Risas) E cando os revisores sistemáticos de Cochrane intentaban reunir os datos de todos os ensaios sobre se o Tamiflu axudaba nisto ou non, acharon que varios destes ensaios quedaran sen publicar. Non podían acceder aos resultados. Cando comezaron a obter os rexistros deses ensaios por diferentes medios, pola Lei pola liberdade da información, acosando a varias organizacións, o que descubriron era inconsistente. Cando intentaron obter os informes dos estudos, os documentos de 10 000 páxinas coa mellor interpretación posible da información, dixéronlles que non podían telos. Se queredes ler a correspondencia completa e as escusas e explicacións da farmacéutica, podedes vela na edición desta semana da revista <i>PLOS Medicine</i>.
And the most staggering thing of all of this, to me, is that not only is this a problem, not only do we recognize that this is a problem, but we've had to suffer fake fixes. We've had people pretend that this is a problem that's been fixed. First of all, we had trials registers, and everybody said, oh, it's okay. We'll get everyone to register their trials, they'll post the protocol, they'll say what they're going to do before they do it, and then afterwards we'll be able to check and see if all the trials which have been conducted and completed have been published. But people didn't bother to use those registers. And so then the International Committee of Medical Journal Editors came along, and they said, oh, well, we will hold the line. We won't publish any journals, we won't publish any trials, unless they've been registered before they began. But they didn't hold the line. In 2008, a study was conducted which showed that half of all of trials published by journals edited by members of the ICMJE weren't properly registered, and a quarter of them weren't registered at all. And then finally, the FDA Amendment Act was passed a couple of years ago saying that everybody who conducts a trial must post the results of that trial within one year. And in the BMJ, in the first edition of January, 2012, you can see a study which looks to see if people kept to that ruling, and it turns out that only one in five have done so.
O máis sorprendente de todo isto, para min, é que non só é isto un problema, non só recoñecemos que é un problema, senón que temos que aturar as falsas solucións. Houbo xente que finxiu que este era un problema que xa se solucionara. Temos os rexistros dos ensaios. Todos dixeron: “Pediremos a todos que rexistren os seus ensaios e publiquen o protocolo, dirán o que van facer antes de facelo e despois poderemos comprobar se todos os ensaios completados se publicaron.” Pero non se molestaron en usar os rexistros. Entón, o Comité Internacional de Editores de Revistas Médicas dixo: “Xa veredes. Non publicaremos revistas nin ensaios salvo que estivesen rexistrados antes de comezar.” Pero non cumpriron. En 2008 houbo un estudo que demostrou que a metade dos ensaios publicados polas revistas de membros do Comité Internacional de Editores de Revistas Médicas non estaban debidamente rexistrados, e un cuarto deles nin foran rexistrados. Finalmente aprobouse a Lei de modificación da FDA hai un par de anos, que di que todo aquel que faga un ensaio ten que publicar os resultados dentro dun ano. Na revista <i>BMJ</i>, na primeira edición de xaneiro de 2012, pódese ver un estudo que analiza se esta lei se cumpriu, e resulta que só un de cada cinco o fixo.
This is a disaster. We cannot know the true effects of the medicines that we prescribe if we do not have access to all of the information.
Isto é un desastre. Non podemos saber os verdadeiros efectos dos medicamentos que receitamos se non temos acceso a toda a información.
And this is not a difficult problem to fix. We need to force people to publish all trials conducted in humans, including the older trials, because the FDA Amendment Act only asks that you publish the trials conducted after 2008, and I don't know what world it is in which we're only practicing medicine on the basis of trials that completed in the past two years. We need to publish all trials in humans, including the older trials, for all drugs in current use, and you need to tell everyone you know that this is a problem and that it has not been fixed. Thank you very much. (Applause) (Applause)
E non é un problema difícil de resolver. Temos que obrigar á xente a publicar todos os ensaios realizados en humanos, tamén os máis antigos, porque a lei só obriga a publicar os ensaios posteriores a 2008, e non sei que mundo é este no que só practicamos a medicina baseándonos en ensaios completados nos pasados dous anos. Debemos publicar todos os ensaios en humanos, incluíndo os máis antigos, de todos os fármacos de uso actual, e tedes que dicirlles a todos os que coñecedes que isto é un problema e que non se solucionou. Moitas grazas. (Aplausos) (Aplausos)