I've had the real fortune of working at Scripps Research for the last 17 years. It's the largest nonprofit biomedical institution in the country. And I've watched some of my colleagues, who have spent two to three years to define the crystal 3-D structure of a protein.
在过去的 17 年里,我很幸运 能在斯克利普斯研究所工作。 它是美国最大的 非营利生物医学机构。 我看到过一些同事 花了两到三年时间 定义一种蛋白质的晶体三维结构。
Well, now that can be done in two or three minutes. And that's because of the work of AlphaFold, which is a derivative of DeepMind, Demis Hassabis and John Jumper, recognized by the American Nobel Prize in September.
现在两三分钟就搞定了。 由于 AlphaFold 的成果, AlphaFold 是 DeepMind 开发的技术, 戴密斯·哈萨比斯(Demis Hassabis) 和约翰·乔普(John Jumper) 于 9 月获得了美国诺贝尔奖。
What's interesting, this work, which is taking the amino acid sequence in one dimension and predicting the three-dimensional protein at atomic level, [has] now inspired many other of these protein structure prediction models, as well as RNA and antibodies, and even being able to pick up all the missense mutations in the genome, and even being able to come up wit proteins that have never been invented before, that don't exist in nature.
有趣的是,这项成果, 从一维层面提取氨基酸序列 并以原子级别预测三维蛋白质, (已经)启发了许多其他 蛋白质结构预测模型 以及 RNA 和抗体, 甚至能发现基因组中 所有的错义突变, 甚至能提出以前从未被创造、 自然界中不存在的蛋白质。
Now, the only thing I think about this is it was a transformer model, we'll talk about that in a moment, in this award, since Demis and John and their team of 30 scientists don't understand how the transformer model works, shouldn't the AI get an asterisk as part of that award?
我唯一想到的一点是 它是一个 Transformer 模型, 我们之后会谈到, 在这个奖项中,由于戴密斯和约翰 以及他们由 30 名科学家组成的团队 不了解 Transformer 模型的工作原理, 难道 AI 不应该 从这个奖项中分一杯羹吗?
I'm going to switch from life science, which has been the singular biggest contribution just reviewed, to medicine. And in the medical community, the thing that we don't talk much about are diagnostic medical errors. And according to the National Academy of Medicine, all of us will experience at least one in our lifetime. And we know from a recent Johns Hopkins study that these errors have led to 800,000 Americans dead or seriously disabled each year. So this is a big problem. And the question is, can AI help us? And you keep hearing about the term “precision medicine.” Well, if you keep making the same mistake over and over again, that's very precise.
我将从生命科学, 也就是我们刚刚说到最重大的贡献, 谈到医学。 在医学界, 我们不太谈论的是医疗诊断错误。 根据美国国家医学院的说法, 我们所有人一生中 都会经历至少一次。 我们从约翰·霍普金斯大学 最近的一项研究中得知, 这些错误每年会导致 80 万美国人死亡 或严重残疾。 这可是个大问题。 问题是, AI 能帮助我们吗? 你总是会听到“精准医疗”这个词。 如果你一遍又一遍地 犯同样的错误,那确实非常精确。
(Laughter)
(笑声)
We don't need that, we need accuracy and precision medicine. So can we get there?
我们才不要这样, 我们需要准确和精准的医疗。 我们能达成这个目标吗?
Well, this is a picture of the retina. And this was the first major hint, training 100,000 images with supervised learning. Could the machine see things that people couldn't see? And so the question was, to the retinal experts, is this from a man or a woman? And the chance of getting it accurate was 50 percent.
这是一张视网膜的照片。 这是第一个重要迹象, 通过监督学习训练十万张图像。 机器能看见人们看不见的东西吗? 对视网膜专家问这么一个问题: 它来自男性还是女性? 答对的概率是 50%。
(Laughter)
(笑声)
But the AI got it right, 97 percent. So that training, the features are not even fully defined of how that was possible. Well that gets then to all of medical images. This is just representative, the chest X-ray. And in fact with the chest X-ray, the ability here for the AI to pick up, the radiologists, expert radiologists missing the nodule, which turned out to be picked up by the AI as cancerous, and this is, of course, representative of all of medical scans, whether it’s CT scans, MRI, ultrasound. That through supervised learning of large, labeled, annotated data sets, we can see AI do at least as well, if not better, than expert physicians.
但 AI 做对了 97%。 在这种训练中, 能达成这样效果的特征 甚至没有被完整地定义。 来说说各种医疗图像。 举个代表性的例子, 胸部 X 光检查。 其实在胸部 X 光片中, AI 能够发挥的能力是识别, 在放射科医生、 放射科专家医生遗漏了结节时, 发现结节是癌性的, 当然这也代表了所有医学扫描, 无论是 CT 扫描、 核磁共振成像还是超声波。 通过监督学习 带标签和注释的大型数据集, 我们可以看到 AI 的表现 即使不胜过, 也与专家医生相当。
And 21 randomized trials of picking up polyps -- machine vision during colonoscopy -- have all shown that polyps are picked up better with the aid of machine vision than by the gastroenterologist alone, especially as the day goes on, later in the day, interestingly. We don't know whether picking up all these additional polyps changes the natural history of cancers, but it tells you about machine eyes, the power of machine eyes.
还有 21 场检测息肉的随机试验, 在结肠镜检查过程中使用机器视觉, 全都表明,在机器视觉的帮助下, 比单靠胃肠病学家 发现息肉的效果更好, 尤其是到了一天晚些时候。 我们不知道 额外识别出这些息肉 是否会改变癌症的自然病史, 但是它展现了机器眼, 机器眼的力量。
Now that was interesting. But now still with deep learning models, not transformer models, we've seen and learned that the ability for computer vision to pick up things that human eyes can't see is quite remarkable. Here's the retina. Picking up the control of diabetes and blood pressure. Kidney disease. Liver and gallbladder disease. The heart calcium score, which you would normally get through a scan of the heart. Alzheimer's disease before any clinical symptoms have been manifest. Predicting heart attacks and strokes. Hyperlipidemia. And seven years before any symptoms of Parkinson's disease, to pick that up. Now this is interesting because in the future, we'll be taking pictures of our retina at checkups. This is the gateway to almost every system in the body. It's really striking. And we'll come back to this because each one of these studies was done with tens or hundreds [of] thousands of images with supervised learning, and they’re all separate studies by different investigators.
很有意思。 但它用的还是深度学习模型, 不是 Transformer 模型, 我们已经见证并了解到 计算机视觉识别 人眼看不见的东西的能力 非常出色。 这是视网膜。 检测到糖尿病和血压控制。 肾脏疾病。 肝胆疾病。 心脏钙化积分 通常是通过心脏扫描得出的。 在出现临床症状之前 就诊断出阿尔茨海默病。 预测心脏病发作和中风。 高脂血症。 出现帕金森氏病症状的 七年以前发现患病。 这很有趣,因为将来, 我们将在检查的时候 拍下视网膜的照片。 这是通往人体几乎所有系统的门户。 真的很惊人。 我们还会回来讨论这个问题, 因为每项研究 都是通过监督学习 使用成千上万张图像完成的, 是由不同的研究人员 分别进行的研究。
Now, as a cardiologist, I love to read cardiograms. I've been doing it for over 30 years. But I couldn't see these things. Like, the age and the sex of the patient, or the ejection fraction of the heart, making difficult diagnoses that are frequently missed. The anemia of the patient, that is, the hemoglobin to the decimal point. Predicting whether a person, who's never had atrial fibrillation or stroke from the ECG, whether that's going to likely occur. Diabetes, a diagnosis of diabetes and prediabetes, from the cardiogram. The filling pressure of the heart. Hypothyroidism and kidney disease. Imagine getting an electrocardiogram to tell you about all these other things, not really so much about the heart.
作为一名心脏病专家, 我喜欢阅读心电图。 我已经做了 30 多年了。 但我看不到这些东西。 比如,患者的年龄和性别, 或者心脏射血分数, 做出常常被忽略的困难诊断。 患者的贫血, 即血红蛋白容量极低。 通过心电图预测一个 从未发生过房颤或中风的人 是否会出现症状。 糖尿病,根据心电图 诊断糖尿病和糖尿病前期。 心脏充盈压。 甲状腺功能减退和肾脏疾病。 想象一下,让心电图 告诉你这些额外的事, 不仅仅关乎心脏。
Then there's the chest X-ray. Who would have guessed that we could accurately determine the race of the patient, no less the ethical implications of that, from a chest X-ray through machine eyes? And interestingly, picking up the diagnosis of diabetes, as well as how well the diabetes is controlled, through the chest X-ray. And of course, so many different parameters about the heart, which we could never, radiologists or cardiologists, could never be able to come up with what machine vision can do.
然后是胸部 X 光片。 谁能猜到我们可以准确地判断 患者的种族 以及与之相关的伦理意蕴, 而这都是通过由机器眼 看到的胸部 X 光片得出的? 有趣的是,还能通过胸片 诊断糖尿病 和糖尿病的控制情况, 当然,心脏有这么多不同的指标, 无论是放射科医生还是心脏病专家, 我们永远都无法做到 机器视觉能做到的这么多诊断。
Pathologists often argue about a slide, about what does it really show? But with this ability of machine eyes, the driver genomic mutations of the cancer can be defined, no less the structural copy number variants that are accounting or present in that tumor. Also, where is that tumor coming from? For many patients, we don’t know. But it can be determined through AI. And also the prognosis of the patient, just from the slide, by all of the training. Again, this is all just convolutional neural networks, not transformer models.
病理学家总是会争论一张片子 到底展现了什么。 但是,凭借这种机器眼的能力, 可以定义癌症的 驱动基因组突变, 还可以看出导致或出现在 这个肿瘤内的结构拷贝数变异。 还有,肿瘤从何而起? 对于许多患者来说,我们不知道。 但可以通过 AI 确定。 还有患者的预后, 只需要通过各种训练 分析片子得出。 同样,这只是卷积神经网络, 不是 Transformer 模型。
So when we go from the deep neural networks to transformer models, this classic pre-print, one of the most cited pre-prints ever, "Attention is All You Need," the ability to now be able to look at many more items, whether it be language or images, and be able to put this in context, setting up a transformational progress in many fields.
当我们从深度神经网络 转向 Transformer 模型时, 这份经典的预印本, 有史以来被引用次数最多的预印本之一, 《Attention is All You Need》 (意为“注意力足矣”), 可以处理更多对象的能力, 无论是语言还是图像, 并能够将其置于上下文中, 在许多领域取得了变革性进展。
The prototype is, the outgrowth of this is GPT-4. With over a trillion connections. Our human brain has 100 trillion connections or parameters. But one trillion, just think of all the information, knowledge, that's packed into those one trillion. And interestingly, this is now multimodal with language, with images, with speech. And it involves a massive amount of graphic processing units. And it's with self-supervised learning, which is a big bottleneck in medicine because we can't get experts to label images. This can be done with self-supervised learning.
这个模型的原型 或成果就是 GPT-4。 其拥有超过一万亿个连接。 我们的人脑有 100 万亿个 神经连接或参数。 但是,一万亿, 想一想这一万亿连接中 包含的所有信息、知识。 有趣的是,现在已经支持多模态, 包括语言、图像、语音。 还包含大量的图形处理单元。 还有自监督学习, 这是医学界的一大瓶颈, 因为我们不能让专家给图像打标签。 这可以通过自监督学习来完成。
So what does this set up in medicine? It sets up, for example, keyboard liberation. The one thing that both doctors, clinicians and patients would like to see. Everyone hates being data clerks as clinicians, and patients would like to see their doctor when they finally have the visit they've waited for a long time. So the ability to change the face-to-face contact is just one step along the way. By having the liberation from keyboards with synthetic notes that are driven, derived from the conversation, and then all the downstream normal data clerk functions that are done, often off-hours. Now we're seeing in health systems across the United States where people, physicians are saving many hours of time and heading towards ultimately keyboard liberation.
这在医学中起到了什么作用呢? 比如,它带来了键盘解放。 这是医生、临床医师 和患者都希望看到的一件事。 每个临床医师都不想当数据员, 等了好久终于可以看病的时候, 患者希望可以见到医生。 因此,改变面对面接触的能力 只是前进道路中的一步。 借助从对话中得到、生成的合成笔记 将人们从键盘解放出来, 在非工作时间完成 各种数据员的常规后续工作。 我们能在美国各地的卫生系统中看到, 人们、医生节省了大量的时间, 最终走向键盘解放。
We recently published, with the group at Moorfields Eye Institute, led by Pearse Keane, the first foundation model in medicine from the retina. And remember those eight different things that were all done by separate studies? This was all done with one model. This is with 1.6 million retinal images predicting all these different outcome likelihoods. And this is all open-source, which is of course really important that others can build on these models.
最近,我们与皮尔斯·基恩 (Pearse Keane)领导的 莫菲尔德眼科研究所的 研究小组一起发布了 医学界第一个基于视网膜的基础模型。 还记得那八件由不同研究完成的事吗? 都是用一个模型完成的。 用了 160 万张视网膜图像 预测了各种不同结果的可能性。 这都是开源的, 当然非常重要,这样其他人 可以基于这些模型开发。
Now I just want to review a couple of really interesting patients. Andrew, who is now six years old. He had three years of relentlessly increasing pain, arrested growth. His gait suffered with a dragging of his left foot, he had severe headaches. He went to 17 doctors over three years. His mother then entered all his symptoms into ChatGPT. It made the diagnosis of occulta spina bifida, which meant he had a tethered spinal cord that was missed by all 17 doctors over three years. He had surgery to release the cord. He's now perfectly healthy.
我想回顾几个非常有趣的患者。 安德鲁,现年六岁。 三年来,他痛苦持续加剧,成长受阻。 他的步态因左脚 牵扯性疼痛而受到影响, 头痛严重。 他在三年内去看了 17 位医生。 然后,他的母亲将他所有的症状 输入进了 ChatGPT。 它诊断为隐性脊柱裂, 这意味着他患有脊髓栓系, 三年内的所有 17 位医生 都没有注意到。 他接受了脊髓栓系松解手术。 现在非常健康。
(Applause)
(掌声)
This is a patient that was sent to me, who was suffering with, she was told, long COVID. She saw many different physicians, neurologists, and her sister entered all her symptoms after getting nowhere, no treatment for long COVID, there is no treatment validated, and her sister put all her symptoms into ChatGPT. It found out it actually was not long COVID, she had limbic encephalitis, which is treatable. She was treated, and now she's doing extremely well.
这是一位被送到我这里的病人, 她被告知患有“长新冠”。 她看了许多不同的医生、 神经科医生, 她的姐妹把她所有的症状, 在经历了一路碰壁、 长新冠无药可救、 没有经过验证的治疗方法后, 将所有症状都输入了 ChatGPT。 它发现其实并不是长新冠, 而是边缘系统脑炎, 是可以治疗的。 她接受了治疗,现在情况非常好。
But these are not just anecdotes anymore. 70 very difficult cases that are the clinical pathologic conferences at the New England Journal of Medicine were compared to GPT-4, and the chatbot did as well or better than the expert master clinicians in making the diagnosis.
但这些不再只是个例了。 70 例非常困难的病例 登上《新英格兰医学杂志》的 临床病理学会议, 与 GPT-4 进行了比较, 聊天机器人在做出诊断方面的表现 与临床专家相当或更好。
So I just want to close with a recent conversation with my fellow. Medicine is still an apprenticeship, and Andrew Cho is 30 years old, in his second year of cardiology fellowship. We see all patients together in the clinic. And at the end of clinic the other day, I sat down and said to him, "Andrew, you are so lucky. You're going to be practicing medicine in an era of keyboard liberation. You're going to be connecting with patients the way we haven't done for decades." That is the ability to have the note and the work from the conversation to derive things like pre-authorization, billing, prescriptions, future appointments -- all the things that we do, including nudges to the patient. For example, did you get your blood pressure checks and what did they show and all that coming back to you. But much more than that, to help with making diagnoses. And the gift of time that having all the data of a patient that's all teed up before even seeing the patient. And all this support changes the future of the patient-doctor relationship, bringing in the gift of time. So this is really exciting. I said to Andrew, everything has to be validated, of course, that the benefit greatly outweighs any risk. But it is really a remarkable time for the future of health care, it's so damn exciting.
我想以最近与我的同事的对话收尾。 医学仍然采用的是“师徒制”, 安德鲁·赵(Andrew Cho) 今年 30 岁, 是他攻读心脏病学培训的第二年。 我们一起在诊所为所有患者看病。 有一天在看诊结束时, 我坐下来对他说: “安德鲁,你真幸运。 你能在键盘解放的时代 从事医学工作。 你会以我们几十年来前所未有的方式 与患者接触。” 这就是能够从对话中获取笔记 和工作成果的能力, 从而得出诸如预授权、 账单、处方、未来预约之类 我们要做的事, 包括提示患者。 比如,你有没有量血压, 得到的结果是什么意思, 这些都能回到你的手中。 但不仅如此, 还有助于做出诊断。 还有时间上的优势, 在见到病人之前 就已经准备好了患者的所有数据。 这些帮助都改变了医患关系的未来, 带来了时间的恩赐。 这真的很令人兴奋。 我对安德鲁说, 当然,这一切都必须经过验证, 证明好处远大于任何风险。 但是对于医疗保健的未来来说, 这确实是一个重大的时刻, 真是令人兴奋。
Thank you.
谢谢。
(Applause)
(掌声)