I study how the brain processes information. That is, how it takes information in from the outside world, and converts it into patterns of electrical activity, and then how it uses those patterns to allow you to do things -- to see, hear, to reach for an object. So I'm really a basic scientist, not a clinician, but in the last year and a half I've started to switch over, to use what we've been learning about these patterns of activity to develop prosthetic devices, and what I wanted to do today is show you an example of this. It's really our first foray into this. It's the development of a prosthetic device for treating blindness.
我研究的是大脑如何处理信息 也就是 大脑如何接受外界信息 并把它转换成脑电活动模式 然后又如何利用那些模式 来控制行为- 看,听,拿东西 我是个理论基础科学家 但在过去的一年半里 我开始转变方向 利用对刚刚所说的模型的了解 来开发假体装置 我今天想给各位展示 其中一个例子 这是我们对此做的第一个尝试 是一个用于治疗失明的 假体装置
So let me start in on that problem. There are 10 million people in the U.S. and many more worldwide who are blind or are facing blindness due to diseases of the retina, diseases like macular degeneration, and there's little that can be done for them. There are some drug treatments, but they're only effective on a small fraction of the population. And so, for the vast majority of patients, their best hope for regaining sight is through prosthetic devices. The problem is that current prosthetics don't work very well. They're still very limited in the vision that they can provide. And so, you know, for example, with these devices, patients can see simple things like bright lights and high contrast edges, not very much more, so nothing close to normal vision has been possible.
首先从失明这个问题谈起 由于视网膜病变 而全球失明的人数更多 他们因为视网膜的疾病 或者黄斑部变性这样的问题失去视力 然而, 人们对此却无能为力 虽然有一些治疗性药物 但它们只对很少一部分患者有效 而对于大部分病人来说 他们恢复视力的希望 就得寄托于假体设备 问题就是目前的假体效果并不好 这些假体仍然 他们提供的视觉范围还是很有限 比如说 这些装置只能帮病人看到 亮光和高对比度的轮廓 几乎没有其他影像 所以它们没办法让病人看到正常视觉范围的影像
So what I'm going to tell you about today is a device that we've been working on that I think has the potential to make a difference, to be much more effective, and what I wanted to do is show you how it works. Okay, so let me back up a little bit and show you how a normal retina works first so you can see the problem that we were trying to solve. Here you have a retina. So you have an image, a retina, and a brain. So when you look at something, like this image of this baby's face, it goes into your eye and it lands on your retina, on the front-end cells here, the photoreceptors. Then what happens is the retinal circuitry, the middle part, goes to work on it, and what it does is it performs operations on it, it extracts information from it, and it converts that information into a code. And the code is in the form of these patterns of electrical pulses that get sent up to the brain, and so the key thing is that the image ultimately gets converted into a code. And when I say code, I do literally mean code. Like this pattern of pulses here actually means "baby's face," and so when the brain gets this pattern of pulses, it knows that what was out there was a baby's face, and if it got a different pattern it would know that what was out there was, say, a dog, or another pattern would be a house. Anyway, you get the idea.
今天我想告诉诸位的是 我们正在研制的一个设备 将有可能改变这一切 这个设备将会更加有效 我想在这里展示一下它的工作原理 首先我会讲一点背景知识 让大家了解一下正常视网膜的工作原理 这样你就可以明白 我们要解决的问题是什么 这是一个视网膜 现在你可以看到一张图片,一个视网膜和大脑 当你看着面前的事物 比如这张婴儿的脸的图片 图像进入你的眼睛并停留在你的视网膜上, 它停留在视网膜的前端细胞上,也叫感光器 然后视网膜的中间部分,即它的电路系统 就开始处理这个图像 它开始对图像进行操作, 从图像中提取信息 然后把信息转换成代码 代码以电脉冲的形式 送入大脑 所以关键环节就是 把图像转换成代码 我所说的代码 就是字面意思的代码 就像这里的脉冲就是“婴儿脸”的意思 所以当大脑得到脉冲的模型 就知道了 这是个婴儿的脸 如果是不同的模型 那它就知道是别的,比如一条狗 或者房子 总之 就是这个意思
And, of course, in real life, it's all dynamic, meaning that it's changing all the time, so the patterns of pulses are changing all the time because the world you're looking at is changing all the time too. So, you know, it's sort of a complicated thing. You have these patterns of pulses coming out of your eye every millisecond telling your brain what it is that you're seeing. So what happens when a person gets a retinal degenerative disease like macular degeneration? What happens is is that, the front-end cells die, the photoreceptors die, and over time, all the cells and the circuits that are connected to them, they die too. Until the only things that you have left are these cells here, the output cells, the ones that send the signals to the brain, but because of all that degeneration they aren't sending any signals anymore. They aren't getting any input, so the person's brain no longer gets any visual information -- that is, he or she is blind.
当然现实生活是动态的 物体总是变化的 所以脉冲模型也一直变化 因为你看到的世界 是一直在变化的 所以这就有点复杂了 你的眼睛每一毫秒 都会发送不同的脉冲模型 告诉大脑你看到的事物 如果一个人 的视网膜变性 比如患上黄斑部变性,情况会怎么样呢? 这时前端细胞就会死亡 感光器死亡 接着所有的细胞和联结这些细胞的组织 都会死亡 最后剩下的只有 那些输出细胞 就是那些把信号送入大脑的细胞 但是因为这些变性 那些细胞就没有信号可输出了 没有信号输入它们 人的大脑就失去了 视觉信息 这样他就失明了
So, a solution to the problem, then, would be to build a device that could mimic the actions of that front-end circuitry and send signals to the retina's output cells, and they can go back to doing their normal job of sending signals to the brain. So this is what we've been working on, and this is what our prosthetic does. So it consists of two parts, what we call an encoder and a transducer. And so the encoder does just what I was saying: it mimics the actions of the front-end circuitry -- so it takes images in and converts them into the retina's code. And then the transducer then makes the output cells send the code on up to the brain, and the result is a retinal prosthetic that can produce normal retinal output. So a completely blind retina, even one with no front-end circuitry at all, no photoreceptors, can now send out normal signals, signals that the brain can understand. So no other device has been able to do this.
一个解决方案就是 制造一个设备来模仿 前端细胞电路系统的功能 把信号传给视网膜的输出细胞 这样它们就能重新开始正常工作, 把信号输入大脑 这是我们的研究 这就是我们制作的假体的功能 它由两部分组成 我们称为编码器和传感器 就像我之前提到的,编码器 模仿前段细胞的功能- 接受图像,并把它转换成 视网膜能接受的编码 然后传感器使 输出细胞把编码 送进大脑 这样视网膜假体就能产生 正常的视网膜输出 因此一个完全失明的视网膜 即使前端细胞完全不能工作 没有感光器 也能够输出正常信号 并且大脑能理解这些信号 还没有其他的设备能 做到这点
Okay, so I just want to take a sentence or two to say something about the encoder and what it's doing, because it's really the key part and it's sort of interesting and kind of cool. I'm not sure "cool" is really the right word, but you know what I mean. So what it's doing is, it's replacing the retinal circuitry, really the guts of the retinal circuitry, with a set of equations, a set of equations that we can implement on a chip. So it's just math. In other words, we're not literally replacing the components of the retina. It's not like we're making a little mini-device for each of the different cell types. We've just abstracted what the retina's doing with a set of equations. And so, in a way, the equations are serving as sort of a codebook. An image comes in, goes through the set of equations, and out comes streams of electrical pulses, just like a normal retina would produce.
下面 我简略说一下 编码器的功能 因为这真的是个关键部分 很有意思也很酷 我也不知道该不该用“酷”形容 不过就是那个意思 它的功能就是 用一套方程式来替代视网膜电路系统 而且是它最重要的部分 我们可以把这些方程式植入芯片中 所以这其实就是数学 也就是说我们没有 真的换掉视网膜的构成部分 不是说我们做了个迷你设备 来替代每一个不同的细胞 我们只是用方程式 来提取视网膜的工作原理 这样,这些方程式的功能就 类似于密码本 一个图像穿过方程式 变成一串电脉冲 就像正常视网膜所产生的效果一样
Now let me put my money where my mouth is and show you that we can actually produce normal output, and what the implications of this are. Here are three sets of firing patterns. The top one is from a normal animal, the middle one is from a blind animal that's been treated with this encoder-transducer device, and the bottom one is from a blind animal treated with a standard prosthetic. So the bottom one is the state-of-the-art device that's out there right now, which is basically made up of light detectors, but no encoder. So what we did was we presented movies of everyday things -- people, babies, park benches, you know, regular things happening -- and we recorded the responses from the retinas of these three groups of animals. Now just to orient you, each box is showing the firing patterns of several cells, and just as in the previous slides, each row is a different cell, and I just made the pulses a little bit smaller and thinner so I could show you a long stretch of data.
下面我就来用 实际行动展示一下 证明我们确实能够产生输出信号 以及这么做的意义 这是三个放电模型 最上面的来自 一个正常动物 中间的来自一支使用编码传感设备的 失明的动物 最下面的来自一个使用标准假体的 失明动物 最下面是目前能找到的 最尖端的科技设备 主要就是由光探测器组成的 但是没有编码器 我们做的就是把日常事物- 人,婴儿,公园座椅等 这些日常生活中的东西给它们看 然后记录下三组动物视网膜 的不同反应 首先要说明的是,每个长方形都表示 几个细胞的放电模型 和前几张幻灯片中一样 每一行是一个不同的细胞 我把脉冲做得更小更细 这样你可以看到 更详细的数据
So as you can see, the firing patterns from the blind animal treated with the encoder-transducer really do very closely match the normal firing patterns -- and it's not perfect, but it's pretty good -- and the blind animal treated with the standard prosthetic, the responses really don't. And so with the standard method, the cells do fire, they just don't fire in the normal firing patterns because they don't have the right code. How important is this? What's the potential impact on a patient's ability to see? So I'm just going to show you one bottom-line experiment that answers this, and of course I've got a lot of other data, so if you're interested I'm happy to show more. So the experiment is called a reconstruction experiment. So what we did is we took a moment in time from these recordings and asked, what was the retina seeing at that moment? Can we reconstruct what the retina was seeing from the responses from the firing patterns?
如图所示 使用编码传感器设备的 失明动物的模型能够 很好地与正常模型吻合- 虽不完美但已经很好了- 而使用普通假体的 失明动物的模型 效果就不行 用普通假体的时候 细胞的确可以放电 但是并不是以正常的放电方式进行 因为它们没有正确的编码 编码有多重要呢? 它对病人的视力 有多大潜在影响? 我会向各位展示一个 基础实验来回来这个问题 当然我还有别的数据 如果你有兴趣 我可以展示更多 这实验被称作重建实验 我们从记录里 截取某一刻 看看那一刻视网膜都看到了什么? 我们能不能重建 视网膜从那三种放电模型中 看到的东西?
So, when we did this for responses from the standard method and from our encoder and transducer. So let me show you, and I'm going to start with the standard method first. So you can see that it's pretty limited, and because the firing patterns aren't in the right code, they're very limited in what they can tell you about what's out there. So you can see that there's something there, but it's not so clear what that something is, and this just sort of circles back to what I was saying in the beginning, that with the standard method, patients can see high-contrast edges, they can see light, but it doesn't easily go further than that. So what was the image? It was a baby's face. So what about with our approach, adding the code? And you can see that it's much better. Not only can you tell that it's a baby's face, but you can tell that it's this baby's face, which is a really challenging task. So on the left is the encoder alone, and on the right is from an actual blind retina, so the encoder and the transducer. But the key one really is the encoder alone, because we can team up the encoder with the different transducer.
我们对普通假体 和编码传感器的反应 各做了实验 首先我来说明 普通假体的实验结果 可以看出效果很有限 因为放电模型没有正确编码 所以它很难看出来 究竟是什么图像 你可以看到 那里是有些什么东西,但是不清楚到底是什么 这又回到了 我在开头的时候说的 关于普通假体的问题 安装普通假体的病人可以看到高对比度的轮廓 他们能看见亮光 但是仅此而已 到底是什么图像?是婴儿的脸 我们加入了编码后的方法 是什么效果呢? 你可以看出来,效果好多了 你不仅能看出是婴儿的脸 而且还能看清是这样一个孩子的脸 这真是个很具挑战性的任务 左边是单用编码器的结果 右边是一个实际的失明视网膜 和编码传感器™ 但是最关键的就是编码器 因为我们可以把不同的传感器和 编码器结合起来
This is just actually the first one that we tried. I just wanted to say something about the standard method. When this first came out, it was just a really exciting thing, the idea that you even make a blind retina respond at all. But there was this limiting factor, the issue of the code, and how to make the cells respond better, produce normal responses, and so this was our contribution. Now I just want to wrap up, and as I was mentioning earlier of course I have a lot of other data if you're interested, but I just wanted to give this sort of basic idea of being able to communicate with the brain in its language, and the potential power of being able to do that. So it's different from the motor prosthetics where you're communicating from the brain to a device. Here we have to communicate from the outside world into the brain and be understood, and be understood by the brain.
这是仅仅是我们尝试的头一个 我想说一些关于普通假体的问题 它刚刚面世的时候 确实令人振奋 因为这能让一个失明的视网膜有反应 但是缺陷就 在于编码 怎样才能让细胞更好地反应 产生正常的反应 这就是我们作出的贡献 现在我总结一下 就像我之前说的那样 如果你感兴趣的话 我还有很多其他数据 这里我只简单地说一下 这种可以用自己的语言 和大脑交流的方法 以及这个方法的潜在力量 这跟运动修复术不同 运动修复术是把大脑的指令 传递给一个设备 而我们是要把外界信息 输入大脑 并且要使其被大脑理解
And then the last thing I wanted to say, really, is to emphasize that the idea generalizes. So the same strategy that we used to find the code for the retina we can also use to find the code for other areas, for example, the auditory system and the motor system, so for treating deafness and for motor disorders. So just the same way that we were able to jump over the damaged circuitry in the retina to get to the retina's output cells, we can jump over the damaged circuitry in the cochlea to get the auditory nerve, or jump over damaged areas in the cortex, in the motor cortex, to bridge the gap produced by a stroke.
最后我想 强调一下的 是这个方法是广泛适用的 我们用来研究视网膜编码的技术 同样也可以 用于研究其他领域 比如听觉系统 和运动系统,已解决听力障碍 和四肢运动障碍 其原理跟我们 绕过受损的视网膜电路系统 直接进入视网膜输出细胞一样 我们也可以绕过 损坏的耳蜗电路系统 直达听神经 或者绕开运动皮质里坏死的区域 来消除中风带来的 损害
I just want to end with a simple message that understanding the code is really, really important, and if we can understand the code, the language of the brain, things become possible that didn't seem obviously possible before. Thank you.
最后 我想说掌握代码 非常非常重要 如果我们能掌握代码 也就是大脑的语言 一切都豁然开朗 谢谢
(Applause)
(掌声)