Power. That is the word that comes to mind. We're the new technologists. We have a lot of data, so we have a lot of power. How much power do we have? Scene from a movie: "Apocalypse Now" -- great movie. We've got to get our hero, Captain Willard, to the mouth of the Nung River so he can go pursue Colonel Kurtz. The way we're going to do this is fly him in and drop him off. So the scene: the sky is filled with this fleet of helicopters carrying him in. And there's this loud, thrilling music in the background, this wild music. ♫ Dum da ta da dum ♫ ♫ Dum da ta da dum ♫ ♫ Da ta da da ♫ That's a lot of power. That's the kind of power I feel in this room. That's the kind of power we have because of all of the data that we have.
能力 是我们时时在想的 我们是新时代的技术人员 我们有很多信息,这样我们就有能力 多少能力呢? 想像电影“现代启示录”, 这电影绝了 里面有个英雄,卫尔沃德上尉,来到怒河的入海口 去追踪科特兹上校 想像他飞到那里,下飞机 想像这个情景: 天空布满了陪他来的直升飞机 背景音乐震人心肺 非常野性 ♫ Dum da ta da dum ♫ ♫ Dum da ta da dum ♫ ♫ Da ta da da ♫ 充满了力量 这就是我在这个房间里感觉到的能量 这就是我们拥有的能量 就因为我们拥有信息
Let's take an example. What can we do with just one person's data? What can we do with that guy's data? I can look at your financial records. I can tell if you pay your bills on time. I know if you're good to give a loan to. I can look at your medical records; I can see if your pump is still pumping -- see if you're good to offer insurance to. I can look at your clicking patterns. When you come to my website, I actually know what you're going to do already because I've seen you visit millions of websites before. And I'm sorry to tell you, you're like a poker player, you have a tell. I can tell with data analysis what you're going to do before you even do it. I know what you like. I know who you are, and that's even before I look at your mail or your phone.
举个例子 用个人信息 我们能做什么呢? 用这位先生的信息 我们能做什么? 我可以查看你的财政历史 看你是不是按时付账单 来决定给不给你贷款 我可以浏览你的医疗病历,看你的心脏好不好 来决定给不给你保险 我可以跟踪你上网的习惯 当你访问我的网站时,我已经知道你会怎么做了 因为我已经看到你以前是怎么访问成千上万个网站的 我不得不说 你就像是个赌客,你能被看透 就凭你的信息我能看透你将会做什么 在你还没做之前 我知道你喜欢什么,我知道你是谁 在我看你的邮件之前 看你的电话之前
Those are the kinds of things we can do with the data that we have. But I'm not actually here to talk about what we can do. I'm here to talk about what we should do. What's the right thing to do?
现今我们能这样做了 凭着我们有的信息 但我来这里不是谈我们能做什么 我是来谈谈我们应该做什么 如何善用信息?
Now I see some puzzled looks like, "Why are you asking us what's the right thing to do? We're just building this stuff. Somebody else is using it." Fair enough. But it brings me back. I think about World War II -- some of our great technologists then, some of our great physicists, studying nuclear fission and fusion -- just nuclear stuff. We gather together these physicists in Los Alamos to see what they'll build. We want the people building the technology thinking about what we should be doing with the technology.
我看到一些人迷惑了 好像在想:“为什么你问我们应该如何善用信息? 我们只管把技术推出来,用是别人的事情。” 就算你对 但这正是我要说的 想想二次世界大战 很多我们最优秀的技术人员 最优秀的物理学家 在研究核裂变和核聚变 都是关于核的 我们把这些科学家请到洛斯阿拉莫斯国家实验室 来看看他们想用这些技术来做什么 我们希望,人们能够在发明新技术时 时时想着我们应该怎么用这些技术
So what should we be doing with that guy's data? Should we be collecting it, gathering it, so we can make his online experience better? So we can make money? So we can protect ourselves if he was up to no good? Or should we respect his privacy, protect his dignity and leave him alone? Which one is it? How should we figure it out?
话说回来,我们应该怎么用这位先生的信息? 我们应不应该收集他的信息 来使他的上网冲浪的体验更好? 我们应不应该用这些信息来赚钱? 应不应该用这些信息来自我保护 以防万一他干坏事? 还是我们应该尊重他的隐私 为他保留尊严,不要烦他? 哪一个办法更好? 怎么做决定?
I know: crowdsource. Let's crowdsource this. So to get people warmed up, let's start with an easy question -- something I'm sure everybody here has an opinion about: iPhone versus Android. Let's do a show of hands -- iPhone. Uh huh. Android. You'd think with a bunch of smart people we wouldn't be such suckers just for the pretty phones. (Laughter) Next question, a little bit harder. Should we be collecting all of that guy's data to make his experiences better and to protect ourselves in case he's up to no good? Or should we leave him alone? Collect his data. Leave him alone. You're safe. It's fine. (Laughter) Okay, last question -- harder question -- when trying to evaluate what we should do in this case, should we use a Kantian deontological moral framework, or should we use a Millian consequentialist one? Kant. Mill. Not as many votes. (Laughter) Yeah, that's a terrifying result. Terrifying, because we have stronger opinions about our hand-held devices than about the moral framework we should use to guide our decisions.
我知道:集思广益。让我们来听听大家的意见 先来个热身题 简单的问题—— 这个问题我相信大家都有倾向 苹果电话还是谷歌Android电话? 让我们举手表决,支持苹果的 恩 现在支持谷歌Android的 原以为这里都是聪明人 不会只为了漂亮徒有其表而盲目消费。 (笑声) 下一个问题, 有点难 我们应不应该收集这位先生的信息 来让他的体验更好 同时来保护我们自身,万一他干坏事? 还是我们应该不管他? 支持收集信息的 支持不管他的 (这位先生)你是安全的 (笑声) 现在,最后一个问题—— 更难一点—— 当我们试图决定 我们应该怎么做的时候 是应该用康德的义务论道德框架, 还是用米利安的结果主义论? 支持康德的 支持米利安的 没什么人知道哦。 (笑声) 这是个惊人的结果 惊人,因为我们关心 手持设备的电话 比关心道德理论更多 我们应该用这些理论来指导我们的决定
How do we know what to do with all the power we have if we don't have a moral framework? We know more about mobile operating systems, but what we really need is a moral operating system. What's a moral operating system? We all know right and wrong, right? You feel good when you do something right, you feel bad when you do something wrong. Our parents teach us that: praise with the good, scold with the bad. But how do we figure out what's right and wrong? And from day to day, we have the techniques that we use. Maybe we just follow our gut. Maybe we take a vote -- we crowdsource. Or maybe we punt -- ask the legal department, see what they say. In other words, it's kind of random, kind of ad hoc, how we figure out what we should do. And maybe, if we want to be on surer footing, what we really want is a moral framework that will help guide us there, that will tell us what kinds of things are right and wrong in the first place, and how would we know in a given situation what to do.
如何决定我们该怎么来使用我们所拥有的能力 如果我们连道德框架都没有? 我们对移动电话操作系统知道得越来越多 但是我们需要知道的是道德的操作系统 什么是道德操作系统? 我们知道什么是对,什么是错 你做对了的时候,自我感觉挺好 如果你做错了你觉得自责 父母亲教导我们,奖对惩错 但是怎么才能知道什么是对什么是错? 尤其是当每天新技术都层出不穷 说不定我们可以就靠本能 说不定我们需要推选——群众意见集思广益 或者我们赌一把—— 问法律专家的意见,看他们怎么说 也就是说,我们作出决定的办法 很随机 很即兴 或者,如果我们想更保险些 只想要一个好的道德框架,我们能用来自己弄明白的 能帮我们明辨对错 帮我们决定什么情况下做什么
So let's get a moral framework. We're numbers people, living by numbers. How can we use numbers as the basis for a moral framework? I know a guy who did exactly that. A brilliant guy -- he's been dead 2,500 years. Plato, that's right. Remember him -- old philosopher? You were sleeping during that class. And Plato, he had a lot of the same concerns that we did. He was worried about right and wrong. He wanted to know what is just. But he was worried that all we seem to be doing is trading opinions about this. He says something's just. She says something else is just. It's kind of convincing when he talks and when she talks too. I'm just going back and forth; I'm not getting anywhere. I don't want opinions; I want knowledge. I want to know the truth about justice -- like we have truths in math. In math, we know the objective facts. Take a number, any number -- two. Favorite number. I love that number. There are truths about two. If you've got two of something, you add two more, you get four. That's true no matter what thing you're talking about. It's an objective truth about the form of two, the abstract form. When you have two of anything -- two eyes, two ears, two noses, just two protrusions -- those all partake of the form of two. They all participate in the truths that two has. They all have two-ness in them. And therefore, it's not a matter of opinion.
让我们找到一个道德框架 我们用很多数字,生活在数字中 怎么用数字 来建立一个道德框架? 我知道一个人专门干这个的 很聪明的人—— 2500年前就死了 柏拉图。对了 还记得他?老哲学家? 在他的教学课上,你可能睡过去了。 柏拉图,他有很多我们有的担心 他担心如何辨明是非 他希望知道怎么衡量 他担心我们现在做的 不过是交换意见 你一个主意我一个主意 听起来都很有理 我只是前瞻后顾,最后还是没结果 我不需要意见,我需要的是真理 我需要知道什么是有关正义的真理-- 就像我们研究什么是数学的真理一样 在数学中,我们使用客观的事实 比如将一个数字,任何数字——二 个人最爱 二有一些事实可讲 如果你有两个东西 你再加上两个,就是四个 这是真理,不管你讨论的东西是什么 这是关于二的客观真理 抽象性 当你有两个东西,任何东西——两只眼睛,两只耳朵,两个鼻子 或者仅仅是两个小突起—— 他们都是二的表现 都分享了关于二的事实 都有二在其中 所以,个人情绪没关系
What if, Plato thought, ethics was like math? What if there were a pure form of justice? What if there are truths about justice, and you could just look around in this world and see which things participated, partook of that form of justice? Then you would know what was really just and what wasn't. It wouldn't be a matter of just opinion or just appearances. That's a stunning vision. I mean, think about that. How grand. How ambitious. That's as ambitious as we are. He wants to solve ethics. He wants objective truths. If you think that way, you have a Platonist moral framework.
如果柏拉图考虑 道德,就像数学一样,会怎么样? 如果真有纯粹的公正,会怎么样? 如果公正真有绝对的事实, 你可以环顾四周 看能用在什么上 什么东西有着绝对的公正,会怎么样? 这样你就能知道什么是真的对,什么是真的错 和事情的外表 和个人的观点都没有关系 这是个诱人的观点 我是说,想象一下,多么壮观,多么充满雄心 这是我们最大的野心了 他希望解决道德的问题 他希望客观的事实 如果你这么想 你就有柏拉图式的道德观。
If you don't think that way, well, you have a lot of company in the history of Western philosophy, because the tidy idea, you know, people criticized it. Aristotle, in particular, he was not amused. He thought it was impractical. Aristotle said, "We should seek only so much precision in each subject as that subject allows." Aristotle thought ethics wasn't a lot like math. He thought ethics was a matter of making decisions in the here-and-now using our best judgment to find the right path. If you think that, Plato's not your guy. But don't give up. Maybe there's another way that we can use numbers as the basis of our moral framework.
如果你不这么认为 那你就在西方哲学史上有很多同僚 因为人们总爱挑剔这简明的观点。 特别是亚里士多德,他就不买帐 他觉得这不实际 亚里士多德说:“我们看每个物体 都只能精确到这个物体允许的程度。” 亚里士多德认为道德不像是数学 他认为道德是我在当时当地 作出的最好的判断 从而引导我们的决定 如果你这么想,柏拉图就不是你这边的了 但是别放弃 或者还有一条路 我们能用数字,作为我们道德观的基础。
How about this: What if in any situation you could just calculate, look at the choices, measure out which one's better and know what to do? That sound familiar? That's a utilitarian moral framework. John Stuart Mill was a great advocate of this -- nice guy besides -- and only been dead 200 years. So basis of utilitarianism -- I'm sure you're familiar at least. The three people who voted for Mill before are familiar with this. But here's the way it works. What if morals, what if what makes something moral is just a matter of if it maximizes pleasure and minimizes pain? It does something intrinsic to the act. It's not like its relation to some abstract form. It's just a matter of the consequences. You just look at the consequences and see if, overall, it's for the good or for the worse. That would be simple. Then we know what to do.
这个怎么样: 如果你能在任何情况下都算出 衡量所有的选择 算出怎么做更好,什么应该做? 听起来熟悉是不是? 这就是实用主义的道德观 约翰·斯图尔特·密尔是大支持者—— 很友善的一个人—— 可惜死了两百年了 所以实用主义的基础—— 我详细你们都略知一二 刚才三个米尔的支持者肯定知道 我还是讲讲这个怎么运作的 假如道德,道德的内涵 只是为了最大的享受 和最少的痛苦? 这在根本上决定了我们的行为 看起来和任何抽象的形式有关 只是一系列的必然结果 你只看必然结果 看总的来说是好结果还是坏结果 这看起来简单,然后我们就知道怎么做
Let's take an example. Suppose I go up and I say, "I'm going to take your phone." Not just because it rang earlier, but I'm going to take it because I made a little calculation. I thought, that guy looks suspicious. And what if he's been sending little messages to Bin Laden's hideout -- or whoever took over after Bin Laden -- and he's actually like a terrorist, a sleeper cell. I'm going to find that out, and when I find that out, I'm going to prevent a huge amount of damage that he could cause. That has a very high utility to prevent that damage. And compared to the little pain that it's going to cause -- because it's going to be embarrassing when I'm looking on his phone and seeing that he has a Farmville problem and that whole bit -- that's overwhelmed by the value of looking at the phone. If you feel that way, that's a utilitarian choice.
让我们看一个例子 假设我上前 说:“我要没收你的手机。” 不仅仅是因为刚才你的手机响了 而是因为我刚算计了一下 我觉得,这个人看起来可疑 说不定他是在给本拉登的藏身处发消息—— 或者是跟本拉登的接班人发消息—— 他看起来真像是恐怖分子,一个卧底 我得把他揪出来,当我揪他时 我就能防止他将带来一个大危害 防止这个危害的结果非常有意义 和我的行为将带来的小小痛苦相比—— 因为在我检查他的手机的时候 要是看到他只是在玩线上游戏,是挺丢人的 但是这不抵 检查手机能带来的价值 如果你这么想 就是实用主义的选择
But maybe you don't feel that way either. Maybe you think, it's his phone. It's wrong to take his phone because he's a person and he has rights and he has dignity, and we can't just interfere with that. He has autonomy. It doesn't matter what the calculations are. There are things that are intrinsically wrong -- like lying is wrong, like torturing innocent children is wrong. Kant was very good on this point, and he said it a little better than I'll say it. He said we should use our reason to figure out the rules by which we should guide our conduct, and then it is our duty to follow those rules. It's not a matter of calculation.
可能你也并不赞同这个做法 可能你会想,这是他的手机 拿他的手机是不对的 他是个人 他有人权,有尊严 我们不能干涉 他有自主权 不管计算结果是什么 这个举动在本质上是错误的—— 就像撒谎是错误的 像是折磨无辜的孩童是错误的 康德非常坚持这一点 他解释得更好一点 他说我们应该用我们自己的理由 来弄明白决定该怎么做的准则 接下来我们的责任是遵守这些准则 不是关于计算。
So let's stop. We're right in the thick of it, this philosophical thicket. And this goes on for thousands of years, because these are hard questions, and I've only got 15 minutes. So let's cut to the chase. How should we be making our decisions? Is it Plato, is it Aristotle, is it Kant, is it Mill? What should we be doing? What's the answer? What's the formula that we can use in any situation to determine what we should do, whether we should use that guy's data or not? What's the formula? There's not a formula. There's not a simple answer.
别再算了 我们正在这错综复杂的哲学中间 而且已经像这样上千年了 因为这些是很难的问题 我只有十五分钟 所以让我们直奔主题 我们应该怎么做决定? 是听从柏拉图,亚里士多德,康德,还是米尔? 我们该做什么?答案是什么? 能在任何情况下计算出我们应该怎么做的 万灵的公式在哪里? 我们该还是不该用这位先生的数据? 公式是什么? 没有公式 没有简单明了的答案
Ethics is hard. Ethics requires thinking. And that's uncomfortable. I know; I spent a lot of my career in artificial intelligence, trying to build machines that could do some of this thinking for us, that could give us answers. But they can't. You can't just take human thinking and put it into a machine. We're the ones who have to do it. Happily, we're not machines, and we can do it. Not only can we think, we must. Hannah Arendt said, "The sad truth is that most evil done in this world is not done by people who choose to be evil. It arises from not thinking." That's what she called the "banality of evil." And the response to that is that we demand the exercise of thinking from every sane person.
道德观是很难的 道德观需要思想 这是不太容易的 我知道。在我的职业生涯中我花了很多时间 研究机器人 试图造出机器来替我们做决定 来给我们答案 但是他们做不到 你不能就拿人的思维方式 放在机器里 我们要靠自己思考 好的方面是我们不是机器,我们能思考 不光能思考 我们必须思考 汉娜·阿伦特说过 “悲哀的是 世上大多数的罪恶 不是由有恶意的人 故意造成的 而是人没有想清楚。” 这就是她所谓的“平庸的罪恶” 对策是 我们需要练习思考 每个平常人都要
So let's do that. Let's think. In fact, let's start right now. Every person in this room do this: think of the last time you had a decision to make where you were worried to do the right thing, where you wondered, "What should I be doing?" Bring that to mind, and now reflect on that and say, "How did I come up that decision? What did I do? Did I follow my gut? Did I have somebody vote on it? Or did I punt to legal?" Or now we have a few more choices. "Did I evaluate what would be the highest pleasure like Mill would? Or like Kant, did I use reason to figure out what was intrinsically right?" Think about it. Really bring it to mind. This is important. It is so important we are going to spend 30 seconds of valuable TEDTalk time doing nothing but thinking about this. Are you ready? Go.
让我们来做吧,来思考 事实上,让我们现在就做 房间里每个人都这样做: 想像上一次你需要做一个决定的时候 当你担心不能做出正确的决定 当你怀疑:“到底该怎么做呢?” 想象那个时刻 之后再想 “我当时是怎么做决定的?” 我做了什么?是随我意愿的么? 征求他人意见了么?征求法律意见了么? 现在我们有更多选择 “我有没有像米尔会做的那样 衡量最大的享受? 或者像康德,借助原因来弄明白什么是本质上对的?” 想象一下,真的深入其中,这是很重要的 有多重要 我们将会用三十秒,非常宝贵的TED的时间 什么也不做,就思考 准备好了么?开始
Stop. Good work. What you just did, that's the first step towards taking responsibility for what we should do with all of our power.
结束,不错 你刚做的 是向着为我们使用我们的能力 负责任迈出的第一步
Now the next step -- try this. Go find a friend and explain to them how you made that decision. Not right now. Wait till I finish talking. Do it over lunch. And don't just find another technologist friend; find somebody different than you. Find an artist or a writer -- or, heaven forbid, find a philosopher and talk to them. In fact, find somebody from the humanities. Why? Because they think about problems differently than we do as technologists. Just a few days ago, right across the street from here, there was hundreds of people gathered together. It was technologists and humanists at that big BiblioTech Conference. And they gathered together because the technologists wanted to learn what it would be like to think from a humanities perspective. You have someone from Google talking to someone who does comparative literature. You're thinking about the relevance of 17th century French theater -- how does that bear upon venture capital? Well that's interesting. That's a different way of thinking. And when you think in that way, you become more sensitive to the human considerations, which are crucial to making ethical decisions.
现在第二步——试试这个 找到一个朋友,向他们解释 你是怎么做决定的 不是现在,等我讲完了之后。 午饭的时候做 别找另一个搞技术的朋友 找一个和你很不同的 艺术家或者作家—— 或者,哲学家,和他们谈 事实上找个人文主义者谈 为什么?因为他们考虑问题的角度 和我们技术人员是很不一样的 仅仅几天前,街对面 成百上千人聚集起来 都是技术人员和人文主义者 在开一个大的研讨会 他们聚集起来 因为技术人员想了解 从人文主义者的角度想是怎么样的 有从谷歌来的人 和研究比较文学的人谈 你在想十七世纪的法国戏剧的影响—— 是怎么在风险投资下存活的? 这很有趣,是另一个角度看问题。 我们在这样想的时候 就开始对人类的考量更敏感 这对于做出道德的决定是很重要的
So imagine that right now you went and you found your musician friend. And you're telling him what we're talking about, about our whole data revolution and all this -- maybe even hum a few bars of our theme music. ♫ Dum ta da da dum dum ta da da dum ♫ Well, your musician friend will stop you and say, "You know, the theme music for your data revolution, that's an opera, that's Wagner. It's based on Norse legend. It's Gods and mythical creatures fighting over magical jewelry." That's interesting. Now it's also a beautiful opera, and we're moved by that opera. We're moved because it's about the battle between good and evil, about right and wrong. And we care about right and wrong. We care what happens in that opera. We care what happens in "Apocalypse Now." And we certainly care what happens with our technologies.
想像现在 你去找个音乐家的朋友 和他说我们现在的话题 这些信息革命等等—— 甚至哼哼我们的一些电影主题音乐。 ♫ Dum ta da da dum dum ta da da dum ♫ 你的音乐家朋友会打断你说: “你知道,你技术革命的 主题音乐 是个戏剧,是瓦格纳的作品。 关于一个北欧传说 是关于上帝和神话人物 为了魔术珍宝而争斗。” 这很有趣 现在这也是个动人的戏剧 我们都被感动了 因为这是关于一场战争 善与恶之间的 对与错之间的 我们关心对和错 我们关心在戏剧里发生了什么 我们关心在“现代启示录”里发生了什么 我们也关心 用我们的技术会发生什么
We have so much power today, it is up to us to figure out what to do, and that's the good news. We're the ones writing this opera. This is our movie. We figure out what will happen with this technology. We determine how this will all end.
当下我们有这么多的能力 这是我们的责任来搞清楚该怎么做。 这是好消息 我们是写剧本的人 这就是我们的电影 我们清楚这个技术将会带给我们什么。 我们认为事情会这样结束
Thank you.
谢谢
(Applause)
(掌声)