It's getting harder, isn't it, to spot real from fake, AI-generated from human-generated. With generative AI, along with other advances in deep fakery, it doesn't take many seconds of your voice, many images of your face, to fake you, and the realism keeps increasing.
有没有觉得辨别真伪、分辨 AI 还是人类生成的结果越来越难了? 有了生成式 AI, 加上深度伪造领域的其他进步, 不需要几秒钟你的声音、 几张你的照片 就能假扮你, 还越来越逼真。
I first started working on deepfakes in 2017, when the threat to our trust in information was overhyped, and the big harm, in reality, was falsified sexual images. Now that problem keeps growing, harming women and girls worldwide. But also, with advances in generative AI, we're now also approaching a world where it's broadly easier to make fake reality, but also to dismiss reality as possibly faked.
我从 2017 年开始研究深度伪造, 当时我们对信息信任的威胁被夸大了, 其实最大的危害 是伪造的色情图片。 这个问题不断加剧, 危害着全世界的妇女和女孩。 但是,随着生成式 AI 的进步, 我们现在也进入了一个 更容易制造虚假现实的世界, 但也更容易将现实 误认为是虚假的。
Now, deceptive and malicious audiovisual AI is not the root of our societal problems, but it's likely to contribute to them. Audio clones are proliferating in a range of electoral contexts. "Is it, isn't it" claims cloud human-rights evidence from war zones, sexual deepfakes target women in public and in private, and synthetic avatars impersonate news anchors.
欺骗性和恶意的视听 AI 不是我们社会问题的根源, 但它很可能会助长这些问题。 音频克隆在各种选举场合中激增。 “是吗,不是吗” 的说法 掩盖了战区的人权证据, 性深度伪造不分公私地针对女性, 合成形象冒充新闻主播。
I lead WITNESS. We're a human-rights group that helps people use video and technology to protect and defend their rights. And for the last five years, we've coordinated a global effort, "Prepare, Don't Panic," around these new ways to manipulate and synthesize reality, and on how to fortify the truth of critical frontline journalists and human-rights defenders.
我是首席证人。 我们是一个人权组织, 帮助人们使用视频和技术 保护和捍卫自己的权利。 在过去的五年中, 我们组织了一场全球活动, 名为“做好准备,不要惊慌”, 围绕这些操纵和合成现实的新方式, 以及如何巩固如履薄冰的一线记者和 人权捍卫者带来的真相。
Now, one element in that is a deepfakes rapid-response task force, made up of media-forensics experts and companies who donate their time and skills to debunk deepfakes and claims of deepfakes. The task force recently received three audio clips, from Sudan, West Africa and India. People were claiming that the clips were deepfaked, not real. In the Sudan case, experts used a machine-learning algorithm trained on over a million examples of synthetic speech to prove, almost without a shadow of a doubt, that it was authentic. In the West Africa case, they couldn't reach a definitive conclusion because of the challenges of analyzing audio from Twitter, and with background noise.
其中一个部分是 深度伪造快速反应工作组, 由媒体取证专家和愿意付出 时间和技能的公司组成, 揭穿深度伪造和对深度伪造的指控。 工作组最近收到了三段 来自苏丹、西非和印度的录音片段。 人们声称这些片段 是深度伪造的,不是真实的。 在苏丹的案例中, 专家们使用了一种机器学习算法, 由超过一百万个合成语音样本训练而成, 几乎毋庸置疑地证明了它是真实的。 以西非为例, 他们无法得出明确的结论, 因为分析来自推特的音频有挑战, 而且有背景噪音。
The third clip was leaked audio of a politician from India. Nilesh Christopher of “Rest of World” brought the case to the task force. The experts used almost an hour of samples to develop a personalized model of the politician's authentic voice. Despite his loud and fast claims that it was all falsified with AI, experts concluded that it at least was partially real, not AI. As you can see, even experts cannot rapidly and conclusively separate true from false, and the ease of calling "that's deepfaked" on something real is increasing.
第三段是一位印度政客泄露的音频。 《世界其他地区》的尼莱什·克里斯托弗 (Nilesh Christopher) 向工作组提交了这个案件。 专家们使用了将近一个小时的样本 开发出了政客真实声音的个性化模型。 尽管他大张旗鼓地迅速声称 这些都是用 AI 伪造的, 但专家得出的结论是, 这至少是部分真实的,而不是 AI。 如你所见, 即使是专家也无法迅速 又言之凿凿地分辨真伪。 而且对真实事物说 “那是深度伪造的” 也越来越容易了。
The future is full of profound challenges, both in protecting the real and detecting the fake. We're already seeing the warning signs of this challenge of discerning fact from fiction. Audio and video deepfakes have targeted politicians, major political leaders in the EU, Turkey and Mexico, and US mayoral candidates. Political ads are incorporating footage of events that never happened, and people are sharing AI-generated imagery from crisis zones, claiming it to be real.
未来充满了重大的挑战, 无论是在保护真实还是检测虚假上。 我们已经看到了区分事实与虚构 这一挑战的预警信号。 音频和视频深度伪造的目标是政客、 欧盟、土耳其和墨西哥的主要政治领导人 以及美国市长候选人。 政治广告中包含了 从未发生过的事件的片段, 人们正在分享危机地区的 AI 生成的图像, 声称这些图像是真实的。
Now, again, this problem is not entirely new. The human-rights defenders and journalists I work with are used to having their stories dismissed, and they're used to widespread, deceptive, shallow fakes, videos and images taken from one context or time or place and claimed as if they're in another, used to share confusion and spread disinformation. And of course, we live in a world that is full of partisanship and plentiful confirmation bias.
同样,这并不是一个全新的问题。 我共事的人权捍卫者和记者 习惯于他们的报道被无视, 习惯于广为流传、 欺骗性的、肤浅的赝品、 从一个环境、时间或地点 拍摄的视频和图像, 再声称它们取自 另一个环境、时间或地点, 习惯于混淆视听、散布虚假信息。 当然,我们生活在一个充满党派偏见 和大量证实偏差的世界中。
Given all that, the last thing we need is a diminishing baseline of the shared, trustworthy information upon which democracies thrive, where the specter of AI is used to plausibly believe things you want to believe, and plausibly deny things you want to ignore.
鉴于此, 我们最不需要的就是不断降低 民主赖以蓬勃发展的共享、 可信信息的底线, AI 这个幕后黑手 会被用于让你名正言顺地 相信你想相信的事, 名正言顺地否认你想无视的事。
But I think there's a way we can prevent that future, if we act now; that if we "Prepare, Don't Panic," we'll kind of make our way through this somehow. Panic won't serve us well. [It] plays into the hands of governments and corporations who will abuse our fears, and into the hands of people who want a fog of confusion and will use AI as an excuse.
但我认为,有这么一个方法 可以让我们防止这样的未来出现, 如果我们现在就采取行动, 如果我们“做好准备,不要惊慌”, 我们就能以某种方式度过难关。 恐慌对我们没有好处。 (它)落入了政府和企业的手中, 他们会滥用我们的恐惧, 也落入了那些想要混乱的迷雾 并以人工智能为借口的人手中。
How many people were taken in, just for a minute, by the Pope in his dripped-out puffer jacket? You can admit it.
有多少人信以为真, 至少相信了那么一下, 教皇穿着炫酷的羽绒服? 你就承认吧。
(Laughter)
(笑声)
More seriously, how many of you know someone who's been scammed by an audio that sounds like their kid? And for those of you who are thinking "I wasn't taken in, I know how to spot a deepfake," any tip you know now is already outdated. Deepfakes didn't blink, they do now. Six-fingered hands were more common in deepfake land than real life -- not so much. Technical advances erase those visible and audible clues that we so desperately want to hang on to as proof we can discern real from fake.
更严重的是, 有多少人有认识的人 被像他们孩子的语音骗过? 对于那些在想“我可没被骗, 我知道怎么分辨深度伪造”的人来说, 你知道的各种小知识已经过时了。 深度伪造以前不会眨眼, 但现在会眨眼。 六根手指的手在深度伪造的世界 比现实生活常见多了, 也不见得如此。 技术进步抹去了 那些可看见、可听见的痕迹, 这些我们迫切希望抓住的救命稻草, 作为我们可以辨别真假的证据。
But it also really shouldn’t be on us to make that guess without any help. Between real deepfakes and claimed deepfakes, we need big-picture, structural solutions. We need robust foundations that enable us to discern authentic from simulated, tools to fortify the credibility of critical voices and images, and powerful detection technology that doesn't raise more doubts than it fixes.
但我们也不该独自 承担做出判断的责任。 在真正的深度伪造 和声称的深度伪造之间, 我们需要宏观的、 结构化的解决方案。 我们需要坚实的基础, 使我们区分真实和仿真, 我们需要增强重要声音 和图像可信度的工具 以及不会引起更多疑虑的 强大检测技术。
There are three steps we need to take to get to that future. Step one is to ensure that the detection skills and tools are in the hands of the people who need them. I've talked to hundreds of journalists, community leaders and human-rights defenders, and they're in the same boat as you and me and us. They're listening to the audio, trying to think, "Can I spot a glitch?" Looking at the image, saying, "Oh, does that look right or not?" Or maybe they're going online to find a detector. And the detector they find, they don't know whether they're getting a false positive, a false negative, or a reliable result.
要走向这样的未来, 我们需要采取三个步骤。 第一步是确保检测技能和工具 掌握在有需要的人手中。 我已经与数百名记者、 社区领袖和人权捍卫者进行了对话, 他们与你、我和我们不谋而合。 他们听音频的时候会想: “我能听出破绽吗?” 看着图片,说: “哦,看起来对不对?” 或者他们也许要上网找一个检测器。 而他们找到的检测器, 他们不知道自己得到的是假阳性、假阴性 还是可靠的结果。
Here's an example. I used a detector, which got the Pope in the puffer jacket right. But then, when I put in the Easter bunny image that I made for my kids, it said that it was human-generated. This is because of some big challenges in deepfake detection. Detection tools often only work on one single way to make a deepfake, so you need multiple tools, and they don't work well on low-quality social media content. Confidence score, 0.76-0.87, how do you know whether that's reliable, if you don't know if the underlying technology is reliable, or whether it works on the manipulation that is being used? And tools to spot an AI manipulation don't spot a manual edit.
举个例子。 我用了检测器, 它查出了穿着羽绒服的教皇。 但是,当我输入我为孩子们制作的 复活节兔子照片时, 它说它是人类生成的。 这是因为深度伪造检测 存在一些重大挑战。 检测工具通常只能用于 检测制作深度伪造的一种途径, 因此你需要多种工具, 而且它们不能很好地处理 低质量的社交媒体内容。 置信度分数为 0.76 到 0.87, 你该如何得知是否可靠, 如果你都不知道底层技术是否可靠, 或者它受到了操控? 而且识别 AI 操控的工具 无法识别手动编辑。
These tools also won't be available to everyone. There's a trade-off between security and access, which means if we make them available to anyone, they become useless to everybody, because the people designing the new deception techniques will test them on the publicly available detectors and evade them. But we do need to make sure these are available to the journalists, the community leaders, the election officials, globally, who are our first line of defense, thought through with attention to real-world accessibility and use. Though at the best circumstances, detection tools will be 85 to 95 percent effective, they have to be in the hands of that first line of defense, and they're not, right now.
这些工具也不是 所有人都可以使用的。 安全性和可访问性之间 需要权衡取舍, 意味着如果我们 让任何人都能使用它们, 它们就会对所有人都毫无用处, 因为设计新的欺骗技术的人 会在公开可用的检测器上 对其进行测试, 找到钻空子的方法。 但是,我们确实需要确保 全球的记者、社区领袖、 选举官员都能获得这些信息, 他们是我们的第一道防线, 慎重考虑现实世界中的可访问性和用途。 尽管在最乐观的情况下, 检测工具的有效性 可以达到 85% 至 95%, 但是它们必须被掌握在 第一道防线的手中, 然而现实并非如此。
So for step one, I've been talking about detection after the fact. Step two -- AI is going to be everywhere in our communication, creating, changing, editing. It's not going to be a simple binary of "yes, it's AI" or "phew, it's not." AI is part of all of our communication, so we need to better understand the recipe of what we're consuming.
第一步,我一直在谈论事后检测。 第二步,AI 将在 我们的沟通中无处不在, 创建、修改、编辑。 它不是“是的,是 AI” 或 “哦,不是”的非黑即白。 AI 是我们所有沟通的一部分, 所以我们得更好地了解 我们究竟在使用什么。
Some people call this content provenance and disclosure. Technologists have been building ways to add invisible watermarking to AI-generated media. They've also been designing ways -- and I've been part of these efforts -- within a standard called the C2PA, to add cryptographically signed metadata to files. This means data that provides details about the content, cryptographically signed in a way that reinforces our trust in that information. It's an updating record of how AI was used to create or edit it, where humans and other technologies were involved, and how it was distributed. It's basically a recipe and serving instructions for the mix of AI and human that's in what you're seeing and hearing. And it's a critical part of a new AI-infused media literacy.
有人称之为“内容来源和披露”。 技术专家们一直在研究 向 AI 生成的媒体 添加不可见水印的方法。 他们还在设计一种方法, 我也参与其中, 在符合 C2PA 这一标准的前提下, 将经过加密签名的元数据 加入文件中。 也就是说, 提供内容详细信息的数据, 经过了加密签名, 借此增强我们对该信息的信任。 它是一份不断更新的记录,记下了 AI 如何被用于创建、编辑内容, 人类和其他技术如何参与其中, 它是如何被传播的。 它就是一份食谱和使用说明, 描述的对象就是你所见所闻的 AI 和人类的融合。 这是 AI 参与的 新型媒体素养的关键部分。
And this actually shouldn't sound that crazy. Our communication is moving in this direction already. If you're like me -- you can admit it -- you browse your TikTok “For You” page, and you're used to seeing videos that have an audio source, an AI filter, a green screen, a background, a stitch with another edit. This, in some sense, is the alpha version of this transparency in some of the major platforms we use today. It's just that it does not yet travel across the internet, it’s not reliable, updatable, and it’s not secure.
其实听起来没有那么离谱。 我们的沟通正在朝这个方向转变。 如果你和我一样,就承认吧, 你在浏览抖音的“为你推荐”页面, 你都看惯了带有音频源、 AI 滤镜、绿屏、背景、 加入混剪的视频。 从某种意义上说, 这是我们当今使用的 一些主流平台展现透明度的初级版本。 只是它还没有在互联网上传播, 它不可靠、不可更新,而且不安全。
Now, there are also big challenges in this type of infrastructure for authenticity. As we create these durable signs of how AI and human were mixed, that carry across the trajectory of how media is made, we need to ensure they don't compromise privacy or backfire globally. We have to get this right.
这类基础架构在真实性方面 也面临着巨大的挑战。 当我们留下了 AI 和人类结合的持久记录时, 遍布媒体制作的整个过程, 我们得确保它们不会在全球范围内 侵害隐私或适得其反。 我们必须做对这一点。
We can't oblige a citizen journalist filming in a repressive context or a satirical maker using novel gen-AI tools to parody the powerful ... to have to disclose their identity or personally identifiable information in order to use their camera or ChatGPT. Because it's important they be able to retain their ability to have anonymity, at the same time as the tool to create is transparent. This needs to be about the how of AI-human media making, not the who.
我们不能强迫公民记者 在受压迫的环境下拍摄, 也不能强迫讽刺作者 使用新型的生成式 AI 工具 模仿有权有势的人... 强迫披露他们的身份 或个人身份信息 才能使用他们的相机或 ChatGPT。 因为在保证创造的工具是透明的同时, 保障保持匿名的权利非常重要。 必须有关于 AI-人类创作媒体的方式, 而不是谁去创作。
This brings me to the final step. None of this works without a pipeline of responsibility that runs from the foundation models and the open-source projects through to the way that is deployed into systems, APIs and apps, to the platforms where we consume media and communicate.
这就说到了最后一步。 如果没有责任链, 包括基础模型、开源项目 直至部署到系统、API 和应用程序、 我们使用媒体、用于交流的平台, 所有这些都行不通。
I've spent much of the last 15 years fighting, essentially, a rearguard action, like so many of my colleagues in the human rights world, against the failures of social media. We can't make those mistakes again in this next generation of technology. What this means is that governments need to ensure that within this pipeline of responsibility for AI, there is transparency, accountability and liability.
在过去的 15 年中,我的大部分时间 投入了最后一搏, 如同我在人权届的很多同志们, 抗争社交媒体的失败。 我们不能再在下一代技术中犯这些错误。 这意味着政府 必须确保在 AI 的责任链上, 有透明度、职责义务、法律责任。
Without these three steps -- detection for the people who need it most, provenance that is rights-respecting and that pipeline of responsibility, we're going to get stuck looking in vain for the six-fingered hand, or the eyes that don't blink. We need to take these steps. Otherwise, we risk a world where it gets easier and easier to both fake reality and dismiss reality as potentially faked.
如果没有这三个步骤—— 为最需要的人提供检测器、 尊重权利的来源、 责任链, 我们就是在徒劳地寻找 六根手指的手、不会眨眼的眼睛。 我们必须采取这些步骤。 否则,我们就会有风险 看见这样的世界,越来越容易 捏造现实, 又把现实当作可能是伪造的。
And that is a world that the political philosopher Hannah Arendt described in these terms: "A people that no longer can believe anything cannot make up its own mind. It is deprived not only of its capacity to act but also of its capacity to think and to judge. And with such a people you can then do what you please." That's a world I know none of us want, that I think we can prevent.
政治哲学家汉娜·阿伦特 (Hannah Arendt) 这样描述了这个世界: “一个再也无法相信任何东西的人 无法下定决心。 他/她不仅被剥夺了行动能力, 还被剥夺了思考和判断的能力。 这样的人能让你随心所欲。” 我知道这是我们都不想见到的世界, 我认为我们可以避免让它来临。
Thanks.
谢谢。
(Cheers and applause)
(欢呼和掌声)