I'm James. I'm a writer and artist, and I make work about technology. I do things like draw life-size outlines of military drones in city streets around the world, so that people can start to think and get their heads around these really quite hard-to-see and hard-to-think-about technologies. I make things like neural networks that predict the results of elections based on weather reports, because I'm intrigued about what the actual possibilities of these weird new technologies are. Last year, I built my own self-driving car. But because I don't really trust technology, I also designed a trap for it.
من .جهیمسم نوسهرو هونهرمهندم .و لە بواری تهكنهلۆجیا كار دهكهم وێنەی ئامێرە سەربازییەکان دەکێشم ،له شهقامهكانی شارەکانیی جیهان بۆ ئەوەی خەڵک بیربكهنه وه و سەیری دهوروبهریان بكهن بۆ ئەو تەکنەلۆژیایەی کە قورسە .ببینرێت و بیری لێبکرێتەوە شهبهكهی نیوراڵ دروست دهكهم كه ئهنجامی ههڵبژاردن پێشبینی دهكات ،لهسهر بنهمای ڕاپۆرتی ئاو ههوا چونكه خۆم بۆ مەڵاس دابوو ئهگهره ڕاستهقینهكانی .ئهو تهكنهلۆجیا سهیرانه چین پار، ئۆتۆمبێلێکی خۆلێخۆری .تایبەت بە خۆم دروست کرد بهڵام لهبهرئهوهی بڕوام ،به تهكنهلۆجیا نییه .تهلێكم بۆی دروستكرد
(Laughter)
(پێكهنین)
And I do these things mostly because I find them completely fascinating, but also because I think when we talk about technology, we're largely talking about ourselves and the way that we understand the world. So here's a story about technology.
زۆر کات ئهو شتانه دهكهم چونكه چێژێکی ،تهواوی لێ دەبینم بهڵام پێموایه کاتێک سەبارەت ،بە تهكنهلۆجیا دهدوێین بهزۆری دهربارهی خۆمان دەدوێین .و چۆن لە جیهان تێدەگەین چیرۆكێك دهربارهی تهكنهلۆجیا .ههیه
This is a "surprise egg" video. It's basically a video of someone opening up loads of chocolate eggs and showing the toys inside to the viewer. That's it. That's all it does for seven long minutes. And I want you to notice two things about this. First of all, this video has 30 million views.
.ئهوه ڤیدیۆی " هێلكهی سهرسامکار"ه ڤیدیۆی كهسێكه هێلكه چوكلاتیهكانی دهكاتهوه له ناویدا یاری یپێشانی بینهران .دهدات تەواو. بۆ حهوت .خولهك تەها ئەوەندە دهكات .دهمهوێ لێرهدا تێبینی دوو شت بكهن پێش ههموشت، ئهو ڤیدیۆیه .٣٠ملیۆن بینهری ههیه
(Laughter)
(پێكهنین)
And the other thing is, it comes from a channel that has 6.3 million subscribers, that has a total of eight billion views, and it's all just more videos like this -- 30 million people watching a guy opening up these eggs. It sounds pretty weird, but if you search for "surprise eggs" on YouTube, it'll tell you there's 10 million of these videos, and I think that's an undercount. I think there's way, way more of these. If you keep searching, they're endless. There's millions and millions of these videos in increasingly baroque combinations of brands and materials, and there's more and more of them being uploaded every single day. Like, this is a strange world. Right?
،شتهكهی دیكهش له كهناڵێكهوه وهرگیراوه ،٦.٣ ملیۆن سەبسکرایبەری ههیه و بینهری سهرجەم ،ڤیدیۆکانی ههشت بلیۆنه و زۆر ڤیدیۆی تری هاوشێوهی ئهوهی تیایە ٣٠ملیۆن خهڵك سهیری ئهو .كوڕه دهكات كه هێلكهكه دهكاتهوه ئهوه سهیر دهردهكهوێت، بهڵام ئهگهر ،له یوتیوب بۆ"هێلكهی سهرسامی" بگهڕێیت ئهوا ١٠ ملیۆن ،لهو جۆره ڤیدیۆیانهت پێدهڵێت وا بزانم ڕوو له زیاد .بوونه .پێموایه ڕێگەی زیاتر بۆ ئهوه ههیه .ئهگهر بهدوای بگهڕێی، بێ كۆتان ملیۆنان و ملیۆنان ڤیدیۆی ئاوا ،كه ڕێژەیەکی زۆر مارکە و مادەی ههیه و زیاترو زیاتر لهوجۆرانه .ڕۆژانه بڵاو دهكرێنهو ئهوه جیهانێكی سهیره، ڕاسته؟
But the thing is, it's not adults who are watching these videos. It's kids, small children. These videos are like crack for little kids. There's something about the repetition, the constant little dopamine hit of the reveal, that completely hooks them in. And little kids watch these videos over and over and over again, and they do it for hours and hours and hours. And if you try and take the screen away from them, they'll scream and scream and scream. If you don't believe me -- and I've already seen people in the audience nodding -- if you don't believe me, find someone with small children and ask them, and they'll know about the surprise egg videos. So this is where we start. It's 2018, and someone, or lots of people, are using the same mechanism that, like, Facebook and Instagram are using to get you to keep checking that app, and they're using it on YouTube to hack the brains of very small children in return for advertising revenue.
بهڵام شتكه ئهوهیه، ئهوانهی .سهیری دهكهن گەورە نین منداڵن، منداڵی .بچوك .ئهو ڤیدیۆیانه بۆ منداڵ شتێکی خۆشە ،شتێك ههیه دهربارهی دوبارهكردنهوه ،گەڕانەوەی پێشی سەرشێتانەی ڤیدیۆکان .ئهوانه بهشێوهیهكی گشتی ئالودەن منداڵی بچوك ئهو ڤیدیۆیانه ،چهند جارێك دوباره سهیر دهكهن بۆ چهند كاتژمێرێك و .كاتژمێرێك ئهوه دهكهن ،ئهگهر تۆ شاشهكهیان لێ بگۆڕی هاوار دهكهن وهاوار .دهكهن ئهگهر بڕوا بهمن ناكهی خهڵكم بینیوه له ئامادهبووهكان سهر دهلهقێنن ئهگهر باوهڕناكهی له منداڵێك ،و كهسێك بپرسه و دهربارهی ڤیدیۆی .هێلكهی سهرسامی دهزانن .لێرەوە دەستپێدەکەین ،ئهوه ٢٠١٨ یه، كهسێك یا چهندین كهس ههمان ڕێچکە، وهك، فهیسبوك و ئینستگرام بهكار دێنن ،تا بهردهوام بهرنامهكه بهكاربێنن و له یوتیوب بهكاری دێنن تا مێشكی منداڵی بچوك هاك بكهن .بۆ بەدەست هێنانی قازانج لە رێگەی رێکلام
At least, I hope that's what they're doing. I hope that's what they're doing it for, because there's easier ways of making ad revenue on YouTube. You can just make stuff up or steal stuff. So if you search for really popular kids' cartoons like "Peppa Pig" or "Paw Patrol," you'll find there's millions and millions of these online as well. Of course, most of them aren't posted by the original content creators. They come from loads and loads of different random accounts, and it's impossible to know who's posting them or what their motives might be. Does that sound kind of familiar? Because it's exactly the same mechanism that's happening across most of our digital services, where it's impossible to know where this information is coming from. It's basically fake news for kids, and we're training them from birth to click on the very first link that comes along, regardless of what the source is. That's doesn't seem like a terribly good idea.
بهلایهنی كهمەوە .هیوا دهخوازم ئەم کارە بکەن ،هیوا دهخوازم بۆ قازانج ئەم کارە بکەن چونكه ڕێگەی ئاسانتر له یوتیوب .بۆ بەدەست هێنانی داهات ڕیكلام هەیە .دهتوانی شتهكان دروست بکەی یا بیدزی ئهگهر بهدوای كارتۆنی بهناوبانگی منداڵان بگهڕێی ،"وهك "پیپا پیگ" یا "پاو پتڕۆڵ ههروهها به ملیۆنهها .ڤیدیۆی سەرهێڵی دهدۆزیهوه بێگومان، زۆیان لهلایهن دروستكهری .سهرهكی بڵاو نهكراونهتهوه له ئهكاونته ههڕهمهكه ،جیاوازهكانهوه دێن كه مهحاڵه بزانی كێ بڵاویان دهكاتهوه .و پاڵنهرهكهیان چییه ئاشنان بەمە؟ چونكه ئهوه ههمان میكانزمه كه له خزمهتگوزاری ژمێریاری ،زۆربهمان ڕوودهدات ئەستەمە بزانیت ئهو .زانیاریانه لهكوێوه دێن ،ههواڵێكی فێڵاویه بۆ منداڵ و ڕاهینانیان پێدهكهین له لهدایك بوونهوه بۆئهوهی كلیك لهسهریهكهم لینك ،بكات دیته پێشی .بهبێ گوێدان بهسهرچاوهكهی .لهوه ناچێت ئهوه بیرۆكهیهكی باش بێت
Here's another thing that's really big on kids' YouTube. This is called the "Finger Family Song." I just heard someone groan in the audience. This is the "Finger Family Song." This is the very first one I could find. It's from 2007, and it only has 200,000 views, which is, like, nothing in this game. But it has this insanely earwormy tune, which I'm not going to play to you, because it will sear itself into your brain in the same way that it seared itself into mine, and I'm not going to do that to you. But like the surprise eggs, it's got inside kids' heads and addicted them to it. So within a few years, these finger family videos start appearing everywhere, and you get versions in different languages with popular kids' cartoons using food or, frankly, using whatever kind of animation elements you seem to have lying around. And once again, there are millions and millions and millions of these videos available online in all of these kind of insane combinations. And the more time you start to spend with them, the crazier and crazier you start to feel that you might be.
لێره شتێكی بهڕاستی گهوره .ههیه له یوتیوبی منداڵ ."كه پێیدهڵێن "گۆرانی پهنجهی خێزانی .ناڵهی كهسێكم بیست له ئامادهبوهكان ."ئهوه"گۆرانی پهنجهی خێزانیه .ئهوه یهكهمه دهتوانم بیدۆزمهوه ئهوه لهساڵی٢٠٠٧، تهنها ٢٠٠ ههزار ،بینهری ههبوو .كه وهك هیچ نهبێ لهناو یاریهكه ،بهڵام تۆنێکی ناخۆشی شێتانەی ههیه ،من بۆتانی ناژهنم چونكه میشك داخ دهكات ،بهههمان شێوه هی منیش داخ دهكات من ئهوه ناكهم بۆ .ئێوە ،وهك هێلكهی سهرسامی دهچیته ناو مێشكهی منداڵ .و ئاڵودهی دهبن لهچهند ساڵێكی كهمدا ئهم ڤیدۆیی پهنجهی خێزانییه لە هەموو جێیەک .بڵاوبووەوە بە چەند زمانێکی جیاواز .دەیبینیتەوە بە بەکارهێنانی خواردن و .کارتۆنی بەناوبانگ یان، بهكارهیێنانی ههر جۆره توخمێكی ههست بزوێن .تا درۆبكهن لهگهڵ دهوروبهر و جارێكی دیكه، ملیۆن و ملیۆنانی ئهم ڤیدۆیانه بهردهسته لهههموجۆره .شته تێكهڵاویهكان له كاتی زیاتر دهستبهبهكار ،هێنانیان دهكهی شێتانهو شێتانه دهست پێدهكهی .تا ههست بكهی كهتۆ لهوانهبیت
And that's where I kind of launched into this, that feeling of deep strangeness and deep lack of understanding of how this thing was constructed that seems to be presented around me. Because it's impossible to know where these things are coming from. Like, who is making them? Some of them appear to be made of teams of professional animators. Some of them are just randomly assembled by software. Some of them are quite wholesome-looking young kids' entertainers. And some of them are from people who really clearly shouldn't be around children at all.
،وئهمه ئهوشوێنهیه كه دهستم پێكرد هەستی نامۆییەکی زۆر و تێنەگەیشتنێکی زۆر كه چۆن ئهم شته دروستكرا بۆ .پیێشكهشكردنی دهوروبهرم چونكه مهحاڵه بزانیت .ئهو شتانه لهكوێوه دێن كێ ئهوانهی دروستكردووه؟ دهردهكهوێت لهلایهن تیمی .شارهزای ههستبزوێن دروستكرابن كه به ههرهمهكی له .بهرنامهكان كۆیاندهكهنهوه ههندێكیان سهیركردنێكی بهتامیان .ههیه منداڵ چێژی لێدهبینێ و ههندێك له لایهن خهڵكێكه .كه بهڕاستی نایهوێ لهلایی منداڵ بێ
(Laughter)
(پێكهنین)
And once again, this impossibility of figuring out who's making this stuff -- like, this is a bot? Is this a person? Is this a troll? What does it mean that we can't tell the difference between these things anymore? And again, doesn't that uncertainty feel kind of familiar right now?
جارێكی دیكه، كێ ئهو ئهستهمییه دروست دهكات ئهوه ڕۆبۆته؟ یا كهسێكه؟ یا عهرهبانهیه؟ مانای چییه كه چیتر ناتوانین جیاوازی لە نێوان ئەو شتانە بکەین؟ دوباره، دوو دڵ نیت له ئێستادا ههست بهئاشنابوون بكهی؟
So the main way people get views on their videos -- and remember, views mean money -- is that they stuff the titles of these videos with these popular terms. So you take, like, "surprise eggs" and then you add "Paw Patrol," "Easter egg," or whatever these things are, all of these words from other popular videos into your title, until you end up with this kind of meaningless mash of language that doesn't make sense to humans at all. Because of course it's only really tiny kids who are watching your video, and what the hell do they know? Your real audience for this stuff is software. It's the algorithms. It's the software that YouTube uses to select which videos are like other videos, to make them popular, to make them recommended. And that's why you end up with this kind of completely meaningless mash, both of title and of content.
ڕێگای سهرهكی تا خهڵك ڤیدیۆكانیان سهیربكهن لهبیرت بێبت، بینهر واته پاره ئهوهیه ناونیشانی ڤیدیۆكهو .وشه گرینگهكان دهخهنهڕوو "وهك، "هێلكهی سهرسامی ،"یان "پاو پهتڕۆل "،" ئهستهر ئێگ ،یان ههر شتێكی لهوجۆره له ناونیشانهكهی وشهی ،ڤیدیۆ بهناوبانگهكان بهكاربێنی ههتا لهگهڵ ئهو زمانه تێكهڵه بێمانایه كۆتای پێدێنی ئهوه بههیچ شێوهیهك .بۆ مرۆڤ ناشێت چونكه بهدڵنیایهوه تهنها ،منداڵی بچوك سهیری ئهوانە دهكهن ئەوان چی دەزانن؟ ئامادهبونی ڕاستهقینه .بۆ ئهو شته بهرنامهیهكه .كه ئالۆگریتمهكانه بهرنامهیهكه یوتیوب بهكاری دههێنێت ،تا ڤیدیۆ هاوشێوهكان ههڵبژێرێت .تا ئهوان بهناوبانگ و پهسند بكات و لهبهر ئهوهیه تۆ ،بهبێمانایی كۆتای پێدههێنیت .له ناونیشان و ناوهڕۆكهكهی
But the thing is, you have to remember, there really are still people within this algorithmically optimized system, people who are kind of increasingly forced to act out these increasingly bizarre combinations of words, like a desperate improvisation artist responding to the combined screams of a million toddlers at once. There are real people trapped within these systems, and that's the other deeply strange thing about this algorithmically driven culture, because even if you're human, you have to end up behaving like a machine just to survive.
،بهڵام، ئهوشتهی دهبێت له بیرت بێت تا ئێستاش خهڵك ههن له ناو ئالگۆڕیزمی گهشبینی زۆر میهرهبانن ناچاركران نواندن بكهن له تێكهڵكردنی وشه ،بهشێوهیهكی سهیر وهك هونهرمهندێكی بێ هیوا و كاردانهوهی بۆ تێكهڵكردنی .ملیۆنێك منداڵ بهیهك جار خهڵكی باش لهناو ،سیستهمهكه به تهڵهوه بونه ئهوە شتێكی تری سهیر و ،شاراوهیه دهربارهی ئالگۆڕیزم ،چونكه ئهگهر تۆ مرۆڤ بیت پێویسته وهك ئامێرێك ڕهفتار بكهیت .تهنها بۆ خۆپاراستن
And also, on the other side of the screen, there still are these little kids watching this stuff, stuck, their full attention grabbed by these weird mechanisms. And most of these kids are too small to even use a website. They're just kind of hammering on the screen with their little hands. And so there's autoplay, where it just keeps playing these videos over and over and over in a loop, endlessly for hours and hours at a time. And there's so much weirdness in the system now that autoplay takes you to some pretty strange places. This is how, within a dozen steps, you can go from a cute video of a counting train to masturbating Mickey Mouse. Yeah. I'm sorry about that. This does get worse. This is what happens when all of these different keywords, all these different pieces of attention, this desperate generation of content, all comes together into a single place. This is where all those deeply weird keywords come home to roost. You cross-breed the finger family video with some live-action superhero stuff, you add in some weird, trollish in-jokes or something, and suddenly, you come to a very weird place indeed.
ههروهها، لهسهر لایهكی دیكهی ،شاشهكه تهنها منداڵی بچوكن سهیری ،ئهو شتانه دهكهن ئهو میكانزمه سهیرانه .سهرنجیان ڕادهكێشن ئهو منداڵانه زۆر بچوكن .بۆ بهكارهێنانی ئهو ماڵپهڕانه .بهو دهسته بچوكهیان شاشهكه دهكوتن ،ئەوتۆپلەی هەیە ،یاریهكه بهردهوام دهكا زیاترو زیاتر بهردهوام بۆ چهند كاتژمێرێك لهیهك .كات و ئێستا زۆر شتی نامۆ له سیستهمهكه ههیه كه ئەوتۆپلەی دهتبات بۆ .زۆر شوێنی جوان و سهیر ،بهشێوهیهك له ناو دهرزهنێك ههنگاو دهتوانی له ناو ڤییدێۆیهكی شهمهندهفهری ژماردن بڕۆی .بۆ دهسپهڕ لێدان له میكی ماوس .بهڵێ، من دڵگرانم دهربارهی ئهوه .ئهوه خراپتری دهكات كه ڕودهدات ،كاتێك ئهو كیبۆڕده جیاوازانه ،ههموو ئهو پارچه جیاوازانهی سهرنج ،ئهو نهوه بێ هیوایانه له ناوهڕۆك ههموو بهیهكههوه لهیهك شوێن .كۆبونهتهوه ئەو کیبۆردانە لە ماڵەکانماندا .ڕەگیان داکوتاوە به ڤیدیۆی پهنجهی خێزانی تێدهپهڕی ،لهگهڵ ههندێ پاڵهوانی نایاب دهژی ههندێك شتی سهیر زیاد دهكهیت ،وهك نوكته یان شتی تر .لهناكاو دهچیت بۆ شوێنێكی سهیر
The stuff that tends to upset parents is the stuff that has kind of violent or sexual content, right? Children's cartoons getting assaulted, getting killed, weird pranks that actually genuinely terrify children. What you have is software pulling in all of these different influences to automatically generate kids' worst nightmares. And this stuff really, really does affect small children. Parents report their children being traumatized, becoming afraid of the dark, becoming afraid of their favorite cartoon characters. If you take one thing away from this, it's that if you have small children, keep them the hell away from YouTube.
ئهو شتهی دایك و باوك پێی نیگهران دهبن ئهو شته یا جۆرێكی توند و تیژیه یاخود جنسیه، وایه؟ منداڵه كارتۆنیهكان دهبن .بە شتی نەشیاو ،به كوشتن گاڵتهی سهیر كه له ڕاستیدا .به گشتی منداڵ دهتۆقێنن ههموو ئهو كاریگهرییه جیاوازانه ئۆتۆماتیكیهن دروستدهبێت .لهسهر خهوی ناخۆشی منداڵ ئهوه بهڕاستی كاریگهری زۆر .ههیه لهسهر منداڵی بچوك باوان ڕاپۆرت لهسهر كۆسپی دهرونی ،منداڵ دهدهن ،وهك بوونی ترس له تاریكی یا ترس لهكارێكتهرهكانی .ئهو كارتۆنهی حهزیان لێیه ئهگهر منداڵی بچوكت ههیه ،دهتهوێت له یهك شت لایان بدهیت .له یوتیوب دوریان بخهوه
(Applause)
(چەپڵە)
But the other thing, the thing that really gets to me about this, is that I'm not sure we even really understand how we got to this point. We've taken all of this influence, all of these things, and munged them together in a way that no one really intended. And yet, this is also the way that we're building the entire world. We're taking all of this data, a lot of it bad data, a lot of historical data full of prejudice, full of all of our worst impulses of history, and we're building that into huge data sets and then we're automating it. And we're munging it together into things like credit reports, into insurance premiums, into things like predictive policing systems, into sentencing guidelines. This is the way we're actually constructing the world today out of this data. And I don't know what's worse, that we built a system that seems to be entirely optimized for the absolute worst aspects of human behavior, or that we seem to have done it by accident, without even realizing that we were doing it, because we didn't really understand the systems that we were building, and we didn't really understand how to do anything differently with it.
،بهڵام شتێكی دی، دهربارهی ئهوه من دڵنیانیم كه ئێمه بهڕاستی .چۆن دهگهین بهوخاڵه ،ئێمه ههموو ئهو شت ،و دهسهڵاتهمان بهدهست هێنا بهشێوهیهك .هیچ كهس بهو شێوه نهیتوانیوه تاوهكو ئێستاش، ئهو ڕێگەیهیه كه .جیهانی لهسهر بنیات دهنێین ،ئێمه ههمو ئهو داتایانه دهبهین ،بهشێكی زۆریان خراپن بهشێكی زۆری داتای مێژوین ،كه پڕیهتی له لایهن گری كویرانه ،و پریهتی له مهیلی خراپی ئێمه بۆ مێژوو و ئهوانمان كردوه به داتای گهورهو زۆر .دواتر دهیكهینه ئۆتۆماتیكی و ههمویانمان بهدهست هێناوه ،شتێكمان لێدروستكردوه وهك ڕاپۆرتی كارد ،یان بیمهی دڵنیای ،یان وهك سیستهمی پێشبینی كراوی پۆلیسی .یان بڕیاردان لهسهر یاسا گشتیهكان ئهمه ئهو ڕێگەیهیه ئهوڕۆ دونیای لهسهر دروست دهكهین .لهو داتایانه ،و نازانم چ خراپهیهكی ههیه كه ئێمه سیستهمێكی وامان دروستكردوه کە تەواو لە گونجاو دەچێت ،بۆ خراپترین لایهنی ئهخلاقی مرۆڤ یان ئێمه ئهوهمان به هەڵکەوت ،دروست كردوه بهبێ ئهوهی بزانین ،ئهوهمان دروست كردوه لهبهر ئهوهی ئێمه لهو سیستهمه ،تێناگهین كه دروستمان كردوه و لهوهش تێنهگهیشتوین چۆن ههرشتی .بكهین بهشێوهیهكی جیاواز لهوه
There's a couple of things I think that really seem to be driving this most fully on YouTube, and the first of those is advertising, which is the monetization of attention without any real other variables at work, any care for the people who are actually developing this content, the centralization of the power, the separation of those things. And I think however you feel about the use of advertising to kind of support stuff, the sight of grown men in diapers rolling around in the sand in the hope that an algorithm that they don't really understand will give them money for it suggests that this probably isn't the thing that we should be basing our society and culture upon, and the way in which we should be funding it.
لێرهدا دوشت ههیه كه من پێم وایه ڕۆڵیان ههبووه ،بهتایبهتی له یوتیوب ،یهكهمیان ڕیكلامه كه به مهبهستی پاره پهیداكردنه ،بهبێ هیچ جیاوازیهكی تر له كار هیچ گرنگیدانێك بهو خهڵكهی ،كه له ڕاستیدا ئهو كاره پیشدهخهن ،كۆكردنهوه و دهسهڵات .و جیاكردنهوهی ئهو شتانه من پێم وایه ههستدهكهی لهچۆنیهتی بهكارهێنانی ڕیكلامدا ،له پشتگیری ئهو ستافهدا دیمهنی بینینی پیاوێكی بهدهرپێی منداڵان لهناو خۆڵ بێوبچێت بههیوای ئهوهی ئهلگۆریزم كه تهواو لێی تێناگهن پارهیان پێ بدات بۆ ئهوه پیشنیار دهكات ئهوه ئهو شته نییه بتوانین كۆمهوڵگە وكلتورهكهمانی ،لهسهر بنیاتبنێین بەو ڕێگهیهی بتوانین .پشتگیری دارای بكهین
And the other thing that's kind of the major driver of this is automation, which is the deployment of all of this technology as soon as it arrives, without any kind of oversight, and then once it's out there, kind of throwing up our hands and going, "Hey, it's not us, it's the technology." Like, "We're not involved in it." That's not really good enough, because this stuff isn't just algorithmically governed, it's also algorithmically policed. When YouTube first started to pay attention to this, the first thing they said they'd do about it was that they'd deploy better machine learning algorithms to moderate the content. Well, machine learning, as any expert in it will tell you, is basically what we've started to call software that we don't really understand how it works. And I think we have enough of that already. We shouldn't be leaving this stuff up to AI to decide what's appropriate or not, because we know what happens. It'll start censoring other things. It'll start censoring queer content. It'll start censoring legitimate public speech. What's allowed in these discourses, it shouldn't be something that's left up to unaccountable systems. It's part of a discussion all of us should be having.
و خاڵی دوهم كه ڕۆڵی ههیه ،لهو به ئۆتۆماتیك كردنه كه بڵاوكردنهوهی ئهو ههمو تهكنهلۆجیایهیه ،بهخێراترین شێوه دهردهكهون ،بهبێ هیچ چاودێریهك ،و لهدهرهوهی ئهوێ شێوازی بهرزكردنهوهی دهستمان ".ڕۆیشتنمان، "ئێمه نین تهكنهلۆجیایه ".وهك "ئێمه تێوه نهگلاوین ،ئهوه زۆر باش نییه یان بهس نییه چونكه ئهو ستافه بهشێوهیهكی ،ڕێكخراو بهڕیوه نابرێت .لهڕوی ڕێكخراوهوه چاودێری دهكرێت كاتێك یوتیوب دهست دهكات ،بهوهی گرنگی پێبدهیت یهكهم شت دهیڵێن شتێكی لهبارهوه دهكهن دهیانهوێ ئامێری باشترفیربونی ئهلگۆریزم بڵاوبكهنهوه .بۆ نوێگهری ناوهرۆكهكه باش، ئامێری فێربون، وهك ،ههرئهزمونێك بهتۆ دهڵێت بەو ناوە دەستمان پێکرد سۆفتوێر كه لهسهرهتاوه .نازانین چۆن كاردهكات .پێموایه ئێستا بهشی پێویستمان ههیه پێویست ناكات ئهم ستافه جێبێڵین بۆ ئهی ئای تا بڕیار بدات ،كه چی گونجاوه یان نا .چونكه دهزانین چی دهبیت .دهست دهكات به سهرژمێری شتهكان .سهرژمێری ناوهڕۆكی سهیرو نائاسای .و سهرژمێری قسهی دروستی گشتی ،ئهوهی لهو گفتوگۆیانه ڕێگه پێدراوه نابێت شتێك بێت كه بهرز بكرێتهوه .بۆ سیستهمێكی نهژمێردراو ئهمه بهشێكه له گفتوگۆیهك .كه پێویسته ههمانبیت
But I'd leave a reminder that the alternative isn't very pleasant, either. YouTube also announced recently that they're going to release a version of their kids' app that would be entirely moderated by humans. Facebook -- Zuckerberg said much the same thing at Congress, when pressed about how they were going to moderate their stuff. He said they'd have humans doing it. And what that really means is, instead of having toddlers being the first person to see this stuff, you're going to have underpaid, precarious contract workers without proper mental health support being damaged by it as well.
بیرهێنهرهوهیهك جێدێڵم جێگرەوەیەکی .زۆر بەهێز نییە یوتیوب ئاشكرای كردوه به نیازن ڤێرژنێك بڵاو بكهنهوه تایبەت بە منداڵان كه ڕۆڵی نێوانگیری ببینێت .لهلایهن مرۆڤهوه فهیسبوك--زوكبێرگ لهكۆبونهوهدا ،زۆرشتی هاوشێوهی وت كاتێك باسی چونیهتی ڕۆڵی .نیوهنگیری ستافهكهیانی كرد گوتی كهسانێك ههن ئهوانه .دهكهن ،مهبهستی ئهوه بوو پێش ئهوهی منداڵهكان یهكهم ،كهس بن ئهو ستافه ببینن ،دهبێ كرێیهكی كهمت دابێ وهك عهقدێكی ئیشكردن بهبێ یارمهتی تهندروست باشی عهقڵی .خراپ دهبێت
(Laughter)
(پێكهنین)
And I think we can all do quite a lot better than that.
من پێم وایه دهتوانین زۆر لهوه .باشتر بكهین
(Applause)
(چەپڵە)
The thought, I think, that brings those two things together, really, for me, is agency. It's like, how much do we really understand -- by agency, I mean: how we know how to act in our own best interests. Which -- it's almost impossible to do in these systems that we don't really fully understand. Inequality of power always leads to violence. And we can see inside these systems that inequality of understanding does the same thing. If there's one thing that we can do to start to improve these systems, it's to make them more legible to the people who use them, so that all of us have a common understanding of what's actually going on here.
،ئهو بیرۆكه، پێموایه دوو شت ،بۆمن پێكهوه دهنیت بریکار چهند له ئێمه له بریکار :تێدهگهین، مهبهستم ئێمه چۆن دهزانین چۆن كاردهكهین .بهپێی ئهو توانایهی ههمانه كه--ئهوهی ئهستهمه كردنی لهم سیستهمه كه ئێمه بهتهواوی .لێی تێناگهین و نهبونی یا كهمی هێز سهردهكێشێت .بۆ توندو تیژی و ئێمه دهزانین لهم سیستهمانەدا كهمی یا نهبونی تێگهشتن .ههمان شت دهكات ئهگهریهك شت ههبێت كه ،بۆ بهرهو پێشبردنی ئهو سیستهمه بیكهم كه وای لێبكهم زیاتر ڕونبێت ،بۆ ئهوانهی بهكاری دێنن لهبهرئهوهی تێگهشتنی هاوبهشمان ههیه .كه لێره چی دهگوزهرێت
The thing, though, I think most about these systems is that this isn't, as I hope I've explained, really about YouTube. It's about everything. These issues of accountability and agency, of opacity and complexity, of the violence and exploitation that inherently results from the concentration of power in a few hands -- these are much, much larger issues. And they're issues not just of YouTube and not just of technology in general, and they're not even new. They've been with us for ages. But we finally built this system, this global system, the internet, that's actually showing them to us in this extraordinary way, making them undeniable. Technology has this extraordinary capacity to both instantiate and continue all of our most extraordinary, often hidden desires and biases and encoding them into the world, but it also writes them down so that we can see them, so that we can't pretend they don't exist anymore. We need to stop thinking about technology as a solution to all of our problems, but think of it as a guide to what those problems actually are, so we can start thinking about them properly and start to address them.
شتێك، پێموایه زۆرترین شت دهربارهی ئهم سیستهمه كه به هیوام توانیبێتم بهتهواوهتی ،دهربارهی یوتیوب ڕوونم كردبێتهوه .و ههموو شتێكه ،ئهو بابهته بهرپرسیاریهتی و بریكاریه ،هی ناڕونی و ئاڵۆزی توندوتیژی وبهركارهێنان كه سهرچاوه دهگرێت لهكۆكردنهوهی دهسهڵات لهههندێك دهستی كهمدا .ئهوانه ،كێشهی زۆر زۆر گهورهن بهتهنها بۆ یوتیوب كێشه نین ،و نه بۆ تهكنهلۆجیا بهگشتی .كێشهیهكی نوێ نییه .ماوهیهكی زۆره لهگهڵمانن لهكۆتایدا ئهو سیستهمه جیهانیهمان ،دروست كرد، ئینتهرنێت كه بهڕیگهیهكی له ڕاده بهدهر ،پیشانی ئێمهی دهدهن .كه حاشا ههڵنهگر بن تهكنهلوجیا ئهو توانا زۆرهی ههیه بۆ ههردوو نوێنهرایهتی و بهردهوامی ،ئێمه به ڕادهیهكی زۆر حهزه شاراوه ولایهنگیریمان به كۆد ،دهخاته پێش چاوی دونیا بهڵام له ههمانكاتیش دهیاننوسێتهوه ،بۆیه به ئاسانی دهیبینین كه ناتوانین پیشانیان بدهین ،چونكه له هیچ كوێ بونیان نییه بیرنهكهینهوه له تهكنهلۆجیا ههموو ،كێشهكانمان چارهسهر دهكات بهڵام وهك نیشانهیهك بۆدیاریكردنی ،جۆری كێشهكه بیریلێبكهوه دهتوانین به دروستی بیریان لێبكهینهوه .و دهست به ناسینیان بكهین
Thank you very much.
.زۆر سوپاستان دهكهم
(Applause)
(چهپڵه لێدهدهن)
Thank you.
.سوپاستاندهكهم
(Applause)
(چەپڵە)
Helen Walters: James, thank you for coming and giving us that talk. So it's interesting: when you think about the films where the robotic overlords take over, it's all a bit more glamorous than what you're describing. But I wonder -- in those films, you have the resistance mounting. Is there a resistance mounting towards this stuff? Do you see any positive signs, green shoots of resistance?
هیلان واڵتهرز: جیمس، سوپاس بۆ .پێدانی ئهو قسانەت :كهوایه سهرنج ڕاكێشه كاتێك كه سهیری فیلمێك دهكهیت ،ڕۆبۆتێكی گهوره شتێك بهدهست دههێنێت ئهوه تۆزێك خۆشیه به .بهراورد بهوهی باستكرد بهڵام من لهو فیلمانه ترسم .لهوهیه ڕێگریت ههبێت ئایا هیچ ڕێگریهك ههیه لهبهردهم ئهو ستافه؟ ئایا هیچ ئاماژهیهكی پۆزەتیڤ بهرهو پێش چون دهبینیت، لهو ڕێگریه؟
James Bridle: I don't know about direct resistance, because I think this stuff is super long-term. I think it's baked into culture in really deep ways. A friend of mine, Eleanor Saitta, always says that any technological problems of sufficient scale and scope are political problems first of all. So all of these things we're working to address within this are not going to be addressed just by building the technology better, but actually by changing the society that's producing these technologies. So no, right now, I think we've got a hell of a long way to go. But as I said, I think by unpacking them, by explaining them, by talking about them super honestly, we can actually start to at least begin that process.
جهیمس: من هیچ ،دهربارهی ڕێگری ڕاستهوخۆ نازانم چونكه ستافهكه نایابن .و ههرواش دهمێننهوه ههستدهكهم بهباشی و قوڵی .تێكهڵی كهلتوربوونه ،هاوڕێیهكم، ئیلینه سایتا ههمیشه دهڵێت ههر كێشهیهكی تهكنهلۆجیا له بوارێكی پێویستدا .پێش ههر شتێك كێشهیهكی سیاسیه ههموو ئهو شتانهی ئیشی لهسهر دهكهین تا بیانناسێنین ناتوانرێ تهنها به ،بونیادنانی تهكنهلجیا باشتر بناسرێن بەڵکو بە گۆڕینی ئەو کۆمەڵگەیەی .کە تەکنەلۆژیا بەرهەمدێنێت پێم وایه ڕێگهیهكی زۆر قورس .و دوور له ڕێمان دایه وهك وتم، بههۆی قسهكردن ،لهسهریان وزیاتر ڕونكردنهوهیان ،بهشێوهیهكی ڕاستگۆییانه دهتوانین دهست بكهین .به دروستكردنی ئهو پڕۆسه
HW: And so when you talk about legibility and digital literacy, I find it difficult to imagine that we need to place the burden of digital literacy on users themselves. But whose responsibility is education in this new world?
و كاتێك تۆ دهدوێیت لهسهر ،ڕوونی خوێندهواری قورسه بیهێنمه پێش چاوی خۆم ئهو خوێندهواریه ژمارهیهك ئهركی .بخهینه ئهستۆی بهكارهێنهران خۆیان بهڵام بهرپرسیاریهتیهكهی دهكهوێته ئهستۆی پهروهرده؟
JB: Again, I think this responsibility is kind of up to all of us, that everything we do, everything we build, everything we make, needs to be made in a consensual discussion with everyone who's avoiding it; that we're not building systems intended to trick and surprise people into doing the right thing, but that they're actually involved in every step in educating them, because each of these systems is educational. That's what I'm hopeful about, about even this really grim stuff, that if you can take it and look at it properly, it's actually in itself a piece of education that allows you to start seeing how complex systems come together and work and maybe be able to apply that knowledge elsewhere in the world.
جیمس: پێموایه ئهم بهرپرسیاریهته ،پهیوهسته به ههمومانهوه ،له ههموو شتێك كه دهیكهین ،و دروستی دهكهین دهبێت دروستبكرێت به شێوهی داتاو گفتوگۆی لهبارهوه بكرێت ;لهگهڵ ههر كهسێك خۆشی دوور بگرێت كه پڕۆگرامێك دروست ناكهین خهڵك فریو بدات یا توشی سهرسوڕمانیان بكات ،به شێوهیهك تا دروستترین شت بكهن بهڵام له ڕاستیدا تێوهگلاون لهههر ،ههنگاوێكی پهروهرده كردنیان چونكه ههریهك لهو پڕۆگرامانه .پهروهردهین ئهوهش تاكه هۆیه دهربارهی ئهو ،بابهت و گروپه دڵخۆشی كردوم ،ئهگهر بتوانی بهجوانی لێی بڕوانیت لهناو خۆیدا ههندێك پهروهردهی ههڵگرتوه كه ڕێگهت دهدات ببینیت پڕۆگرامهكان چهند ئاڵۆزن و پێكهوه كاردهكهن لهوانهیه ههر زانیاریهكت دهست .بكهوێ له ههر كوێیهكی جیهان بێت
HW: James, it's such an important discussion, and I know many people here are really open and prepared to have it, so thanks for starting off our morning.
،پێشكهشكار: جهیمس ،گفتوگۆیهكی گرینگبوو و دهزانم خهڵكێكی زۆر لێرهن مێشكیان ،كراوهیه و وهری دهگرن ،سوپاس ئهو بهیانیه لهگهڵمان بوویت
JB: Thanks very much. Cheers.
.جهیمس: سوپاس
(Applause)
(چەپڵە)