Roy Price er en mand som de fleste af jer nok aldrig har hørt om, selvom han måske er ansvarlig for 22 mere eller mindre middelmådige minutter af jeres liv den 19. april 2013. Han kan også have været ansvarlig for 22 meget underholdende minutter, men ikke for særligt mange af jer. Og alt det stammer fra en beslutning som Roy var nød til at tage omkring tre år siden.
Roy Price is a man that most of you have probably never heard about, even though he may have been responsible for 22 somewhat mediocre minutes of your life on April 19, 2013. He may have also been responsible for 22 very entertaining minutes, but not very many of you. And all of that goes back to a decision that Roy had to make about three years ago.
Ser i, Roy Price er en overordnet leder hos Amazon Studios. Det er Amazons TV produktionsselskab. Han er 47 år gammel, slank, strithår, beskriver sig selv på Twitter som "film, TV, teknologi, taco." Og Roy Price har et meget ansvarsfuldt job, fordi det er hans ansvar at vælge de serier, det originale indhold som Amazon skal producere. Og det er selvfølgelig et meget konkurrerende miljø. Det er jo allerede så mange TV-serier, så Roy kan ikke bare vælge en hvilket som helst serie. Han bliver nød til at finde serier som er rigtig, rigtig gode. Med andre ord, skal han finde serier som er helt til højre på denne kurve her.
So you see, Roy Price is a senior executive with Amazon Studios. That's the TV production company of Amazon. He's 47 years old, slim, spiky hair, describes himself on Twitter as "movies, TV, technology, tacos." And Roy Price has a very responsible job, because it's his responsibility to pick the shows, the original content that Amazon is going to make. And of course that's a highly competitive space. I mean, there are so many TV shows already out there, that Roy can't just choose any show. He has to find shows that are really, really great. So in other words, he has to find shows that are on the very right end of this curve here.
Denne kurve her er fordeling af bedømmelser af omkring 2.500 TV serier på hjemmesiden IMDB, og bedømmelserne går fra et til 10, og højden her viser hvor mange serier der får den bedømmelse. Så hvis din serie får en bedømmelse på ni point eller højere er det en vinder. Så har du en top to procent serie. Det er serier som "Breaking Bad," "Game of Thrones," "The Wire," Altså alle de her serier som er vanedannende, hvor når du har set en sæson, siger din hjerne bare, "Hvor kan jeg få flere af de her afsnit?" Den slags serier. På venstre side, for at gøre det klart, i den her side, er der serier som hedder "Toddlers and Tiaras" -
So this curve here is the rating distribution of about 2,500 TV shows on the website IMDB, and the rating goes from one to 10, and the height here shows you how many shows get that rating. So if your show gets a rating of nine points or higher, that's a winner. Then you have a top two percent show. That's shows like "Breaking Bad," "Game of Thrones," "The Wire," so all of these shows that are addictive, whereafter you've watched a season, your brain is basically like, "Where can I get more of these episodes?" That kind of show. On the left side, just for clarity, here on that end, you have a show called "Toddlers and Tiaras" --
(Latter)
(Laughter)
- hvilket burde sige det hele omkring hvad der sker i den ende af kurven.
-- which should tell you enough about what's going on on that end of the curve.
Roy Price er ikke bekymret for at ende på den venstre side af kurven, fordi jeg tror at du virkelig skal have noget tankevirksomhed for at komme under "Toddlers and Tiaras." Så det han er bekymret omkring er bulen her i midten, bulen med gennemsnitlig TV, I ved, den slags serier som ikke er rigtig gode eller rigtig dårlige - de begejstrer ikke rigtig. Så han bliver nød til at sikre sig, at han er på den rigtige side af denne.
Now, Roy Price is not worried about getting on the left end of the curve, because I think you would have to have some serious brainpower to undercut "Toddlers and Tiaras." So what he's worried about is this middle bulge here, the bulge of average TV, you know, those shows that aren't really good or really bad, they don't really get you excited. So he needs to make sure that he's really on the right end of this.
Så der er pres på, og det er selvfølgelig også første gang, at Amazon overhovedet laver sådan noget, så Roy Price vil ikke tage nogle chancer. Han vil skabe succes. Han har brug for en garanteret succes, og derfor afholder han en konkurrence.
So the pressure is on, and of course it's also the first time that Amazon is even doing something like this, so Roy Price does not want to take any chances. He wants to engineer success. He needs a guaranteed success, and so what he does is, he holds a competition.
Han tager en bunke idéer til TV-serier og gennem en evaluering af de idéer bliver otte kandidater til TV-serier valgt og så laver han kun den første episode til hver serie og lægger dem online så alle kan se dem gratis. Og når Amazon giver dig noget gratis så tager du imod det, ikke? Så millioner af seere ser disse episoder.
So he takes a bunch of ideas for TV shows, and from those ideas, through an evaluation, they select eight candidates for TV shows, and then he just makes the first episode of each one of these shows and puts them online for free for everyone to watch. And so when Amazon is giving out free stuff, you're going to take it, right? So millions of viewers are watching those episodes.
Hvad de ikke indser er at mens de ser deres serie bliver de faktisk selv iagttaget. De bliver iagttaget af Roy Price og hans hold som optager alting, De optager når nogen trykker play. når nogen trykker pause, hvilke dele de springer over, hvilke dele de ser igen. Så de samler millioner af data point, fordi de vil have de data point til når de skal beslutte hvilken serie de vil lave. Som sagt så gjort, de samlede al data, de analyserede denne og et svar dukkede op, og det svar er, "Amazon bør lave en komedieserie om fire republikanske senatorer." De lavede den serie.
What they don't realize is that, while they're watching their shows, actually, they are being watched. They are being watched by Roy Price and his team, who record everything. They record when somebody presses play, when somebody presses pause, what parts they skip, what parts they watch again. So they collect millions of data points, because they want to have those data points to then decide which show they should make. And sure enough, so they collect all the data, they do all the data crunching, and an answer emerges, and the answer is, "Amazon should do a sitcom about four Republican US Senators." They did that show.
Er der nogen der kender navnet på den serie? (Publikum: "Alpha House.") Ja, "Alpha House," men det virker som om der ikke er så mange af jer som faktisk husker den serie, fordi den endte med ikke at blive så god. Det er faktisk kun en gennemsnitlig serie endda bogstavelig talt, fordi gennemsnittet på denne kurve er 7,4 og "Alpha House" er på 7,5, så en lige over gennemsnittet serie, men helt sikkert ikke hvad Roy Price og hans hold gik efter. I mellemtiden, i et andet firma, lykkedes det en anden leder at skaffe en top serie ved hjælp af data analysering, og hans navn er Ted, Ted Sarandos, som er Chief Content Officer hos Netflix. og ligesom Roy, er han på en konstant mission for at finde en god TV-serie, og til det bruger han også data, bortset fra, at han gør det lidt anderledes. I stedet for at afholde en konkurrence, han - og hans hold selvfølgelig - så på al den data de allerede havde om Netflix seere. I ved, de bedømmelser de giver deres serier, afspilningshistorik, hvilke serier folk kan lide, osv. Og så bruger de den data til at opdage alle disse små dele af informationer om seererne: hvilken slags serier, producere, skuespillere de kan lide. Og når de havde samlet alle disse dele, tog de en chance, og de besluttede at lave - ikke en komedieserie om fire senatorer - men en drama serie om en enkelt senator. Kender i denne serie?
So does anyone know the name of the show? (Audience: "Alpha House.") Yes, "Alpha House," but it seems like not too many of you here remember that show, actually, because it didn't turn out that great. It's actually just an average show, actually -- literally, in fact, because the average of this curve here is at 7.4, and "Alpha House" lands at 7.5, so a slightly above average show, but certainly not what Roy Price and his team were aiming for. Meanwhile, however, at about the same time, at another company, another executive did manage to land a top show using data analysis, and his name is Ted, Ted Sarandos, who is the Chief Content Officer of Netflix, and just like Roy, he's on a constant mission to find that great TV show, and he uses data as well to do that, except he does it a little bit differently. So instead of holding a competition, what he did -- and his team of course -- was they looked at all the data they already had about Netflix viewers, you know, the ratings they give their shows, the viewing histories, what shows people like, and so on. And then they use that data to discover all of these little bits and pieces about the audience: what kinds of shows they like, what kind of producers, what kind of actors. And once they had all of these pieces together, they took a leap of faith, and they decided to license not a sitcom about four Senators but a drama series about a single Senator. You guys know the show?
(Latter)
(Laughter)
Ja, "House of Cards," og Netflix selvfølgelig ramte rigtig med den serie, i hvert fald de første to sæsoner,
Yes, "House of Cards," and Netflix of course, nailed it with that show, at least for the first two seasons.
(Latter) (Klapsalver)
(Laughter) (Applause)
"House of Cards" får 9,1 på denne kurve, så det er præcis der hvor de gerne ville have den skulle være.
"House of Cards" gets a 9.1 rating on this curve, so it's exactly where they wanted it to be.
Så er spørgsmålet selvfølgelig hvad skete der her? Du har to konkurrencedygtige, data-kløgtige firmaer. De samler alle de her millioner af data point, Og det fungerer fantastisk for en af dem og det virker ikke for den anden. Hvorfor? Fordi logisk set burde det egentlig fungerer hver gang. Hvis du samler millioner af data point til en beslutning du skal tage, så burde du kunne tage en ret god beslutning. Du har 200 års statistik at falde tilbage på. Det forstærker du med kraftfulde computere. Det mindste du kan forvente er vel god TV, ikke?
Now, the question of course is, what happened here? So you have two very competitive, data-savvy companies. They connect all of these millions of data points, and then it works beautifully for one of them, and it doesn't work for the other one. So why? Because logic kind of tells you that this should be working all the time. I mean, if you're collecting millions of data points on a decision you're going to make, then you should be able to make a pretty good decision. You have 200 years of statistics to rely on. You're amplifying it with very powerful computers. The least you could expect is good TV, right?
Og hvis data analyse ikke virker på den måde, så er det faktisk lidt skræmmende, fordi vi lever i en tid hvor vi bruger data mere og mere til at tage seriøse valg langt udover TV. Er der nogen her der kender firmaet Multi-Health Systems? Ingen. Okay, det er faktisk godt. Okay, så Multi-Health Systems er et software firma, og jeg håber ikke at nogen i dette lokale nogensinde kommer i kontakt med den software, fordi så betyder det at I er i fængsel.
And if data analysis does not work that way, then it actually gets a little scary, because we live in a time where we're turning to data more and more to make very serious decisions that go far beyond TV. Does anyone here know the company Multi-Health Systems? No one. OK, that's good actually. OK, so Multi-Health Systems is a software company, and I hope that nobody here in this room ever comes into contact with that software, because if you do, it means you're in prison.
(Latter)
(Laughter)
Hvis nogen her i USA er i fængsel og ansøger om prøveløsladelse, så er det meget sandsynligt, at data analyse software fra det firma vil blive brugt til at bestemme om der skal gives prøveløsladelse. Så det er det samme princip som Amazon og Netflix, men i stedet for at bestemme om en TV-serie bliver god eller dårlig, bestemmer man om en fange bliver god eller dårlig. Og middelmådig TV, 22 minutter, det kan være ret dårligt, men flere år i fængsel, går jeg udfra, er endnu værre.
If someone here in the US is in prison, and they apply for parole, then it's very likely that data analysis software from that company will be used in determining whether to grant that parole. So it's the same principle as Amazon and Netflix, but now instead of deciding whether a TV show is going to be good or bad, you're deciding whether a person is going to be good or bad. And mediocre TV, 22 minutes, that can be pretty bad, but more years in prison, I guess, even worse.
Og uheldigvis er der faktisk bevis for. at denne data analyse, på trods af masser af date, ikke altid producerer optimale resultater. Og det er ikke fordi et firma som Multi-Health Systems ikke bruger den data korrekt. Selv de mest data-kyndige firmaer fejler. Ja, selv Google fejler noglegange.
And unfortunately, there is actually some evidence that this data analysis, despite having lots of data, does not always produce optimum results. And that's not because a company like Multi-Health Systems doesn't know what to do with data. Even the most data-savvy companies get it wrong. Yes, even Google gets it wrong sometimes.
I 2009 annoncerede Google, at de var i stand til, med data analyse, at forudse udbrud ad influenza, den slemme slags, ved at lave data analyse på deres Google søgninger. Og det virkede fantastisk, og det blev en stor historie i nyhederne, inklusiv højdepunktet i videnskabelig succes: en udgivelse i tidskriftet "Nature." Det virkede fantastisk år efter år efter år, indtil det et år fejlede. Og ingen kunne forklare præcis hvorfor. Det virkede bare ikke det år, og det blev selvfølgelig også en nyhed, med efterfølgende tilbagetrækning af udgivelsen i tidskriftet "Nature." Så selv de mest data-kyndige firmaer, Amazon og Google, fejler sommetider. Og på trods af disse fiaskoer, rykker data hastigt ind det virkelig livs beslutningstagen - ind på arbejdspladsen, retshåndhævelsen, lægevidenskab. Så vi må hellere være sikker på. at data hjælper.
In 2009, Google announced that they were able, with data analysis, to predict outbreaks of influenza, the nasty kind of flu, by doing data analysis on their Google searches. And it worked beautifully, and it made a big splash in the news, including the pinnacle of scientific success: a publication in the journal "Nature." It worked beautifully for year after year after year, until one year it failed. And nobody could even tell exactly why. It just didn't work that year, and of course that again made big news, including now a retraction of a publication from the journal "Nature." So even the most data-savvy companies, Amazon and Google, they sometimes get it wrong. And despite all those failures, data is moving rapidly into real-life decision-making -- into the workplace, law enforcement, medicine. So we should better make sure that data is helping.
Personligt har jeg set megen af denne kamp med data selv, da jeg arbejder med databehandletgenetik, hvilket også er et felt hvor mange meget smart folk bruger utrolige mængder data til at træffe nogle ret seriøse beslutninger, såsom valg vedrørende kræft terapi eller udvikle et medikament. Og i årenes løb har jeg lagt mærke til et mønster eller en slags regel, om du vil, om forskellen mellem succesfuld beslutningstagen med data og fejlende beslutningstagen, og jeg fandt dette mønster værd at dele, og det er sådan her.
Now, personally I've seen a lot of this struggle with data myself, because I work in computational genetics, which is also a field where lots of very smart people are using unimaginable amounts of data to make pretty serious decisions like deciding on a cancer therapy or developing a drug. And over the years, I've noticed a sort of pattern or kind of rule, if you will, about the difference between successful decision-making with data and unsuccessful decision-making, and I find this a pattern worth sharing, and it goes something like this.
Hver gang du løser et kompleks problem, gør du i virkeligheden to ting. Den første er at du deler problemet op i mindre stykker så du kan analysere disse stykker i dybden og så laver du selvfølgelig den anden del. Du samler alle disse stykker igen for at komme frem til din konklusion. Og noglegange skal du begynde forfra, men det er altid disse to ting: skille det ad og samle det igen.
So whenever you're solving a complex problem, you're doing essentially two things. The first one is, you take that problem apart into its bits and pieces so that you can deeply analyze those bits and pieces, and then of course you do the second part. You put all of these bits and pieces back together again to come to your conclusion. And sometimes you have to do it over again, but it's always those two things: taking apart and putting back together again.
Og den vigtige del er, at data og data analyse er kun brugbart til den første del. Data og data analyse, uanset hvor stærk, kan kun hjælpe med at skille problemet ad og forstå stykkerne. Det er ikke lavet til at samle stykkerne igen og komme frem til en konklusion. Der er et andet værktøj der kan gøre det og alle har det, og det værktøj er hjernen. Hvis det er en ting hjernen er god til, så er det at samle stykker sammen igen, selv hvis du ikke har al informationen, og nå en god konklusion, især hvis det er en eksperts hjerne.
And now the crucial thing is that data and data analysis is only good for the first part. Data and data analysis, no matter how powerful, can only help you taking a problem apart and understanding its pieces. It's not suited to put those pieces back together again and then to come to a conclusion. There's another tool that can do that, and we all have it, and that tool is the brain. If there's one thing a brain is good at, it's taking bits and pieces back together again, even when you have incomplete information, and coming to a good conclusion, especially if it's the brain of an expert.
Og det er derfor jeg tror, at Netflix var så succesfulde fordi de brugte data og hjerne der hvor de passede i processen. De brugte først data til at forstå en masse stykker af deres seere som de ellers ikke kunne have forstået på sådan et niveau, men beslutningen om at tage alle disse stykker og samle dem igen og lave en serie som "House of Cards," var ikke i nærheden af data. Ted Sandaros og hans hold tog beslutningen om at lave den serie, hvilket i øvrigt også betød, at de løb en ret stor personlig risiko med den beslutning. Og Amazon, på den anden side, de gjorde det på den forkerte led. De brugte data hele vejen i deres beslutningstagen, først da de afholdte konkurrencen med TV idéer senere da de valgte at lave "Alpha House" til en serie. Hvilket selvfølgelig var en sikker beslutning for dem fordi de altid kunne pege på dataen og sige: "Dette er hvad dataen fortalte os." Men det første ikke til det exceptionelle resultat som de håbede på.
And that's why I believe that Netflix was so successful, because they used data and brains where they belong in the process. They use data to first understand lots of pieces about their audience that they otherwise wouldn't have been able to understand at that depth, but then the decision to take all these bits and pieces and put them back together again and make a show like "House of Cards," that was nowhere in the data. Ted Sarandos and his team made that decision to license that show, which also meant, by the way, that they were taking a pretty big personal risk with that decision. And Amazon, on the other hand, they did it the wrong way around. They used data all the way to drive their decision-making, first when they held their competition of TV ideas, then when they selected "Alpha House" to make as a show. Which of course was a very safe decision for them, because they could always point at the data, saying, "This is what the data tells us." But it didn't lead to the exceptional results that they were hoping for.
Data er selvfølgelig et enormt brugbart værktøj til at tage bedre valg men jeg tror, at det går galt når data begynder at styre de valg. Uanset hvor stærk data er, er det kun et værktøj og for at huske på dette, finder jeg denne her meget brugbar. Mange af jer vil ...
So data is of course a massively useful tool to make better decisions, but I believe that things go wrong when data is starting to drive those decisions. No matter how powerful, data is just a tool, and to keep that in mind, I find this device here quite useful. Many of you will ...
(Latter)
(Laughter)
Før der var data, var dette beslutningstager-enheden man brugte.
Before there was data, this was the decision-making device to use.
(Latter)
(Laughter)
Mange af kender denne. Dette legetøj her hedder en magisk 8-ball og den er helt utrolig, fordi hvis du har et valg du skal tage. et ja/nej spørgsmål, det eneste du skal gøre er at ryste kuglen og så får du et svar - "Højst sandsynlig" - lige her i dette vindue i realtid. Jeg vil demonstrere derude senere.
Many of you will know this. This toy here is called the Magic 8 Ball, and it's really amazing, because if you have a decision to make, a yes or no question, all you have to do is you shake the ball, and then you get an answer -- "Most Likely" -- right here in this window in real time. I'll have it out later for tech demos.
(Latter)
(Laughter)
Sagen er, at jeg har taget nogle valg i mit liv hvor jeg skulle have lyttet til kuglen, set i bakspejlet. Men, I ved, hvis man har data til rådighed vil man erstatte den med noget langt mere sofistikeret, såsom data analyse for at komme frem til en bedre beslutning. Men det ændrer ikke den basale opsætning. Kuglen bliver måske klogere og klogere og klogere, men jeg mener stadig, at vi skal tage beslutningerne hvis vi vil opnå noget ekstraordinært, i den højre ende af kurven. Og det mener jeg faktisk er en meget opmuntrende besked, at selv overfor masser af data betaler det sig stadig at tage beslutninger, at være en ekspert til det du laver og take chancer. For når alt kommer til alt, er det ikke data det er chancer der vil føre dig til den højre ende af kurven.
Now, the thing is, of course -- so I've made some decisions in my life where, in hindsight, I should have just listened to the ball. But, you know, of course, if you have the data available, you want to replace this with something much more sophisticated, like data analysis to come to a better decision. But that does not change the basic setup. So the ball may get smarter and smarter and smarter, but I believe it's still on us to make the decisions if we want to achieve something extraordinary, on the right end of the curve. And I find that a very encouraging message, in fact, that even in the face of huge amounts of data, it still pays off to make decisions, to be an expert in what you're doing and take risks. Because in the end, it's not data, it's risks that will land you on the right end of the curve.
Tak.
Thank you.
(Klapsalver)
(Applause)