Up until now, our communication with machines has always been limited to conscious and direct forms. Whether it's something simple like turning on the lights with a switch, or even as complex as programming robotics, we have always had to give a command to a machine, or even a series of commands, in order for it to do something for us. Communication between people, on the other hand, is far more complex and a lot more interesting because we take into account so much more than what is explicitly expressed. We observe facial expressions, body language, and we can intuit feelings and emotions from our dialogue with one another. This actually forms a large part of our decision-making process. Our vision is to introduce this whole new realm of human interaction into human-computer interaction so that computers can understand not only what you direct it to do, but it can also respond to your facial expressions and emotional experiences. And what better way to do this than by interpreting the signals naturally produced by our brain, our center for control and experience.
Sve do sada je, naša komunikacija sa strojevima uvijek bila ograničena na svjesne i izravne postupke. Bez obzira radi li se o nečem jednostavnom, poput paljenja svjetla prekidačem, ili o nečem složenom poput programiranja robota, oduvijek smo trebali dati naredbu stroju, ili čak niz naredbi, kako bi on nešto napravio za nas. Komunikacija između ljudi, s druge strane, je mnogo složenija i puno zanimljivija, jer uzimamo u obzir puno više od onoga što se izričito iskazuje. Promatramo izraz lica, govor tijela, i možemo osjetiti emocije u razgovoru s drugima. To je zapravo veliki dio našeg procesa donošenja odluka. Naša vizija je uvesti tu potpuno novu dimenziju ljudske komunikacije u interakciju između čovjeka i računala, kako bi računala mogla razumijeti ne samo ono što im naredite da naprave, već kako bi mogla reagirati na vaš izraz lica i emocionalna iskustva. A koji je bolji način za to napraviti od interpretacije signala koje prirodno proizvodi naš mozak, naš centar za kontrolu i iskustvo.
Well, it sounds like a pretty good idea, but this task, as Bruno mentioned, isn't an easy one for two main reasons: First, the detection algorithms. Our brain is made up of billions of active neurons, around 170,000 km of combined axon length. When these neurons interact, the chemical reaction emits an electrical impulse, which can be measured. The majority of our functional brain is distributed over the outer surface layer of the brain, and to increase the area that's available for mental capacity, the brain surface is highly folded. Now this cortical folding presents a significant challenge for interpreting surface electrical impulses. Each individual's cortex is folded differently, very much like a fingerprint. So even though a signal may come from the same functional part of the brain, by the time the structure has been folded, its physical location is very different between individuals, even identical twins. There is no longer any consistency in the surface signals.
Zvuči kao prilično dobra ideja, ali taj zadatak, kao što je Bruno spomenuo, nije jednostavan iz dva glavna razloga: Prvo, algoritmi za detekciju. Naš mozak je napravljen od milijardi aktivnih neurona i oko 170,000 km ukupne dužine aksona. Kada su ti neuroni u interakciji, kemijska reakcija emitira električne impulse koji se mogu mjeriti. Većina naših moždanih funkcija je raspodijeljena po vanjskoj površini mozga. A kako bi se povećalo područje koje je dostupno za mentalne aktivnosti, površina mozga je jako naborana. Sva ta kortikalna pregibanja predstavljaju značajan izazov za interpretaciju površinskih električnih impulsa. Korteks svake osobe je naboran drugačije, nešto poput otiska prsta. Pa premda signali možda dolaze iz istog funkcijskog dijela mozga, ali zbog različitog nabiranja mozga njihova fizička lokacija značajno varira među pojedinicima, čak i među identičnim blizanicima. Tako nema nikakve konzistentnosti u površinskim signalima.
Our breakthrough was to create an algorithm that unfolds the cortex, so that we can map the signals closer to its source, and therefore making it capable of working across a mass population. The second challenge is the actual device for observing brainwaves. EEG measurements typically involve a hairnet with an array of sensors, like the one that you can see here in the photo. A technician will put the electrodes onto the scalp using a conductive gel or paste and usually after a procedure of preparing the scalp by light abrasion. Now this is quite time consuming and isn't the most comfortable process. And on top of that, these systems actually cost in the tens of thousands of dollars.
Naše otkriće je što smo napravili algoritam koji je "izravnao" korteks, kako bismo mogli mapirati signale bliže njihovu izvoru, što nas čini sposobnim za rad s masama. Drugi izazov jest sam uređaj za promatranje moždanih valova. EEG mjerenja tipično rade s mrežom senzora na glavi, poput ove koju možete vidjeti na slici. Tehničar će staviti elektrode izravno na glavu koristeći provodivi gel ili pastu i obično nakon pripreme glave laganim struganjem kože. To sve zahtjeva dosta vremena i nije najudobniji proces. I pored svega, ti sustavi koštaju desetine tisuća dolara.
So with that, I'd like to invite onstage Evan Grant, who is one of last year's speakers, who's kindly agreed to help me to demonstrate what we've been able to develop.
Imajući to na umu, željela bih pozvati Evana Granta, koji je jedan o prošlogodišnjih govornika koji je ljubazno pristao pomoći mi demonstrirati ono što smo uspjeli razviti.
(Applause)
(Pljesak)
So the device that you see is a 14-channel, high-fidelity EEG acquisition system. It doesn't require any scalp preparation, no conductive gel or paste. It only takes a few minutes to put on and for the signals to settle. It's also wireless, so it gives you the freedom to move around. And compared to the tens of thousands of dollars for a traditional EEG system, this headset only costs a few hundred dollars. Now on to the detection algorithms. So facial expressions -- as I mentioned before in emotional experiences -- are actually designed to work out of the box with some sensitivity adjustments available for personalization. But with the limited time we have available, I'd like to show you the cognitive suite, which is the ability for you to basically move virtual objects with your mind.
Dakle, uređaj koji vidite jest 14-kanalni, jako precizni EEG sustav za mjerenje. Ne zahtjeva nikakvu pripremu glave, nikakav provodivi gel ili pastu. Treba mu samo nekoliko minuta da se postavi i da se signali smire. Također je bežičan, tako da vam daje slobodu kretanja. I u usporedbi s desetinama tisuća dolara za tradicionalni EEG sustav, ovaj uređaj košta samo nekoliko stotina dolara. A sada nešto o algoritmu za prepoznavanje signala. Dakle izrazi lica -- bitni za spoznaju o emocionalnim iskustvima -- su dizajnirani tako da funkcionira bez podešavanja a osjetljivost se može fino prilagoditi u ovisnosti o korisniku. Ali s obzirom da nam je vrijeme ograničeno, željela bih vam pokazati kognitivnu aplikaciju, koja omogućuje da umom pomičete virtualne objekte.
Now, Evan is new to this system, so what we have to do first is create a new profile for him. He's obviously not Joanne -- so we'll "add user." Evan. Okay. So the first thing we need to do with the cognitive suite is to start with training a neutral signal. With neutral, there's nothing in particular that Evan needs to do. He just hangs out. He's relaxed. And the idea is to establish a baseline or normal state for his brain, because every brain is different. It takes eight seconds to do this, and now that that's done, we can choose a movement-based action. So Evan, choose something that you can visualize clearly in your mind.
Evan nije prije koristio ovaj sustav, zato prvo moramo napraviti novi profil za njega. On očito nije Joanne -- dakle "dodati ćemo korisnika". Evan. OK. Dakle prva stvar koju moramo napraviti u kognitivnoj aplikaciji jest započeti s podešavanjem na neutralni signal. Na neutralnom, Evan ne treba ništa posebno napraviti. Miran je. Opušten. I ideja je da se formira baza ili normalno stanje njegova mozga, jer je svaki mozak drugačiji. Potrebno je osam sekundi za ovo. I sada kada je to napravljeno, možemo izabrati akciju vezanu za pokret. Dakle Evane izaberi nešto što možeš jasno vizualizirati u svom umu.
Evan Grant: Let's do "pull."
Evan Grant: "Idemo pokušati "povući"."
Tan Le: Okay, so let's choose "pull." So the idea here now is that Evan needs to imagine the object coming forward into the screen, and there's a progress bar that will scroll across the screen while he's doing that. The first time, nothing will happen, because the system has no idea how he thinks about "pull." But maintain that thought for the entire duration of the eight seconds. So: one, two, three, go. Okay. So once we accept this, the cube is live. So let's see if Evan can actually try and imagine pulling. Ah, good job! (Applause) That's really amazing.
Tan Le: "OK. Izaberimo "povući"." Dakle ideja je da Evan mora zamisliti objekt koji dolazi prema naprijed na ekran. Ovdje je pokazivač napretka koji klizi preko ekrana dok on to radi. Prvi puta, ništa se neće dogoditi, jer sustav nema pojma na koji način on zamišlja "povuci". Ali zadrži tu misao svih osam sekundi trajanja. Dakle: jedan, dva, tri, počni. OK. Kako smo jednom unijeli naredbu, kocka postaje živa. Pogledajmo može li Evan stvarno zamisliti "povlačenje". Ah, dobar posao! (Pljesak) To je prilično zadivljujuće.
(Applause)
(Pljesak)
So we have a little bit of time available, so I'm going to ask Evan to do a really difficult task. And this one is difficult because it's all about being able to visualize something that doesn't exist in our physical world. This is "disappear." So what you want to do -- at least with movement-based actions, we do that all the time, so you can visualize it. But with "disappear," there's really no analogies -- so Evan, what you want to do here is to imagine the cube slowly fading out, okay. Same sort of drill. So: one, two, three, go. Okay. Let's try that. Oh, my goodness. He's just too good. Let's try that again.
Imamo još malo vremena na raspolaganju, pa ću zamoliti Evana da izvede stvarno težak zadatak. A ovaj je težak jer treba zamisliti nešto što ne postoji u stvarnom svijetu. To je naredba "nestani". Ono što želimo -- barem vezano s akcijom kretanja to radimo stalno, tako da je možemo vizualizirati. Ali s "nestani", ne postoje stvarne analogije. Evane, zato sada želimo da zamisliš kocku kako polako nestaje, OK. Isti postupak. Dakle: jedan, dva, tri, počni. OK. Idemo pokušati to. Oh, moj Bože. On je jednostavno predobar. Pokušajmo ponovno.
EG: Losing concentration.
EG: "Gubim koncentraciju."
(Laughter)
(Smijeh)
TL: But we can see that it actually works, even though you can only hold it for a little bit of time. As I said, it's a very difficult process to imagine this. And the great thing about it is that we've only given the software one instance of how he thinks about "disappear." As there is a machine learning algorithm in this --
TL: "Ali možemo vidjeti da to stvarno radi, premda se ne možeš koncentrirati duže od trenutka." Ponavljam, jako je težak proces ovo zamisliti. I moćna stvar oko toga jest da smo dali softveru samo jedan primjer kako on misli o "nestati". Postoji algoritam stroja koji uči --
(Applause)
(Pljesak)
Thank you. Good job. Good job.
Hvala. Dobar posao. Dobar posao.
(Applause)
(Pljesak)
Thank you, Evan, you're a wonderful, wonderful example of the technology.
Hvala, Evane, ti si divan, divno si prikazao tehnologiju.
So, as you can see, before, there is a leveling system built into this software so that as Evan, or any user, becomes more familiar with the system, they can continue to add more and more detections, so that the system begins to differentiate between different distinct thoughts. And once you've trained up the detections, these thoughts can be assigned or mapped to any computing platform, application or device.
Kao što ste mogli vidjeti, postoji višerazinski sustav ugrađen u ovaj softver kako bi Evan, ili neki drugi korisnik, kada se priviknu na sustav, mogu nastaviti dodavati sve više obrazaca kako bi sustav mogao početi razlikovati različite obrasce misli. I jednom kada ste uvježbali obrasce, te misli se mogu dodijeliti ili prebaciti na bilo koju računalnu platformu, aplikaciju ili uređaj.
So I'd like to show you a few examples, because there are many possible applications for this new interface. In games and virtual worlds, for example, your facial expressions can naturally and intuitively be used to control an avatar or virtual character. Obviously, you can experience the fantasy of magic and control the world with your mind. And also, colors, lighting, sound and effects can dynamically respond to your emotional state to heighten the experience that you're having, in real time. And moving on to some applications developed by developers and researchers around the world, with robots and simple machines, for example -- in this case, flying a toy helicopter simply by thinking "lift" with your mind.
Zato bih vam voljela pokazati nekoliko primjera, jer postoji jako puno mogućih primjena ovog novog sučelja. U igrama i virtualnom svijetu, na primjer, vaš izraz lica može prirodno i intuitivno biti korišten za upravljanje avatarom ili virtualnim likom. Očito, možete iskusiti čaroliju mašte i upravljati svijetom svojim mislima. I također, boje, osvjetljenje, zvuk i efekti, mogu dinamično odgovarati na vaše emotivno stanje kako bi naglasili iskustvo koje imate, u realnom vremenu. Evo nekih primjera koji su razvili istraživači širom svijeta, s robotima i jednostavnim strojevima, na primjer -- u ovom slučaju, letenje igračkom helikopterom jednostavno razmišljajući o uzletanju.
The technology can also be applied to real world applications -- in this example, a smart home. You know, from the user interface of the control system to opening curtains or closing curtains. And of course, also to the lighting -- turning them on or off. And finally, to real life-changing applications, such as being able to control an electric wheelchair. In this example, facial expressions are mapped to the movement commands.
Tehnologija se može primjeniti u svakodnevnom životu -- u ovom primjeru, u pametnoj kući. Znate, od korisničkog sučelja kontrolnog sustava do otvaranja ili zatvaranja zastora. Naravno i osvijetljenje -- paleći ili gaseći ih. I konačno, aplikacija koje mogu izmjeniti život poput toga da smo sposobni upravljati električnim kolicima. U ovom primjeru, izrazi lica su povezani s naredbama za kretanje.
Man: Now blink right to go right. Now blink left to turn back left. Now smile to go straight.
Muškarac: Sada namigni desnim okom kako bi išao desno. Sada namigni lijevim okom kako bi išao lijevo. Sada se nasmiji za ravno.
TL: We really -- Thank you.
TL: Mi stvarno -- Hvala vam.
(Applause)
(Pljesak)
We are really only scratching the surface of what is possible today, and with the community's input, and also with the involvement of developers and researchers from around the world, we hope that you can help us to shape where the technology goes from here. Thank you so much.
Mi samo grebemo po površini onoga što je danas moguće. A s idejama zajednice, i uključenošću developera i istraživača širom svijeta, nadamo se da ćete nam pomoći oblikovati kamo ova tehnologije treba ići dalje. Hvala vam puno.