Up until now, our communication with machines has always been limited to conscious and direct forms. Whether it's something simple like turning on the lights with a switch, or even as complex as programming robotics, we have always had to give a command to a machine, or even a series of commands, in order for it to do something for us. Communication between people, on the other hand, is far more complex and a lot more interesting because we take into account so much more than what is explicitly expressed. We observe facial expressions, body language, and we can intuit feelings and emotions from our dialogue with one another. This actually forms a large part of our decision-making process. Our vision is to introduce this whole new realm of human interaction into human-computer interaction so that computers can understand not only what you direct it to do, but it can also respond to your facial expressions and emotional experiences. And what better way to do this than by interpreting the signals naturally produced by our brain, our center for control and experience.
Sve do nedavno, komunikacija sa mašinama je bila ograničena na namerne i direktne postupke. Svejedno da li je u pitanju nešto jednostavno, recimo, paljenje svetla prekidačem, ili složeno, poput programiranja robota, mašini smo uvek morali zadati naredbu, ili, čak, niz naredbi, kako bi ona tad nešto izvršila. Komunikacija između ljudi je daleko složenija i zanimljivija jer obuhvata mnogo više od onoga što se iskazuje rečima. Bitan je izraz lica, govor tela, jer neposredno opažamo osećanja sagovornika iz samog dijaloga ali i ponašanja. To je jako veliki deo procesa donošenja odluka. Naš cilj je da uvedemo tu novu dimenziju komunikacije u interakciju čoveka sa kompjuterom, da omogućimo kompjuteru da razume ne samo šta treba da uradi, već da reaguje i na izraze lica i ispoljavanje emocija. A nema boljeg načina da postignemo cilj od tumačenja signala koje i inače emituje čovekov mozak, centar za kontrolu i opažanje.
Well, it sounds like a pretty good idea, but this task, as Bruno mentioned, isn't an easy one for two main reasons: First, the detection algorithms. Our brain is made up of billions of active neurons, around 170,000 km of combined axon length. When these neurons interact, the chemical reaction emits an electrical impulse, which can be measured. The majority of our functional brain is distributed over the outer surface layer of the brain, and to increase the area that's available for mental capacity, the brain surface is highly folded. Now this cortical folding presents a significant challenge for interpreting surface electrical impulses. Each individual's cortex is folded differently, very much like a fingerprint. So even though a signal may come from the same functional part of the brain, by the time the structure has been folded, its physical location is very different between individuals, even identical twins. There is no longer any consistency in the surface signals.
Jeste, zvuči kao jako lepa ideja, ali, kao što Bruno reče, nije je lako ostvariti, iz dva razloga: Prvi su algoritmi za detekciju. Naš mozak sačinjavaju milijarde aktivnih neurona, ukupne dužine aksona preko 170.000 km. Pri radu neurona, hemijske reakcije u njima prave električne impulse koji su merljivi. Deo mozga koji vrši nama potrebne funkcije je raspoređen po spoljnoj površini mozga. A radi uvećanja površine te aktivne oblasti kora mozga je veoma naborana. Zbog svog tog savijanja i nabiranja veliki je problem tačno razabrati električne impulse. Kora mozga svakog čoveka se nabira drugačije, jedinstveno, kao otisci prstiju. Mada signali dolaze iz istog moždanog centra, zbog nabiranja površine mozga lokacije samih centara vrlo variraju među ljudima, čak i kod identičnih blizanaca. Tako se gubi poreklo površinskih električnih signala.
Our breakthrough was to create an algorithm that unfolds the cortex, so that we can map the signals closer to its source, and therefore making it capable of working across a mass population. The second challenge is the actual device for observing brainwaves. EEG measurements typically involve a hairnet with an array of sensors, like the one that you can see here in the photo. A technician will put the electrodes onto the scalp using a conductive gel or paste and usually after a procedure of preparing the scalp by light abrasion. Now this is quite time consuming and isn't the most comfortable process. And on top of that, these systems actually cost in the tens of thousands of dollars.
Proboj smo napravili izradom algoritma koji "izravna" površinu velikog mozga, pa signale sad tačnije mapiramo na njihova izvorišta, što nam omogućava masovnu primenu. Drugi problem je sam uređaj za hvatanje moždanih talasa. EEG snimci se obično rade sa mrežicom za glavu načičkanom senzorima, kao ova na slici. Tehničar stavlja elektrode direktno na površinu glave namazanu provodnim gelom ili namazom, a sve to nakon pripremanja i plitke abrazije kože glave. To sve zahteva dosta vremena a nije baš ni prijatno. Povrh svega, ovakvi sistemi koštaju desetihe hiljada dolara.
So with that, I'd like to invite onstage Evan Grant, who is one of last year's speakers, who's kindly agreed to help me to demonstrate what we've been able to develop.
Sa svim tim na umu, molim vas da pozdravite Evana Granta, prošlogodišnjeg govornika, koji je ljubazno pristao da mi pomogne da pokažem šta smo napravili.
(Applause)
(Aplauz)
So the device that you see is a 14-channel, high-fidelity EEG acquisition system. It doesn't require any scalp preparation, no conductive gel or paste. It only takes a few minutes to put on and for the signals to settle. It's also wireless, so it gives you the freedom to move around. And compared to the tens of thousands of dollars for a traditional EEG system, this headset only costs a few hundred dollars. Now on to the detection algorithms. So facial expressions -- as I mentioned before in emotional experiences -- are actually designed to work out of the box with some sensitivity adjustments available for personalization. But with the limited time we have available, I'd like to show you the cognitive suite, which is the ability for you to basically move virtual objects with your mind.
Uređaj koji vidite je 14-kanalni, vrlo precizni EEG snimač. Ne zahteva prethodnu pripremu glave, nikakav provodni gel. Namesti se i kalibriše za dva-tri minuta. Takođe je bežičan, i omogućava punu slobodu kretanja. Za razliku od dosadašnjih EEG sistema, koji koštaju desetine hiljada dolara, ovaj uređaj košta nekoliko stotina dolara. Nazad na algoritme detekcije. Prepoznavanje izraza lica, -- bitno za prepoznavanje emotivnog stanja -- funkcioniše bez prethodnog podešavanja, a osetljivost se može naknadno fino podešavati prema korisniku. Ali, pošto nam je vreme ograničeno, prikazaću vam kognitivnu aplikaciju koja daje mogućnost da umom pomerate virtuelne objekte.
Now, Evan is new to this system, so what we have to do first is create a new profile for him. He's obviously not Joanne -- so we'll "add user." Evan. Okay. So the first thing we need to do with the cognitive suite is to start with training a neutral signal. With neutral, there's nothing in particular that Evan needs to do. He just hangs out. He's relaxed. And the idea is to establish a baseline or normal state for his brain, because every brain is different. It takes eight seconds to do this, and now that that's done, we can choose a movement-based action. So Evan, choose something that you can visualize clearly in your mind.
Evan prvi put koristi sistem, pa prvo moramo da mu napravimo lični profil. Očigledno, on nije "Džoana", dodaću korisnika... "Evan". OK... Prvo je nephodno da kalibrišemo aplikaciju na neutralan signal. Na neutralnom, Evan ne treba da radi ništa konkretno. Treba da je miran i opušten. Cilj je da se snimi "normalno" stanje njegovog mozga, jer svaki je mozak drugačiji. Ovo traje osam sekundi. Sad smo to završili, možemo da radimo neki pokret. Evane, izaberi nešto što možeš jasno da zamisliš.
Evan Grant: Let's do "pull."
Evan Grant: Hajde da "povučemo".
Tan Le: Okay, so let's choose "pull." So the idea here now is that Evan needs to imagine the object coming forward into the screen, and there's a progress bar that will scroll across the screen while he's doing that. The first time, nothing will happen, because the system has no idea how he thinks about "pull." But maintain that thought for the entire duration of the eight seconds. So: one, two, three, go. Okay. So once we accept this, the cube is live. So let's see if Evan can actually try and imagine pulling. Ah, good job! (Applause) That's really amazing.
Tan Le: Ok, izabrali smo "povlačenje". E sad, Evan bi sada trebalo da zamišlja kako povlači objekat ka ekranu, ka sebi. Tu je i pokazivač koji će da prati kalibraciju. Prvi put se neće ništa desiti, jer sistem ne zna kako Evan misli o "povlačenju". Ali ako zadrži tu misao svih osam sekundi... Ajmo: jedan, dva, tri, kreni! OK. Nakon što smo uneli komandu, kocka može odmah da reaguje. Hajde da vidimo da li Evan može da je povuče. Aaa, odlično! (aplauz) To je bilo neverovatno.
(Applause)
(tapšu)
So we have a little bit of time available, so I'm going to ask Evan to do a really difficult task. And this one is difficult because it's all about being able to visualize something that doesn't exist in our physical world. This is "disappear." So what you want to do -- at least with movement-based actions, we do that all the time, so you can visualize it. But with "disappear," there's really no analogies -- so Evan, what you want to do here is to imagine the cube slowly fading out, okay. Same sort of drill. So: one, two, three, go. Okay. Let's try that. Oh, my goodness. He's just too good. Let's try that again.
Imamo još nešto vremena, pa ću zamoliti Evana da probamo teži zadatak. Ovaj zadatak je teži jer treba zamisliti nešto što ne postoji u svakodnevici. To je komanda "nestani". Komande vezane za kretanje su lake, to radimo i zamišljamo i inače, ali za "nestani" nema dobrih analogija. Dakle, Evane, zamisli kocku kako polako nestaje. Isti je postupak. Jedan, dva, tri. Ok, sad da probamo. O moj bože. Mnogo je dobar. Probaj opet.
EG: Losing concentration.
Evan: Gubim koncentraciju.
(Laughter)
(smeju se)
TL: But we can see that it actually works, even though you can only hold it for a little bit of time. As I said, it's a very difficult process to imagine this. And the great thing about it is that we've only given the software one instance of how he thinks about "disappear." As there is a machine learning algorithm in this --
Tan Le: Vidimo da proces počinje, mada ne možeš dugo da održiš misao. Kažem opet, jako je teško zamišljati "nestajanje". Ono što je bitno je da smo programu samo jednom dali kako Evan misli "nestani". Program je u stanju da uči --
(Applause)
(aplauz)
Thank you. Good job. Good job.
Hvala. Bravo, bravo.
(Applause)
(tapšu)
Thank you, Evan, you're a wonderful, wonderful example of the technology.
Hvala ti, Evane, divno si, divno prikazao ovu tehnologiju.
So, as you can see, before, there is a leveling system built into this software so that as Evan, or any user, becomes more familiar with the system, they can continue to add more and more detections, so that the system begins to differentiate between different distinct thoughts. And once you've trained up the detections, these thoughts can be assigned or mapped to any computing platform, application or device.
Kao što vidite, postoji više nivoa i komandi u progamu, tako da Evan, i bilo koji drugi korisnik, kad se privikne na sistem, može da dodaje sve više i više obrazaca, te će sistem biti u stanju da razlikuje više različitih pojedinačnih misli. A jednom kad se potpuno izvežbate, te komande se mogu dodeliti ili prebaciti drugim kompjuterskim platformama, programima ili uređajima.
So I'd like to show you a few examples, because there are many possible applications for this new interface. In games and virtual worlds, for example, your facial expressions can naturally and intuitively be used to control an avatar or virtual character. Obviously, you can experience the fantasy of magic and control the world with your mind. And also, colors, lighting, sound and effects can dynamically respond to your emotional state to heighten the experience that you're having, in real time. And moving on to some applications developed by developers and researchers around the world, with robots and simple machines, for example -- in this case, flying a toy helicopter simply by thinking "lift" with your mind.
Prikazaću vam samo nekoliko primera, jer ima puno mogućih primena za ovaj novi interfejs. U igrama i virtuelnim svetovima, vaš izraz lica se može prirodno i interaktivno koristiti da upravljate avatarom ili virtuelnim likom. Možete iskusiti fantastiku magije i kontrolisati svet svojim mislima. Boje, osvetljenje, zvuk i efekti, sve može dinamički da reaguje na vaše raspoloženje da dodatno pojača doživljaj. Evo nekih primena razvijanih širom sveta sa robotima i jednostavnim uređajima. Ovo je mali helikopter, koji poleće na misaonu komandu "poleti".
The technology can also be applied to real world applications -- in this example, a smart home. You know, from the user interface of the control system to opening curtains or closing curtains. And of course, also to the lighting -- turning them on or off. And finally, to real life-changing applications, such as being able to control an electric wheelchair. In this example, facial expressions are mapped to the movement commands.
Ista tehnologija se može primeniti u svakodnevnom životu, recimo u "pametnoj kući". Iz interfejsa kontrolog sistema možete razvući zavese, navući zavese. Naravno, osvetljenje. Možete paliti ili gasiti svetla. I najzad, primene koje menjaju živote ljudima, poput mogućnosti da se upravlja električnim kolicima. U ovom primeru, izrazi lica se povezuju na kontrole kretanja.
Man: Now blink right to go right. Now blink left to turn back left. Now smile to go straight.
"Sad namigni desnim da skreneš desno." "Sad levim, da skreneš levo." "Sad se smeši da ideš pravo."
TL: We really -- Thank you.
TL: Hvala...
(Applause)
(aplauz)
We are really only scratching the surface of what is possible today, and with the community's input, and also with the involvement of developers and researchers from around the world, we hope that you can help us to shape where the technology goes from here. Thank you so much.
Stvarno smo tek na početku shvatanja šta je sve sa ovim moguće. A sa uplivom zajednice, sa tehničkom saradnjom, sa istraživačima širom sveta, nadamo se da ćete nam pomoći da oblikujemo budućnost ove tehnologije. Hvala vam puno.