Throughout the history of computers we've been striving to shorten the gap between us and digital information, the gap between our physical world and the world in the screen where our imagination can go wild. And this gap has become shorter, shorter, and even shorter, and now this gap is shortened down to less than a millimeter, the thickness of a touch-screen glass, and the power of computing has become accessible to everyone. But I wondered, what if there could be no boundary at all? I started to imagine what this would look like. First, I created this tool which penetrates into the digital space, so when you press it hard on the screen, it transfers its physical body into pixels. Designers can materialize their ideas directly in 3D, and surgeons can practice on virtual organs underneath the screen. So with this tool, this boundary has been broken. But our two hands still remain outside the screen. How can you reach inside and interact with the digital information using the full dexterity of our hands? At Microsoft Applied Sciences, along with my mentor Cati Boulanger, I redesigned the computer and turned a little space above the keyboard into a digital workspace. By combining a transparent display and depth cameras for sensing your fingers and face, now you can lift up your hands from the keyboard and reach inside this 3D space and grab pixels with your bare hands. (Applause) Because windows and files have a position in the real space, selecting them is as easy as grabbing a book off your shelf. Then you can flip through this book while highlighting the lines, words on the virtual touch pad below each floating window. Architects can stretch or rotate the models with their two hands directly. So in these examples, we are reaching into the digital world. But how about reversing its role and having the digital information reach us instead? I'm sure many of us have had the experience of buying and returning items online. But now you don't have to worry about it. What I got here is an online augmented fitting room. This is a view that you get from head-mounted or see-through display when the system understands the geometry of your body. Taking this idea further, I started to think, instead of just seeing these pixels in our space, how can we make it physical so that we can touch and feel it? What would such a future look like? At MIT Media Lab, along with my advisor Hiroshi Ishii and my collaborator Rehmi Post, we created this one physical pixel. Well, in this case, this spherical magnet acts like a 3D pixel in our space, which means that both computers and people can move this object to anywhere within this little 3D space. What we did was essentially canceling gravity and controlling the movement by combining magnetic levitation and mechanical actuation and sensing technologies. And by digitally programming the object, we are liberating the object from constraints of time and space, which means that now, human motions can be recorded and played back and left permanently in the physical world. So choreography can be taught physically over distance and Michael Jordan's famous shooting can be replicated over and over as a physical reality. Students can use this as a tool to learn about the complex concepts such as planetary motion, physics, and unlike computer screens or textbooks, this is a real, tangible experience that you can touch and feel, and it's very powerful. And what's more exciting than just turning what's currently in the computer physical is to start imagining how programming the world will alter even our daily physical activities. (Laughter) As you can see, the digital information will not just show us something but it will start directly acting upon us as a part of our physical surroundings without disconnecting ourselves from our world. Today, we started by talking about the boundary, but if we remove this boundary, the only boundary left is our imagination. Thank you. (Applause)
Kroz istoriju kompjutera pokušavali smo da smanjimo jaz između nas i digitalne informacije, jaz između fizičkog sveta i sveta na ekranu gde naša mašta može da se rasplamsa. I ovaj jaz je postajao manji, manji, i još manji i sada je smanjen na manje od milimetra, debljinu stakla tač-skrina i moć računarstva je postala dostupna svima. Ali zapitao sam se, šta ako ne bi uopšte bilo granice? Počeo sam da zamišljam kako bi ovo izgledalo. Prvo sam konstruisao ovaj alat koji prodire duboko u digitalni prostor i kada ga pritisnete na ekran on prenosi svoje fizičko telo u piksele. Dizajneri mogu da materijalizuju svoje ideje direktno u 3D, a hirurzi mogu da vežbaju na virtuelnim organima ispod ekrana. Sa ovim alatom ova granica je srušena. Ali naše ruke još uvek ostaju van ekrana. Kako možete da doprete unutra i baratate digitalnim informacijama koristeći punu pokretljivost svojih ruku? U Primenjenim naukama Majkrosofta, zajedno sa mojim mentorom Keti Bulanž, redizajnirao sam kompjuter i pretvorio mali prostor iznad tastature u digitalni radni prostor. Kombinujući providni ekran i dubinske kamere za detektovanje prstiju i lica, sad možete da podignete ruke sa tastature i pružite ih u ovaj 3D prostor i uhvatite piksele svojim rukama. (Aplauz) Zbog toga što prozori i fajlovi imaju položaj u realnom prostoru, njihovo odabiranje je isto tako lako kao i uzimanje knjige sa police. Možete da listate knjigu, obeležavate redove i reči na virtuelnom tač-pedu ispod svakog lebdećeg prozora. Arhitekte mogu da izdužuju ili rotiraju modele direktno sa svoje dve ruke. U ovim primerima mi dodirujemo digitalni svet. Ali kako bi bilo da obrnemo uloge i da digitalna informacija dodiruje nas? Sigurno su mnogi od nas imali iskustvo u kupovini i vraćanju stvari na internetu. Ali ne morate da brinete o tome. Ovde imam uvećanu sobu za presvlačenje na internetu. Ovo je pogled koji dobijate od ekrana koji se kači na glavu ili providnog ekrana gde sistem razume geometriju vašeg tela. Razvijajući dalje ovu ideju, počeo sam da razmišljam: umesto da samo vidimo ove piksele u našem prostoru, kako bismo mogli da ih učinimo fizičkim tako da možemo da ih dodirnemo i osetimo? Kako bi izgledala takva budućnost? U Medijskoj laboratoriji MIT-a, zajedno sa mojim savetnikom Hiroši Išijem i mojim saradnikom Remijem Postom, napravili smo ovaj jedan fizički piksel. U ovom slučaju, ovaj sferični magnet se ponaša kao 3D piksel u našem prostoru što znači da i kompjuteri i ljudi mogu pomerati ovaj predmet bilo gde u okviru ovog malog 3D prostora. U suštini smo poništili gravitaciju i iskontrolisali pokret kombinujući magnetnu levitaciju i mehaničko delovanje i tehnologije senzora. Digitalnim programiranjem objekta, oslobodili smo ga ograničenja vremena i prostora, što znači da sada ljudski pokreti mogu da se snime i ponovo reprodukuju i da trajno ostanu u fizičkom svetu. Tako da koreografija može da se nauči fizički sa daljine i poznati šut Majkla Džordana može da se ponovi nebrojeno puta kao fizička realnost. Studenti mogu da koriste ovo kao alat za učenje o složenim pojmovima kao što su: kretanje planeta, fizika i za razliku od kompjuterskih ekrana ili udžbenika, ovo je pravo, opipljivo iskustvo koje možete dodirnuti i osetiti i vrlo je moćno. I ono što je još uzbudljivije od pretvaranja onoga što je na kompjuteru u fizičko, je početak razmišljanja o tome kako će programiranje sveta promeniti čak i naše dnevne fizičke aktivnosti. (Smeh) Kao što možete da vidite, digitalna informacija nam neće samo pokazivati nešto nego će direktno uticati na nas kao deo našeg fizičkog okruženja bez našeg isključivanja iz sveta. Danas smo počeli sa pričom o granici, ali ako sklonimo tu granicu, jedina granica koja će ostati je naša mašta. Hvala vam. (Aplauz)