Throughout the history of computers we've been striving to shorten the gap between us and digital information, the gap between our physical world and the world in the screen where our imagination can go wild. And this gap has become shorter, shorter, and even shorter, and now this gap is shortened down to less than a millimeter, the thickness of a touch-screen glass, and the power of computing has become accessible to everyone. But I wondered, what if there could be no boundary at all? I started to imagine what this would look like. First, I created this tool which penetrates into the digital space, so when you press it hard on the screen, it transfers its physical body into pixels. Designers can materialize their ideas directly in 3D, and surgeons can practice on virtual organs underneath the screen. So with this tool, this boundary has been broken. But our two hands still remain outside the screen. How can you reach inside and interact with the digital information using the full dexterity of our hands? At Microsoft Applied Sciences, along with my mentor Cati Boulanger, I redesigned the computer and turned a little space above the keyboard into a digital workspace. By combining a transparent display and depth cameras for sensing your fingers and face, now you can lift up your hands from the keyboard and reach inside this 3D space and grab pixels with your bare hands. (Applause) Because windows and files have a position in the real space, selecting them is as easy as grabbing a book off your shelf. Then you can flip through this book while highlighting the lines, words on the virtual touch pad below each floating window. Architects can stretch or rotate the models with their two hands directly. So in these examples, we are reaching into the digital world. But how about reversing its role and having the digital information reach us instead? I'm sure many of us have had the experience of buying and returning items online. But now you don't have to worry about it. What I got here is an online augmented fitting room. This is a view that you get from head-mounted or see-through display when the system understands the geometry of your body. Taking this idea further, I started to think, instead of just seeing these pixels in our space, how can we make it physical so that we can touch and feel it? What would such a future look like? At MIT Media Lab, along with my advisor Hiroshi Ishii and my collaborator Rehmi Post, we created this one physical pixel. Well, in this case, this spherical magnet acts like a 3D pixel in our space, which means that both computers and people can move this object to anywhere within this little 3D space. What we did was essentially canceling gravity and controlling the movement by combining magnetic levitation and mechanical actuation and sensing technologies. And by digitally programming the object, we are liberating the object from constraints of time and space, which means that now, human motions can be recorded and played back and left permanently in the physical world. So choreography can be taught physically over distance and Michael Jordan's famous shooting can be replicated over and over as a physical reality. Students can use this as a tool to learn about the complex concepts such as planetary motion, physics, and unlike computer screens or textbooks, this is a real, tangible experience that you can touch and feel, and it's very powerful. And what's more exciting than just turning what's currently in the computer physical is to start imagining how programming the world will alter even our daily physical activities. (Laughter) As you can see, the digital information will not just show us something but it will start directly acting upon us as a part of our physical surroundings without disconnecting ourselves from our world. Today, we started by talking about the boundary, but if we remove this boundary, the only boundary left is our imagination. Thank you. (Applause)
Skozi zgodovino razvoja računalnikov se trudimo premostiti vrzel med človekom in digitalno informacijo, mejo med našim fizičnim svetom in svetom v zaslonu, v katerem je naša domišljija brez ovir. Meja med svetovoma je postajala vse tanjša, tanjša, in še tanjša, danes pa je stanjšana na manj kot milimeter, na debelino stekla pri zaslonu na dotik, in tako je računalništvo postalo dostopno vsakomur. A vprašal sem se, kaj pa če sploh ne bi bilo nikakršne meje? Začel sem si predstavljati, kako bi to izgledalo. Najprej sem izdelal orodje, ki prodre v digitalni prostor, tako da z močnim pritiskom na zaslon svojo fizično obliko pretvori v piksle. Oblikovalci lahko svoje ideje ustvarjajo neposredno v 3D in kirurgi lahko vadijo na virtualnih organih pod zaslonom. S tem orodjem je ta meja premagana. Toda naši dve roki še vedno ostajata izven zaslona. Kako seči vanj in vplivati na digitalno informacijo tako, da lahko v celoti izkoristimo svojo ročno spretnost? Pri Microsoft Applied Sciences sem skupaj z mentorico Cati Boulanger preoblikoval računalnik tako, da sem malo prostora nad tipkovnico preuredil v digitalno delovno površino. Kombinacija prosojnega zaslona in globinskih kamer za zaznavanje vaših prstov in obraza omogoča, da zdaj svoje roke lahko dvignete od tipkovnice in sežete v 3D prostor, ter primete piksle z lastnimi rokami. (Aplavz.) Ker so okna in datoteke razporejeni v realnem prostoru, jih je tako preprosto izbrati, kot bi vzeli knjigo s police. Lahko listate po knjigi, medtem ko označujete vrstice ali besede na virtualni ploščici na dotik pod vsakim lebdečim oknom. Tako lahko arhitekti raztegnejo ali zavrtijo modele neposredno s svojima rokama. V navedenih primerih mi segamo v digitalni svet. Kaj pa če zamenjamo vlogi in digitalne informacije sežejo do nas? Prepričan sem, da imamo mnogi med nami izkušnje s kupovanjem in vračanjem blaga na spletu. Zdaj vam glede tega ni več treba skrbeti. Tule lahko spremljate spletno preoblačilnico. To je pogled, ki ga vidite na naglavnem ali prosojnem prikazovalniku, ko sistem spozna geometrijo vašega telesa. Idejo sem razvijal dalje in pomislil, kako bi, namesto da so zgolj vidni v prostoru, piksle opredmetili, tako da jih lahko otipamo in čutimo? Kako bi izgledala taka prihodnost? V MIT Media Labu sem skupaj s svojim svetovalcem Hiroshijem Ishiijem in sodelavcem Rehmijem Postom izdelal ta edinstven fizični piksel. No, v tem primeru ta sferični magnet deluje kot 3D piksel v našem prostoru, kar pomeni, da ga lahko tako računalniki kot ljudje premikajo kamor koli znotraj tega omejenega 3D prostora. Kar smo napravili, je v osnovi izničenje gravitacije in nadzorovanje gibanja s kombinacijo magnetnega lebdenja in tehnologije mehanskega aktiviranja in krmiljenja. Z digitalnim programiranjem objekt osvobodimo omejitev, ki jih postavljata čas in prostor, kar pomeni, da se lahko zdaj gibanje človeškega telesa posname in ponovno predvaja, ter trajno ostane v fizičnem svetu. Tako je mogoče koreografijo učiti fizično na daljavo in slavni meti Michaela Jordana se lahko ponavljajo znova in znova kot fizična realnost. Študenti lahko to uporabijo kot pripomoček za učenje kompleksnih pojmov, kot je gibanje planetov, fizika, toda nasprotno od računalniških zaslonov in učbenikov tak način predstavlja resnično, otipljivo izkušnjo, ki jo otipaš in čutiš, in je zelo intenzivna. Še bolj vznemirljivo kot zgolj pretvarjati trenutne vsebine iz računalnika v fizično obliko, pa si je zamisliti, kako bo programiranje sveta spremenilo tudi naše vsakodnevne fizične aktivnosti. (Smeh.) Kot vidite, nam digitalne informacije ne bodo zgolj nečesa prikazovale, ampak bodo neposredno delovale na nas kot del našega fizičnega okolja, ne da bi se nam bilo potrebno ločiti od svojega sveta. Danes smo začeli z govorom o meji, toda če to mejo odstranimo, je edina omejitev naša domišljija. Hvala. (Aplavz.)