Throughout the history of computers we've been striving to shorten the gap between us and digital information, the gap between our physical world and the world in the screen where our imagination can go wild. And this gap has become shorter, shorter, and even shorter, and now this gap is shortened down to less than a millimeter, the thickness of a touch-screen glass, and the power of computing has become accessible to everyone. But I wondered, what if there could be no boundary at all? I started to imagine what this would look like. First, I created this tool which penetrates into the digital space, so when you press it hard on the screen, it transfers its physical body into pixels. Designers can materialize their ideas directly in 3D, and surgeons can practice on virtual organs underneath the screen. So with this tool, this boundary has been broken. But our two hands still remain outside the screen. How can you reach inside and interact with the digital information using the full dexterity of our hands? At Microsoft Applied Sciences, along with my mentor Cati Boulanger, I redesigned the computer and turned a little space above the keyboard into a digital workspace. By combining a transparent display and depth cameras for sensing your fingers and face, now you can lift up your hands from the keyboard and reach inside this 3D space and grab pixels with your bare hands. (Applause) Because windows and files have a position in the real space, selecting them is as easy as grabbing a book off your shelf. Then you can flip through this book while highlighting the lines, words on the virtual touch pad below each floating window. Architects can stretch or rotate the models with their two hands directly. So in these examples, we are reaching into the digital world. But how about reversing its role and having the digital information reach us instead? I'm sure many of us have had the experience of buying and returning items online. But now you don't have to worry about it. What I got here is an online augmented fitting room. This is a view that you get from head-mounted or see-through display when the system understands the geometry of your body. Taking this idea further, I started to think, instead of just seeing these pixels in our space, how can we make it physical so that we can touch and feel it? What would such a future look like? At MIT Media Lab, along with my advisor Hiroshi Ishii and my collaborator Rehmi Post, we created this one physical pixel. Well, in this case, this spherical magnet acts like a 3D pixel in our space, which means that both computers and people can move this object to anywhere within this little 3D space. What we did was essentially canceling gravity and controlling the movement by combining magnetic levitation and mechanical actuation and sensing technologies. And by digitally programming the object, we are liberating the object from constraints of time and space, which means that now, human motions can be recorded and played back and left permanently in the physical world. So choreography can be taught physically over distance and Michael Jordan's famous shooting can be replicated over and over as a physical reality. Students can use this as a tool to learn about the complex concepts such as planetary motion, physics, and unlike computer screens or textbooks, this is a real, tangible experience that you can touch and feel, and it's very powerful. And what's more exciting than just turning what's currently in the computer physical is to start imagining how programming the world will alter even our daily physical activities. (Laughter) As you can see, the digital information will not just show us something but it will start directly acting upon us as a part of our physical surroundings without disconnecting ourselves from our world. Today, we started by talking about the boundary, but if we remove this boundary, the only boundary left is our imagination. Thank you. (Applause)
Pergjate gjithe historise se kompjuterave jemi perpjekur te ngushtojme hendekun midis nesh dhe informacionit digjital, hendekun midis botes sone fizike dhe asaj ne ekran ku imagjinata jone mund te shkoje jashte limiteve. Kjo hapesire eshte bere gjithnje e me e ngushte, gjithnje e me e vogel, sa tashme eshte zvogeluar ne me pak se nje milimeter, trashesia e nje ekrani xhami me prekje, dhe fuqia e kompjuterave i eshte mundesuar gjithkujt. Por mendova, po sikur te mos kish kufij aspak? Nisa te imagjonoja se si mund te ish kjo gje. Si fillim, krijova kete instrument qe penetron ne hapesiren dixhitale, keshtu qe, kur e shtyp fort ne ekran, e transferon trupin ne piksel. Ne kete menyre, dizajnerat mund ti materializojne idete e tyre pernjeheresh ne 3D, ndersa kirurget mund te bejne praktike ne organe vitale nen ekran. Pra, me kete instrument, ky kufi eshte thyer. Megjithate, duart tona serish mbeten jashte ekranit. Si mund te futesh brenda dhe te nderveprosh me informacionin dixhital duke perdorur duart tona? Ne Shkencat e Aplikuara te Microsoft, ne bashkepunim me mentoren time Cati Boulanger, ridizenjova kompjuterin dhe munda te kthej nje hapesire te vogel mbi tastjere ne nje vend pune dixhitale. Duke kombinuar nje ekran dhe kamera ne thellesi per te ndjere gishtat dhe fytyren, tani mund te ngresh gishtat nga tastiera te hysh brenda kesaj hapesire 3D dhe te kapesh piksel me duart e tua. (Duartrokitje) Sepse dritaret dhe dosjet kane nje pozicion ne hapesiren reale, dhe zgjedhja e tyre eshte po aq e lehte sa te marresh nje liber nga rafti. Me pas mund ta shfletosh kete liber ndersa nenvizon rreshtat, fjalet ne tastieren virtuale me prekje nen cdo dritare levizese. Arkitektet mund ti zgjasin ose rrotullojne modelet direkt me duart e tyre. Pra, ne keto shembuj, ne jemi duke iu afruar botes dixhitale. Po sikur ti ndryshonim rolet dhe te benim te mundur qe informacioni dixhital te vinte tek ne? Jam i sigurt qe shume prej nesh kane patur eksperience ne blerjen dhe kthimin e mallrave online. Tashme nuk keni pse merakoseni me. Cfare kam ketu, eshte nje dhome prove online. Kjo eshte pamja qe keni nga ekrani i tejdukshem ose mbi-koke ne momentin kur sistemi e kupton gjeometrine e trupit tend. Duke dashur ta zhvilloj kete ide me tej, nisa te mendoj se si ne vend qe ti shohim keto piksel ne hapesiren tone, si mund ta bejme te mundur qe te mund ta prekim dhe ta ndjejme? Si do te duket kjo e ardhme? Ne Media Lab te MIT, ne bashkepunim me keshilltarin tim Hiroshi Ishii dhe bashkepunetorin tim Rehmi Post, ne krijuam kete piksel fizik. Ne kete rast, ky magnet sferik vepron si nje piksel 3D ne hapesiren tone, cka do te thote se si kompjuterat ashtu edhe njerezit mund ti levizin keto objekte kudo neper kete hapesire te vogel 3D. Ajo cka beme, ishte te anulonim gravitetin dhe te kontrollonim levizshmerine duke kombinuar ngritjen magnetike dhe veprimin mekanik dhe teknologjite e ndjeshmerise. Dhe duke programuar dixhitalisht objektin, ne jemi duke e liruar ate nga kufizimet e hapesires dhe kohes, cka nenkupton se tani, levizjet njerezore mund te regjistrohen dhe te luhen perseri te mbetura pergjithnje ne boten fizike. Pra, koreografia mund te mesohet fizikisht ne distance dhe gjuajtjet e famshme te Michael Jordan mund te replikohen ne vazhdimesi si nje realitet fizik. Studentet mund ta perdorin kete si nje instrument per te mesuar rreth koncepteve komplekse si levizja planetare, fizika, dhe ndryshe nga ekranet e kompjuterave dhe librat, kjo eshte nje eksperience e vertete, e prekshme qe mund te preket dhe ndjehet, dhe eshte shume e fuqishme. Por ajo cka eshte me e bukur se kthimi ne dicka fizike i asaj qe eshte ne kompjuter eshte te fillosh te imagjinosh se si te programuarit e botes do te ndryshonin edhe aktivitetet tona te perditshme. (Te qeshura) Sikunder mund te shihni, informacioni dixhital nuk do te na tregoje thjesht dicka por do te nise menjehere te veproje mbi ne si pjese e rrethinave tona fizike pa na ndare nga bota jone. Sot, e nisem duk biseduar mbi kufijte, por nqse ne i heqim keto kufij, i vetmi kufi qe mbetet eshte imagjinata jone. Faleminderit. (Duartrokitje)