Throughout the history of computers we've been striving to shorten the gap between us and digital information, the gap between our physical world and the world in the screen where our imagination can go wild. And this gap has become shorter, shorter, and even shorter, and now this gap is shortened down to less than a millimeter, the thickness of a touch-screen glass, and the power of computing has become accessible to everyone. But I wondered, what if there could be no boundary at all? I started to imagine what this would look like. First, I created this tool which penetrates into the digital space, so when you press it hard on the screen, it transfers its physical body into pixels. Designers can materialize their ideas directly in 3D, and surgeons can practice on virtual organs underneath the screen. So with this tool, this boundary has been broken. But our two hands still remain outside the screen. How can you reach inside and interact with the digital information using the full dexterity of our hands? At Microsoft Applied Sciences, along with my mentor Cati Boulanger, I redesigned the computer and turned a little space above the keyboard into a digital workspace. By combining a transparent display and depth cameras for sensing your fingers and face, now you can lift up your hands from the keyboard and reach inside this 3D space and grab pixels with your bare hands. (Applause) Because windows and files have a position in the real space, selecting them is as easy as grabbing a book off your shelf. Then you can flip through this book while highlighting the lines, words on the virtual touch pad below each floating window. Architects can stretch or rotate the models with their two hands directly. So in these examples, we are reaching into the digital world. But how about reversing its role and having the digital information reach us instead? I'm sure many of us have had the experience of buying and returning items online. But now you don't have to worry about it. What I got here is an online augmented fitting room. This is a view that you get from head-mounted or see-through display when the system understands the geometry of your body. Taking this idea further, I started to think, instead of just seeing these pixels in our space, how can we make it physical so that we can touch and feel it? What would such a future look like? At MIT Media Lab, along with my advisor Hiroshi Ishii and my collaborator Rehmi Post, we created this one physical pixel. Well, in this case, this spherical magnet acts like a 3D pixel in our space, which means that both computers and people can move this object to anywhere within this little 3D space. What we did was essentially canceling gravity and controlling the movement by combining magnetic levitation and mechanical actuation and sensing technologies. And by digitally programming the object, we are liberating the object from constraints of time and space, which means that now, human motions can be recorded and played back and left permanently in the physical world. So choreography can be taught physically over distance and Michael Jordan's famous shooting can be replicated over and over as a physical reality. Students can use this as a tool to learn about the complex concepts such as planetary motion, physics, and unlike computer screens or textbooks, this is a real, tangible experience that you can touch and feel, and it's very powerful. And what's more exciting than just turning what's currently in the computer physical is to start imagining how programming the world will alter even our daily physical activities. (Laughter) As you can see, the digital information will not just show us something but it will start directly acting upon us as a part of our physical surroundings without disconnecting ourselves from our world. Today, we started by talking about the boundary, but if we remove this boundary, the only boundary left is our imagination. Thank you. (Applause)
在整個電腦的發展歷程中 我們不斷地嘗試縮短 我們與數位資訊之間的距離 也就是實體世界 與螢幕中虛擬世界的距離 在虛擬世界裡 我們總是可以盡情發揮想像力 這兩者之間的距離也確實縮短了 比較近 還更再更近 現在這個距離已經縮短到 不到一毫米了 也就是觸控式螢幕厚度的距離 任何人 都可以使用電腦 但我在想 有沒有可能 我們跟電腦之間變成完全零距離 我開始想像那會是什麼樣子 首先 我做出了這個工具 它可以穿入到數位空間裡 所以當你用力地壓螢幕時 這個工具可以把你的身體 轉換成為螢幕上的像素 做設計的人可以 把他們的想法直接3D實體化 外科醫生也可以在 螢幕中的虛擬器官上做練習 我們跟電腦之間的距離 便隨著這個工具的發明而被打破了 即使如此 我們的手還是停留在螢幕外 有沒有可能直接把手伸進電腦 透過我們靈巧的雙手 直接使用這些數位資訊呢? 我在微軟的應用科學部門裡 和著我的指導教授 Cati Boulanger 重新設計出這台電腦 把鍵盤上面這個小小的空間 轉變成為一個數位化工作區 是透過一個透明顯示器 跟3D距離相機的組合 去感應你的手指跟臉 於是你就可以將你的手 從鍵盤上抬起來 並且就此進入3D的空間 並且可以直接用你的手 來抓住像素 (鼓掌聲) 因為視窗跟檔案 都在真實地存在這個空間裡 選取它們就好像 從書架上拿一本書一樣容易 你也可以這樣翻閱這本書 當要在某句話 或某些字上畫重點的時候 就螢幕下方的觸控板上劃過去即可 建築師可以直接用他們的雙手 來伸展或把模型轉過來 所以在這些例子裡面 我們是真的進入了數位的世界 還有沒有可能是相反過來 讓數位資訊直接跑到我們面前呢 我相信我們當中許多人 都有過在網上購物或退貨的經驗 現在你不需要再擔心了 你所看到的是一個網路實境試衣間 當系統辨識出你的身型之後 這個影像就會 透過頭戴式或透明顯示器穿戴在你身上 延伸這個想法,我開始想 有沒有可能不光是 在三度空間裡用肉眼看到像素 而是讓像素具體化 讓我們可以摸得到 跟感覺得到它呢 這樣的未來會是什麼樣子? 在麻省理工學院的媒體實驗室裡 我跟我的指導教授 Hiroshi Ishii 以及我的合作夥伴 Rehmi Post 一起研發出這一個具像化的畫素 這組模組中的球狀磁鐵 就好像是現實生活中的3D像素 也就是說無論是 使用電腦或人用手 都可以在這個小小的三度空間裡 自由且同步地移動它 基本上我們所做的 就是拿掉重力這個因素 透過結合磁浮 及力學效應 再加上感應技術 來讓它動起來 並透過程式來將物件數位化 就讓它脫離了時間空間的限制 也就是說現在 人的動作可以被紀錄並重新播放 並且在現實生活中被永久保存下來 所以現在芭蕾舞 也可以進行遠距教學 麥可喬丹的傳奇飛人之姿 也可以真實地一再重現 學生也可以使用它 來學習像行星運動或物理學 這類比較複雜的概念 相較於一般的電腦螢幕或教科書 這是個有形有體的真實經驗 你可以觸摸跟實際感受 這會令人印象深刻 更令人興奮的是 不是只是將電腦裡的東西實體化 而是當我們生活週遭 很多東西開始被程式化之後 我們的日常生活也將會隨之改變 (笑聲) 就如各位所看到的 數位資訊將不只提供知識或想法 而是直接在我們面前活生生地呈現出來 就好像是我們生活週遭的一部份 我們不再需要將自己 從這個世界中抽離出來 我們今天是從這層隔閡開始講起 但如果這層隔閡不再復存 那唯一會限制我們的 就只剩下自己的想像力了 謝謝 (鼓掌聲)