Throughout the history of computers we've been striving to shorten the gap between us and digital information, the gap between our physical world and the world in the screen where our imagination can go wild. And this gap has become shorter, shorter, and even shorter, and now this gap is shortened down to less than a millimeter, the thickness of a touch-screen glass, and the power of computing has become accessible to everyone. But I wondered, what if there could be no boundary at all? I started to imagine what this would look like. First, I created this tool which penetrates into the digital space, so when you press it hard on the screen, it transfers its physical body into pixels. Designers can materialize their ideas directly in 3D, and surgeons can practice on virtual organs underneath the screen. So with this tool, this boundary has been broken. But our two hands still remain outside the screen. How can you reach inside and interact with the digital information using the full dexterity of our hands? At Microsoft Applied Sciences, along with my mentor Cati Boulanger, I redesigned the computer and turned a little space above the keyboard into a digital workspace. By combining a transparent display and depth cameras for sensing your fingers and face, now you can lift up your hands from the keyboard and reach inside this 3D space and grab pixels with your bare hands. (Applause) Because windows and files have a position in the real space, selecting them is as easy as grabbing a book off your shelf. Then you can flip through this book while highlighting the lines, words on the virtual touch pad below each floating window. Architects can stretch or rotate the models with their two hands directly. So in these examples, we are reaching into the digital world. But how about reversing its role and having the digital information reach us instead? I'm sure many of us have had the experience of buying and returning items online. But now you don't have to worry about it. What I got here is an online augmented fitting room. This is a view that you get from head-mounted or see-through display when the system understands the geometry of your body. Taking this idea further, I started to think, instead of just seeing these pixels in our space, how can we make it physical so that we can touch and feel it? What would such a future look like? At MIT Media Lab, along with my advisor Hiroshi Ishii and my collaborator Rehmi Post, we created this one physical pixel. Well, in this case, this spherical magnet acts like a 3D pixel in our space, which means that both computers and people can move this object to anywhere within this little 3D space. What we did was essentially canceling gravity and controlling the movement by combining magnetic levitation and mechanical actuation and sensing technologies. And by digitally programming the object, we are liberating the object from constraints of time and space, which means that now, human motions can be recorded and played back and left permanently in the physical world. So choreography can be taught physically over distance and Michael Jordan's famous shooting can be replicated over and over as a physical reality. Students can use this as a tool to learn about the complex concepts such as planetary motion, physics, and unlike computer screens or textbooks, this is a real, tangible experience that you can touch and feel, and it's very powerful. And what's more exciting than just turning what's currently in the computer physical is to start imagining how programming the world will alter even our daily physical activities. (Laughter) As you can see, the digital information will not just show us something but it will start directly acting upon us as a part of our physical surroundings without disconnecting ourselves from our world. Today, we started by talking about the boundary, but if we remove this boundary, the only boundary left is our imagination. Thank you. (Applause)
纵观计算机中的历史 我们一直在努力弥合 我们与数字信息之间的隔阂, 还有我们的现实世界 与虚拟世界之间的差距。 在虚拟世界里,我们的想象力是无穷的。 这一差距已经在缩短, 还会继续缩短, 现在这一差距已经缩短到 小于一毫米, 就是触摸屏的玻璃那么薄, 而且计算机已是 人皆有之。 但我在想,要是没有虚拟和现实世界之间的边界该多好? 我于是就开始大胆尝试这个想法。 我首先造出了这个工具, 它可以深入数字空间, 如果你拿这支笔用力按这个屏幕, 这支笔的笔身就会进到由像素点组成的虚拟世界中。 设计师们可以直接在三维立体空间内 实现他们的想法, 外科医生可以在屏幕里面 练习给虚拟器官动手术。 可见,这个工具打破了虚拟和现实世界之间的边界。 但我们双手还是在屏幕外。 我们怎样才能把手伸进去 和数字信息进行交互, 从而充分利用我们灵活的双手? 我在微软应用科学实验室, 和我的导师卡迪•布兰葛 重新设计了这台计算机 把键盘上方的小空间 挪进数字工作区内。 通过将一个透明显示屏与几个深度摄像头结合, 这几个摄像头可以感知你的手指和脸, 现在你可以从键盘上抬起你的双手 伸进这个三维立体空间去 徒手控制里面的虚拟物质——像素点。 (掌声) 因为窗口和文件在现实空间内有了具体位置, 选取它们变得像从书架上拿一本书那么容易。 然后你可以翻阅这本书 并在每个浮动窗口下面的虚拟触控板上 划出字词和句子。 建筑师们可以直接用他们的双手 拉伸或旋转各种模型。 在这些例子表明, 我们正在进入数字世界。 可是如何才能转变它的角色 从而让数字信息进入我们的世界中? 相信在座很多人都有 在线购物和退货的经验。 不过,现在你不比再担心。 因为,我这里有个增强版在线试衣间。 这是这个系统分析了我们身体的几何特征后, 通过头载式或透视式显示方式 展示给我们看的情形。 不过,我又开始进一步思考这一想法, 怎样才能让我们空间里的这些可见的像素点 转变为实实在在 能摸得到、感觉到的物质? 这种未来会是什么样子的? 在麻省理工学院媒体实验室里,我和我的导师石井裕 还有我的搭档雷米·波斯特, 共同研制出这一“物质像素点”。 在这种情况下,这个球形磁体 就好像是我们空间中的一个三维像素点, 这就意味着计算机和人 可以将这个物体移动到 这一三维立体小空间的任何位置。 我们的基本实现原理是,消除重力对小球的作用 并通过磁悬浮、机械驱动 和传感技术的结合 来控制小球的运动。 通过对这个对象的数字编程, 我们正试着解放 时间和空间对物质的限制,这意味着,现在, 人的运动可以在物质世界中可以被 录制、重放和永久保留。 这样,舞蹈编排就可以被远程而且真实地传授, 迈克尔·乔丹著名的扣篮动作就可以 反反复复、实实在在地被重演。 学生可以把它当作一种工具 去学习那些复杂的概念 比如行星运动、物理, 而且不同于电脑屏幕或教科书的是, 这种学习过程是真实存在的 你可以触摸它、感觉它,很给力! 更令人兴奋的是, 这不止是把当前计算机的东西转变为真实的物质, 而是开始想象,编程的世界甚至将 改变我们日常的体能活动。 (笑声) 如您所见,数字信息 将不仅仅是把东西显示给我们 而是将开始直接作用于我们 成为我们现实环境的一部分 并非切断我们与现实世界的联系。 我今天的演讲是从谈论“边界”问题开始的, 如果我们能除去这一边界, 剩下的就是我们无边无际的想像力了。 谢谢! (掌声)