In my lab, we build autonomous aerial robots like the one you see flying here. Unlike the commercially available drones that you can buy today, this robot doesn't have any GPS on board. So without GPS, it's hard for robots like this to determine their position. This robot uses onboard sensors, cameras and laser scanners, to scan the environment. It detects features from the environment, and it determines where it is relative to those features, using a method of triangulation. And then it can assemble all these features into a map, like you see behind me. And this map then allows the robot to understand where the obstacles are and navigate in a collision-free manner.
在我的实验室, 我们打造了自动飞行的机器人, 就是你们眼前的这种。 不像当今市面上销售的那些, 这个机器人没有GPS定位系统。 在没有GPS的情况下, 这样的机器人很难定位自己。 这个机器人用机载传感器, 相机和激光扫描仪, 来扫描环境。 它能够探测到环境的特征, 并使用三角测量的方式来决定 不同的特征之间 有怎样的联系。 然后它能够把所有这些信息 整合到一张地图上, 就是我背后的这张图。 这个地图能让机器人确定 障碍物的位置, 并巧妙地避开各种碰撞。
What I want to show you next is a set of experiments we did inside our laboratory, where this robot was able to go for longer distances. So here you'll see, on the top right, what the robot sees with the camera. And on the main screen -- and of course this is sped up by a factor of four -- on the main screen you'll see the map that it's building. So this is a high-resolution map of the corridor around our laboratory. And in a minute you'll see it enter our lab, which is recognizable by the clutter that you see.
我接下来要展示的是 一组我们在户外进行的实验, 证明机器人能够在户外 进行长距离飞行。 你们可以看到,在右上方, 是机器人通过照相机看到的影像。 在主屏幕上—— 当然这是以4倍速度在播放展示—— 在主屏幕上,你们可以看到 地图的创建过程。 这是一个高分辨率的地图, 展示了我们实验室周围走廊的样子。 很快你们就能看到飞行机器人 进入了我们的实验室, 这么乱的地方,一看 就知道是我们的实验室。
(Laughter)
(笑声)
But the main point I want to convey to you is that these robots are capable of building high-resolution maps at five centimeters resolution, allowing somebody who is outside the lab, or outside the building to deploy these without actually going inside, and trying to infer what happens inside the building.
但我重点想说的是, 这些飞行器可以创建 高分辨率的地图, 达到5厘米的分辨率, 可以使那些在实验室外, 或者是房屋外的人, 无需进入房间即可观察到这些内容, 并尝试了解房屋内发生的状况。
Now there's one problem with robots like this. The first problem is it's pretty big. Because it's big, it's heavy. And these robots consume about 100 watts per pound. And this makes for a very short mission life. The second problem is that these robots have onboard sensors that end up being very expensive -- a laser scanner, a camera and the processors. That drives up the cost of this robot.
当然这样的机器人也有问题。 首先,它有些大了。 因为很大,所以很重。 这些机器人每磅重量可以消耗 100瓦的电能。 这使得它的运作时间很短。 第二个问题就是, 这些机器人有机载传感器, 它们非常昂贵—— 一个激光扫描仪,一个相机 以及处理器。 这些让这个飞行机器人的 制造成本十分高昂。
So we asked ourselves a question: what consumer product can you buy in an electronics store that is inexpensive, that's lightweight, that has sensing onboard and computation? And we invented the flying phone.
因此我们问了自己一个问题: 你能在电子产品商店买到怎样的物品, 它既不昂贵,又很轻, 还有机载传感器和计算能力? 于是我们发明了会飞的手机。
(Laughter)
(笑声)
So this robot uses a Samsung Galaxy smartphone that you can buy off the shelf, and all you need is an app that you can download from our app store. And you can see this robot reading the letters, "TED" in this case, looking at the corners of the "T" and the "E" and then triangulating off of that, flying autonomously. That joystick is just there to make sure if the robot goes crazy, Giuseppe can kill it.
这个机器人使用了你可以轻松购买到的 三星银河系列手机, 你所需要的只是一个应用程序, 可以从我们的应用程序商店下载。 你们可以看到这个飞行机器人在读字, “TED”, 观察字母“T”和“E”角落的位置, 再应用三角测量法,实现自主飞行。 那个手柄的作用仅限于 当飞行器失控时, Giuseppe就能让它失去功能。
(Laughter)
(笑声)
In addition to building these small robots, we also experiment with aggressive behaviors, like you see here. So this robot is now traveling at two to three meters per second, pitching and rolling aggressively as it changes direction. The main point is we can have smaller robots that can go faster and then travel in these very unstructured environments.
除了制造这些飞行机器人外, 我们还试着让它们做更激烈的动作, 比如这样。 这个飞行机器人目前的速度是 每秒钟2到3米, 在转变方向的时候 激烈地起降和翻转。 重点是,我们还能制造更小,更快的 飞行机器人, 它们可以在非常复杂的环境中飞行。
And in this next video, just like you see this bird, an eagle, gracefully coordinating its wings, its eyes and feet to grab prey out of the water, our robot can go fishing, too.
下一个影片中, 你们可以看到这只鹰, 优雅地协调它的翅膀, 眼睛和爪子之间的配合, 把猎物抓出水面, 我们的机器人也能去捕鱼。
(Laughter)
(笑声)
In this case, this is a Philly cheesesteak hoagie that it's grabbing out of thin air.
在这个试验中,它在空中抓起了 这个特大号菲力芝士牛排三明治。
(Laughter)
(笑声)
So you can see this robot going at about three meters per second, which is faster than walking speed, coordinating its arms, its claws and its flight with split-second timing to achieve this maneuver. In another experiment, I want to show you how the robot adapts its flight to control its suspended payload, whose length is actually larger than the width of the window. So in order to accomplish this, it actually has to pitch and adjust the altitude and swing the payload through. But of course we want to make these even smaller, and we're inspired in particular by honeybees. So if you look at honeybees, and this is a slowed down video, they're so small, the inertia is so lightweight --
你们可以看到这个机器人 以每秒3米左右的速度飞行, 比步行的速度快一些, 同时还能协调它的手臂和爪子, 以极快的速度完成整套动作。 在另一个试验中, 我想要展示, 机器人是如何依据悬浮载重 来调整飞行模式的, 它的总长度大于窗子的高度。 为了完成任务, 它需要向下倾斜,调整高度, 然后把重物摆动过去。 但是,我们希望让机器人变得更小, 而蜜蜂启发了我们。 如果你们观察一下蜜蜂, 在这个慢速的视频里, 它们是这么小, 惯性是这么轻 ——
(Laughter)
(笑声)
that they don't care -- they bounce off my hand, for example. This is a little robot that mimics the honeybee behavior. And smaller is better, because along with the small size you get lower inertia. Along with lower inertia --
它们并不在意—— 它们会撞击我的手,比如说。 这是个小机器人, 可以模拟蜜蜂的行为。 越小越好, 因为尺寸小,惯性就会小。 惯性小——
(Robot buzzing, laughter)
(机器人嗡嗡声,笑声)
along with lower inertia, you're resistant to collisions. And that makes you more robust. So just like these honeybees, we build small robots. And this particular one is only 25 grams in weight. It consumes only six watts of power. And it can travel up to six meters per second. So if I normalize that to its size, it's like a Boeing 787 traveling ten times the speed of sound.
惯性小,发生撞击的 可能性就低一些。 这使得机器人更耐用。 模仿这些蜜蜂, 我们制作了小型机器人。 这个特殊的机器人只有25克重。 它的耗电量仅为6瓦。 它的飞行速度可达每秒6米。 因此,如果我把它的尺寸按比例放大, 就好比一架波音787飞机 以10倍于音速的速度飞行。
(Laughter)
(笑声)
And I want to show you an example. This is probably the first planned mid-air collision, at one-twentieth normal speed. These are going at a relative speed of two meters per second, and this illustrates the basic principle. The two-gram carbon fiber cage around it prevents the propellers from entangling, but essentially the collision is absorbed and the robot responds to the collisions. And so small also means safe. In my lab, as we developed these robots, we start off with these big robots and then now we're down to these small robots. And if you plot a histogram of the number of Band-Aids we've ordered in the past, that sort of tailed off now. Because these robots are really safe.
我还要给你们展示一个例子。 这可能是第一个计划中的 以20分之1正常速度进行的空中相撞。 它们的相对速度为每秒2米, 这里展示了基本的原理。 两克重的碳纤维笼子包围着它 使螺旋桨不会受损, 但最关键的是,撞击被吸收了, 机器人能够对撞击做出反应。 并且,小也意味着安全。 在我的实验室, 最初开发这些机器人的时候, 我们从大的机器人开始 现在着手做小的机器人。 如果你们看一下我们购买 创可贴的数量统计图, 就知道现在我们已经几乎不需要买了。 因为这些机器人非常安全。
The small size has some disadvantages, and nature has found a number of ways to compensate for these disadvantages. The basic idea is they aggregate to form large groups, or swarms. So, similarly, in our lab, we try to create artificial robot swarms. And this is quite challenging because now you have to think about networks of robots. And within each robot, you have to think about the interplay of sensing, communication, computation -- and this network then becomes quite difficult to control and manage. So from nature we take away three organizing principles that essentially allow us to develop our algorithms. The first idea is that robots need to be aware of their neighbors. They need to be able to sense and communicate with their neighbors.
小尺寸也有一些缺点, 但是自然界有很多方法 来弥补这些缺陷。 最基本的想法是, 它们可以聚集在一起组成大型的群落。 因此,在实验室里, 我们也试着去组建机器人群组。 这是非常有难度的工作, 因为我们要考虑飞行机器人网络。 在每一个飞行器中, 我们都需要考虑传感,沟通和计算, 这些互相影响的因素—— 这样的机器人网络 不易控制和管理。 从大自然中,我们汲取了 三个组织原则, 最终帮助我们完成了算法的发展。 第一个是,机器人 需要注意到它的邻居们。 它们要有能力去感知相邻的机器人 并与它们交流。
So this video illustrates the basic idea. You have four robots -- one of the robots has actually been hijacked by a human operator, literally. But because the robots interact with each other, they sense their neighbors, they essentially follow. And here there's a single person able to lead this network of followers. So again, it's not because all the robots know where they're supposed to go. It's because they're just reacting to the positions of their neighbors.
这个视频展示了最基本的想法。 有4个机器人—— 其中的一个被人手动控制了。 但是因为机器人会相互通讯, 它们能够感知到旁边机器人的行动, 也会跟着它移动。 这个人能够领导整个 飞行机器人群体的行动。 再强调一次,这并不是因为 所有的机器人都知道它们要去哪里。 而是它们能根据相邻机器人位置变化 做出相应的反应。
(Laughter)
(笑声)
So the next experiment illustrates the second organizing principle. And this principle has to do with the principle of anonymity. Here the key idea is that the robots are agnostic to the identities of their neighbors. They're asked to form a circular shape, and no matter how many robots you introduce into the formation, or how many robots you pull out, each robot is simply reacting to its neighbor. It's aware of the fact that it needs to form the circular shape, but collaborating with its neighbors it forms the shape without central coordination. Now if you put these ideas together, the third idea is that we essentially give these robots mathematical descriptions of the shape they need to execute. And these shapes can be varying as a function of time, and you'll see these robots start from a circular formation, change into a rectangular formation, stretch into a straight line, back into an ellipse. And they do this with the same kind of split-second coordination that you see in natural swarms, in nature.
下一个实验展示了第二个组织原则。 这个原则与匿名原则有关。 其中的关键在于, 机器人是不知道与它们相邻的 机器人的身份的。 它们被要求形成一个圆圈, 不管你往这个阵列中放多少个机器人, 或者是你拿出来多少个机器人, 每个机器人都会很简单地 根据它相邻机器人的行为做出反应。 它清楚地知道需要形成圆圈, 与它的相邻机器人配合行动, 这个过程不需要中枢系统协调。 综合这两个想法之后, 第三个想法就是我们会 用数学方式来描述 它们需要组成的阵列形状。 这些形状会根据时间发生变化, 你们看到这些飞行器 从一个圆形开始, 接着变为长方形, 然后变成一条直线, 又变回椭圆形。 它们通过瞬间的协调 来完成这些动作, 就像自然界里的蜂群一样。
So why work with swarms? Let me tell you about two applications that we are very interested in. The first one has to do with agriculture, which is probably the biggest problem that we're facing worldwide. As you well know, one in every seven persons in this earth is malnourished. Most of the land that we can cultivate has already been cultivated. And the efficiency of most systems in the world is improving, but our production system efficiency is actually declining. And that's mostly because of water shortage, crop diseases, climate change and a couple of other things.
为什么要模仿蜂群呢? 我们对这项技术的两种应用 非常感兴趣。 第一个有关于农业, 农业应该是全球面临最严峻的问题。 大家都知道, 地球上每7个人中就有1个营养不良。 我们耕种了绝大部分可耕的土地。 世界上很多系统的效率都在提高, 但是我们生产系统的 效率事实上却是在下降。 很大程度上源于水源短缺, 作物病害,气候变化 和许多其他的原因。
So what can robots do? Well, we adopt an approach that's called Precision Farming in the community. And the basic idea is that we fly aerial robots through orchards, and then we build precision models of individual plants. So just like personalized medicine, while you might imagine wanting to treat every patient individually, what we'd like to do is build models of individual plants and then tell the farmer what kind of inputs every plant needs -- the inputs in this case being water, fertilizer and pesticide. Here you'll see robots traveling through an apple orchard, and in a minute you'll see two of its companions doing the same thing on the left side. And what they're doing is essentially building a map of the orchard. Within the map is a map of every plant in this orchard.
那么,这些机器人能做什么呢? 我们称之为社区里的精密耕作。 我们基本的想法是 让这些飞行机器人飞越果园, 为每一株植物搭建精密的模型。 就像个性化的机器, 想象每个病人都有一对一服务, 我们要做的就是 为每株植物单独建立模型, 然后告诉农民 每株植物各需要些什么—— 需要的因素在这里指的是水, 肥料和杀虫剂。 你们现在看到的是 机器人飞过一片苹果园, 很快你们就会看到另外两个机器人 在左侧做着同样的事。 它们在制作果园的地图。 这个地图里有果园里每株植物的位置。
(Robot buzzing)
(机器人嗡嗡声)
Let's see what those maps look like. In the next video, you'll see the cameras that are being used on this robot. On the top-left is essentially a standard color camera. On the left-center is an infrared camera. And on the bottom-left is a thermal camera. And on the main panel, you're seeing a three-dimensional reconstruction of every tree in the orchard as the sensors fly right past the trees. Armed with information like this, we can do several things. The first and possibly the most important thing we can do is very simple: count the number of fruits on every tree. By doing this, you tell the farmer how many fruits she has in every tree and allow her to estimate the yield in the orchard, optimizing the production chain downstream.
我们来看一下 这些地图是什么样子的。 在下一个视频中,你们会看到 机器人上安装的各种摄像头。 左上角的是一个高清彩色摄像机。 左边中间的是一个红外线摄像机。 左下方的是一个热成像摄像机。 在主面板上,你们会看到 每一株果树的三维图像。 这是当机器人飞过果树时 传感器采集到的数据。 有了这些信息, 我们就可以做很多事情。 我们能做的第一件也可能是最重要的事 非常简单: 清点每株果树上果实的数量。 这样,你就能告诉农民 每株果树上有多少果实, 农民就能估算出果园的产量, 并调整下游的生产链。
The second thing we can do is take models of plants, construct three-dimensional reconstructions, and from that estimate the canopy size, and then correlate the canopy size to the amount of leaf area on every plant. And this is called the leaf area index. So if you know this leaf area index, you essentially have a measure of how much photosynthesis is possible in every plant, which again tells you how healthy each plant is. By combining visual and infrared information, we can also compute indices such as NDVI. And in this particular case, you can essentially see there are some crops that are not doing as well as other crops. This is easily discernible from imagery, not just visual imagery but combining both visual imagery and infrared imagery.
我们能做的第二件事是 给植物建构三维立体图, 从而推算出树冠的尺寸, 然后通过树冠尺寸推算出 每株植物的树叶面积。 这被称为树叶面积指数。 那么如果我们知道一棵树的 树叶面积指数, 就能大概测算出这株果树 在进行多少光合作用, 就能知道这株果树的健康状况。 将视觉信息和红外线信息合起来, 我们还能计算出一些指数, 例如常态化差值植生指数。 在这个例子中, 你们可以明显看到 有一些植物并不像其他植物那样健康。 在图像中很容易识别, 不仅是在视觉图像中,而是结合了 视觉和红外线的图像中。
And then lastly, one thing we're interested in doing is detecting the early onset of chlorosis -- and this is an orange tree -- which is essentially seen by yellowing of leaves. But robots flying overhead can easily spot this autonomously and then report to the farmer that he or she has a problem in this section of the orchard.
最后, 我们感兴趣的另一件事 是检测早期的植被萎黄病—— 这是一棵橘子树—— 你们可以看到泛黄的树叶。 当机器人飞过这棵树的顶部 就能很快自主识别出来, 然后报告农民 他/她遇到问题了, 就在果园的这个区域。
Systems like this can really help, and we're projecting yields that can improve by about ten percent and, more importantly, decrease the amount of inputs such as water by 25 percent by using aerial robot swarms.
这样的系统真的非常实用, 我们估计这能帮助农业产量提升10%, 更重要的是,通过使用飞行器蜂群, 水资源的用量能够降低25%。
Lastly, I want you to applaud the people who actually create the future, Yash Mulgaonkar, Sikang Liu and Giuseppe Loianno, who are responsible for the three demonstrations that you saw.
最后,我希望大家为创造未来的人们 致以热烈的掌声, Yash Mulgaonkar, Sikang Liu 和Giuseppe Loianno, 你们看到的三次演示都是他们完成的。
Thank you.
谢谢。
(Applause)
(掌声)