In my lab, we build autonomous aerial robots like the one you see flying here. Unlike the commercially available drones that you can buy today, this robot doesn't have any GPS on board. So without GPS, it's hard for robots like this to determine their position. This robot uses onboard sensors, cameras and laser scanners, to scan the environment. It detects features from the environment, and it determines where it is relative to those features, using a method of triangulation. And then it can assemble all these features into a map, like you see behind me. And this map then allows the robot to understand where the obstacles are and navigate in a collision-free manner.
在我的實驗室,我們製造 自動式飛行測量機器人, 比如現在正在飛的這個。 和那些可以在市場上 買到的飛行器不同的是, 這個飛行器並沒有搭載全球定位系統(GPS)。 沒有定位系統, 飛行器很難確定它們自己所在的位置。 這個飛行器自帶傳感器、 攝像機和激光掃描儀, 用於掃描檢測周圍的環境。 飛行器從周圍環境中探測物體, 用三角測量法, 確定自己和周圍物體的相對位置和距離。 然後將這些物體整合到一張地圖中, 像我身後展示的這樣。 這個地圖可以使飛行器了解 那些外界物體的位置, 從而在導航的時候避免碰撞。
What I want to show you next is a set of experiments we did inside our laboratory, where this robot was able to go for longer distances. So here you'll see, on the top right, what the robot sees with the camera. And on the main screen -- and of course this is sped up by a factor of four -- on the main screen you'll see the map that it's building. So this is a high-resolution map of the corridor around our laboratory. And in a minute you'll see it enter our lab, which is recognizable by the clutter that you see.
接下來我想展示給你們的 是我們在實驗室中做的一系列實驗, 在實驗室中飛行器可以進行長距離飛行。 你們看,在右上方的位置,是飛行器 在攝像機中“看”到的場景, 以及在主屏幕中的樣子, 當然這個視頻是以4倍速度在播放, 在主屏幕上你可以看到地圖的搭建進程。 這是一個高分辨率的實驗室周圍走廊的地圖。 馬上你們將看到飛行器飛入我們的實驗室, 一看這麼亂就知道是我們實驗室。
(Laughter)
(笑聲)
But the main point I want to convey to you is that these robots are capable of building high-resolution maps at five centimeters resolution, allowing somebody who is outside the lab, or outside the building to deploy these without actually going inside, and trying to infer what happens inside the building.
我想說的重點是, 這些飛行器可以創建 高達5釐米*5釐米(每像素)的 高分辨率地圖, 可以使那些在實驗室外或者建築物外的人, 不用進入內部就可以進行部署, 並嘗試推斷建築物內部的事態情況。
Now there's one problem with robots like this. The first problem is it's pretty big. Because it's big, it's heavy. And these robots consume about 100 watts per pound. And this makes for a very short mission life. The second problem is that these robots have onboard sensors that end up being very expensive -- a laser scanner, a camera and the processors. That drives up the cost of this robot.
這些飛行器有一些主要的缺點。 第一,它體積很大。 因為它體積較大,所以比較重。 這些飛行器消耗大概每磅消耗100瓦能量, 這就使得它們只能執行短時間任務。 第二個缺點, 這些飛行器自帶的傳感器很昂貴: 包括一個雷射光光掃描儀、一個攝像機, 還有多個處理器, 這些都使得飛行器的製造成本上升。
So we asked ourselves a question: what consumer product can you buy in an electronics store that is inexpensive, that's lightweight, that has sensing onboard and computation? And we invented the flying phone.
於是我們就問自己: 什麼樣的產品是可以在電子商店裡買到的? 那種價格不貴的、輕便, 並且自帶傳感器和處理器的...... 於是我們發明了“飛行手機”。
(Laughter)
(笑聲)
So this robot uses a Samsung Galaxy smartphone that you can buy off the shelf, and all you need is an app that you can download from our app store. And you can see this robot reading the letters, "TED" in this case, looking at the corners of the "T" and the "E" and then triangulating off of that, flying autonomously. That joystick is just there to make sure if the robot goes crazy, Giuseppe can kill it.
這個飛行器搭載了一個 常見的三星智慧型手機, 另外你只需要一個手機應用, 可以在我們的應用商店下載。 你們可以看到這個飛行器 正在讀TED這幾個字母, 它主要關注T和E這些字母的邊角, 然後應用三角測量法自主飛行。 那個手柄只是放在那,避免飛行器突然失控, Giuseppe同學就能廢了它。
(Laughter)
(笑聲)
In addition to building these small robots, we also experiment with aggressive behaviors, like you see here. So this robot is now traveling at two to three meters per second, pitching and rolling aggressively as it changes direction. The main point is we can have smaller robots that can go faster and then travel in these very unstructured environments.
除了製作這些小飛行器之外, 我們還實驗了一些比較激烈的動作。 這個飛行器的當前速度是2到3米每秒, 較為激烈地起落和翻滾。 重點是,我們可以製作更小的飛行器, 它們飛得更快, 它們可以在不規則的環境中飛行。
And in this next video, just like you see this bird, an eagle, gracefully coordinating its wings, its eyes and feet to grab prey out of the water, our robot can go fishing, too.
在下一個視頻中, 就像你看到的這隻鷹很優雅地扇動翅膀, 和眼睛、爪子一起協調,從水中抓魚, 我們的飛行器也能這麼抓魚。
(Laughter)
(笑聲)
In this case, this is a Philly cheesesteak hoagie that it's grabbing out of thin air.
在這裡,它是從空氣中抓到一個 菲力芝士牛排三明治。
(Laughter)
(笑聲)
So you can see this robot going at about three meters per second, which is faster than walking speed, coordinating its arms, its claws and its flight with split-second timing to achieve this maneuver. In another experiment, I want to show you how the robot adapts its flight to control its suspended payload, whose length is actually larger than the width of the window. So in order to accomplish this, it actually has to pitch and adjust the altitude and swing the payload through. But of course we want to make these even smaller, and we're inspired in particular by honeybees. So if you look at honeybees, and this is a slowed down video, they're so small, the inertia is so lightweight --
這個飛行器大概每秒飛行3公尺, 比我們步行速度更快, 它在瞬間協調了機械臂、機械爪 和飛行動作來完成整個流程。 在另一個實驗中, 我將展示給各位這個飛行器如何自適應飛行, 來控制它的懸掛負載。 這個負載的長度比飛行器 要穿過的窗戶還要大。 為了做到這一點, 它必須利用起落來控制高度, 並且通過擺動將負載擺過窗子。 當然,我們想把這些飛行器變得更小。 我們受到了蜜蜂的啟發。 如果你仔細觀察蜜蜂,在這個慢速的視頻中, 它們體積很小,慣性也很小,
(Laughter)
(笑聲)
that they don't care -- they bounce off my hand, for example. This is a little robot that mimics the honeybee behavior. And smaller is better, because along with the small size you get lower inertia. Along with lower inertia --
所以它們并不在意 - 比如說從我手上彈開。 這是一個模仿蜜蜂動作的小型飛行器。 越小越好。 因為更小的體積會有更小的慣性。 更小的慣性 --
(Robot buzzing, laughter)
(飛行器蜂鳴,笑聲)
along with lower inertia, you're resistant to collisions. And that makes you more robust. So just like these honeybees, we build small robots. And this particular one is only 25 grams in weight. It consumes only six watts of power. And it can travel up to six meters per second. So if I normalize that to its size, it's like a Boeing 787 traveling ten times the speed of sound.
更小的慣性,就更加不怕碰撞, 這使飛行器更加強壯。 所以就像這些蜜蜂一樣, 我們製造了小型飛行器。 你們看到的這個只有25公克重, 它只消耗6瓦能量。 它可以每秒飛行6公尺。 所以如果我把它的尺寸標準化, 它就像是一架以10倍 音速飛行的波音787飛機。
(Laughter)
(笑聲)
And I want to show you an example. This is probably the first planned mid-air collision, at one-twentieth normal speed. These are going at a relative speed of two meters per second, and this illustrates the basic principle. The two-gram carbon fiber cage around it prevents the propellers from entangling, but essentially the collision is absorbed and the robot responds to the collisions. And so small also means safe. In my lab, as we developed these robots, we start off with these big robots and then now we're down to these small robots. And if you plot a histogram of the number of Band-Aids we've ordered in the past, that sort of tailed off now. Because these robots are really safe.
我還想給你們看一個例子。 這可能是第一個計劃性的控制碰撞, 降速20倍播放。 兩個飛行器以相對速度每秒2米飛行, 這闡釋了基本原理。 圍住飛行器的是碳纖維架,重2公克。 用於防止兩架飛行器的螺旋槳卷到一起。 最終碰撞力量被吸收了, 飛行器也對碰撞有良好反應。 所以小尺寸意味著高安全性。 在我的實驗室中, 就像我們研發這些飛行器一樣, 我們從研發大型飛行器開始。 到現在我們研發這些小飛行器。 如果用一個柱狀圖來記錄 我們買過的OK繃的數量, 現在我們幾乎不需要買了。 因為這些飛行器非常安全。
The small size has some disadvantages, and nature has found a number of ways to compensate for these disadvantages. The basic idea is they aggregate to form large groups, or swarms. So, similarly, in our lab, we try to create artificial robot swarms. And this is quite challenging because now you have to think about networks of robots. And within each robot, you have to think about the interplay of sensing, communication, computation -- and this network then becomes quite difficult to control and manage. So from nature we take away three organizing principles that essentially allow us to develop our algorithms. The first idea is that robots need to be aware of their neighbors. They need to be able to sense and communicate with their neighbors.
小體積也有一些缺陷, 但是自然界找到了很多方法來彌補這些缺點。 最基本的概念是牠們 集合到一起組成大型的群族。 所以同樣的在我們實驗室, 我們嘗試創建機器人(飛行器)群族。 這並不容易, 因為我們需要考慮飛行器的網絡操作。 在每一個飛行器中, 我們需要顧及到傳感、通訊 和計算的相互作用 -- 這個網絡結構非常難以控制和管理。 所以我們從大自然中學習到三個組織原則, 最終幫助我們完成了算法。 第一個是,飛行器需要關注 附近的其他飛行器。 它們要能夠探測到附近的 其他飛行器並與其進行通訊。
So this video illustrates the basic idea. You have four robots -- one of the robots has actually been hijacked by a human operator, literally. But because the robots interact with each other, they sense their neighbors, they essentially follow. And here there's a single person able to lead this network of followers. So again, it's not because all the robots know where they're supposed to go. It's because they're just reacting to the positions of their neighbors.
這個視頻闡述了基本概念。 有四台飛行器, 其中一台被實驗員握在手中。 但是因為每個飛行器都與其他飛行器有交互, 它們能探測到彼此, 就會跟隨實驗員手中的飛行器。 所以這個實驗員能夠主導這一群飛行器。 這並不是因為這些飛行器 知道它們該飛往哪裡。 這隻是因為它們在對 彼此的位置進行交互和反饋。
(Laughter)
(笑聲)
So the next experiment illustrates the second organizing principle. And this principle has to do with the principle of anonymity. Here the key idea is that the robots are agnostic to the identities of their neighbors. They're asked to form a circular shape, and no matter how many robots you introduce into the formation, or how many robots you pull out, each robot is simply reacting to its neighbor. It's aware of the fact that it needs to form the circular shape, but collaborating with its neighbors it forms the shape without central coordination. Now if you put these ideas together, the third idea is that we essentially give these robots mathematical descriptions of the shape they need to execute. And these shapes can be varying as a function of time, and you'll see these robots start from a circular formation, change into a rectangular formation, stretch into a straight line, back into an ellipse. And they do this with the same kind of split-second coordination that you see in natural swarms, in nature.
下一個實驗闡述了第二個自然組織法則。 這個原則與匿名原則有關。 重點在於, 每個飛行器並不知道 附近其他飛行器的身份。 這些飛行器被要求組成一個圓形陣列, 不管你增加多少飛行器進去, 或者拿開多少飛行器, 每個飛行器都只是在跟附近的飛行器交互。 它們知道它們需要組成圓陣, 與其他飛行器進行配合, 這個過程並不需要中樞協調。 現在如果我們綜合來看, 第三個原則是我們最終應用到飛行器上的, 用數學來描述它們需要組成的陣列形狀。 這些形狀可以根據一個時間函數進行變化, 我們會看到這些飛行器從一個圓形陣列開始, 接著變為長方形陣,然後是一條直線, 最後變回橢圓形陣。 它們同樣是瞬間協調來完成這些動作, 就像自然界的群族一樣。
So why work with swarms? Let me tell you about two applications that we are very interested in. The first one has to do with agriculture, which is probably the biggest problem that we're facing worldwide. As you well know, one in every seven persons in this earth is malnourished. Most of the land that we can cultivate has already been cultivated. And the efficiency of most systems in the world is improving, but our production system efficiency is actually declining. And that's mostly because of water shortage, crop diseases, climate change and a couple of other things.
那麼為什麼要模仿群族呢? 我們對兩方面的應用很感興趣。 首先是農業, 這可能是世界範圍內最嚴峻的問題。 你們都知道, 世界上每七個人中就有一個人營養不良。 絕大部分可用耕地都已經被開墾耕種。 世界上大多數系統的效率都在提升, 但是我們生產系統的效率卻在降低。 這主要是由於水資源缺乏、 農作物疾病和氣候變化, 和其他的一些原因。
So what can robots do? Well, we adopt an approach that's called Precision Farming in the community. And the basic idea is that we fly aerial robots through orchards, and then we build precision models of individual plants. So just like personalized medicine, while you might imagine wanting to treat every patient individually, what we'd like to do is build models of individual plants and then tell the farmer what kind of inputs every plant needs -- the inputs in this case being water, fertilizer and pesticide. Here you'll see robots traveling through an apple orchard, and in a minute you'll see two of its companions doing the same thing on the left side. And what they're doing is essentially building a map of the orchard. Within the map is a map of every plant in this orchard.
那麼機器人能做些什麼呢? 我們在社區中採用了一種 叫做“精確種植”的方法。 基本思路是用機器人飛行器在果園中飛行, 為每一棵農作物搭建精確的模型。 就像個體化用藥一樣, 對每個病人要對症下藥, 我們所做的是給每一棵農作物創建模型, 並告訴農民每一棵農作物分別需要什麼: 可能是水、也可能是化肥或者殺蟲劑。 你們可以看到飛行器正在蘋果園中穿行, 馬上你們還會看到另外兩台飛行器, 正在左邊做同樣的工作。 它們正在創建果園的地圖 每一棵果樹都將呈現在地圖上。
(Robot buzzing)
(飛行器蜂鳴)
Let's see what those maps look like. In the next video, you'll see the cameras that are being used on this robot. On the top-left is essentially a standard color camera. On the left-center is an infrared camera. And on the bottom-left is a thermal camera. And on the main panel, you're seeing a three-dimensional reconstruction of every tree in the orchard as the sensors fly right past the trees. Armed with information like this, we can do several things. The first and possibly the most important thing we can do is very simple: count the number of fruits on every tree. By doing this, you tell the farmer how many fruits she has in every tree and allow her to estimate the yield in the orchard, optimizing the production chain downstream.
我們來看看這些地圖吧。 下一個視頻中你們將看到搭載在飛行器上的攝像機。 左上方是一個高級彩色攝像機 中間的是一個紅外線攝像機。 左下方是一個熱感攝像機。 當飛行器在飛過果樹時由傳感器採集數據, 在主面板上你將看到每一棵果樹的三維重建。 有了這些數據,我們可以做一些研究。 我們能做的第一件事 也是最重要的事很簡單: 計算每一棵果樹上果實的數量。 有了這個,我們可以告訴農民 她的每一棵果樹上都有多少果子, 由她來估算整個果園的產量, 從而優化下游產業鏈。
The second thing we can do is take models of plants, construct three-dimensional reconstructions, and from that estimate the canopy size, and then correlate the canopy size to the amount of leaf area on every plant. And this is called the leaf area index. So if you know this leaf area index, you essentially have a measure of how much photosynthesis is possible in every plant, which again tells you how healthy each plant is. By combining visual and infrared information, we can also compute indices such as NDVI. And in this particular case, you can essentially see there are some crops that are not doing as well as other crops. This is easily discernible from imagery, not just visual imagery but combining both visual imagery and infrared imagery.
我們能做的第二件事, 是對果樹建模,進行三維重建, 從而估算樹冠的大小, 將每一棵果樹的樹冠尺寸和樹葉面積做關聯。 這被稱作是“葉面積指數”。 如果我們知道一棵樹的葉面積指數, 就能大概知道這棵果樹在進行多少光合作用, 於是就能得知這棵果樹是否健康。 將視覺信息和紅外信息整合起來, 我們就能計算一些指數, 例如歸一化植被指數(NDVI)。 在我們的這個案例中,能夠看到, 有一些作物並不像其他作物那樣健康。 這在圖像中是很容易辨別的。 不是在視覺圖像中,而是在 視覺和紅外線合成的圖像中。
And then lastly, one thing we're interested in doing is detecting the early onset of chlorosis -- and this is an orange tree -- which is essentially seen by yellowing of leaves. But robots flying overhead can easily spot this autonomously and then report to the farmer that he or she has a problem in this section of the orchard.
最後, 我們很想做的一件事是, 檢測早期的樹葉萎黃病。 這是一顆橘子樹, 可以看到泛黃的樹葉。 飛行器在上空飛行的時候可以 輕易地自動檢測到這一現象, 然後向農民報告,在果園中的這一區域 出現了問題。
Systems like this can really help, and we're projecting yields that can improve by about ten percent and, more importantly, decrease the amount of inputs such as water by 25 percent by using aerial robot swarms.
這個系統對農業生產非常有用, 我們估計農業產量將會上升10%, 更重要的是,通過使用飛行器群族, 降低25%的水資源用量。
Lastly, I want you to applaud the people who actually create the future, Yash Mulgaonkar, Sikang Liu and Giuseppe Loianno, who are responsible for the three demonstrations that you saw.
最後,我希望大家向創造了 未來的人們致以掌聲, Yash Mulgaonkar, Sikang Liu 和 Giuseppe Loianno, 他們負責你們看到的三次演示。
Thank you.
謝謝!
(Applause)
(掌聲)