One of the biggest challenges in computer graphics has been being able to create a photo-real, digital human face. And one of the reasons it is so difficult is that, unlike aliens and dinosaurs, we look at human faces every day. They are very important to how we communicate with each other. As a result, we're tuned in to the subtlest things that could possibly be wrong with a computer rendering, in order to believe whether these things are realistic.
電腦繪圖的巨大挑戰之一, 是如何製作一個如相片般真實的 數位人物面孔。 不像外星人和恐龍,它之所以如此困難的原因是, 我們每天都會看到人的面孔, 它們對我們非常重要,因為我們藉此來彼此溝通, 因此,我們會注意到那些通過電腦渲染出來, 不正確的、非常細微的東西, 並用它們來分辨這些事物是否真實的。
And what I'm going to do in the next five minutes is take you through a process where we tried to create a reasonably photo-realistic computer-generated face, using some computer graphics technology we've developed, and also some collaborators at a company called Image Metrics. And we're going to try to do a photo-real face of an actress named Emily O'Brien, who is right there. And that's actually a completely computer-generated rendering of her face. By the end of the talk, we're going to see it move.
在接下來的五分鐘裡面, 我將帶領大家一起看一個過程, 從我們嘗試去創造一個以電腦制作的幾近相片般真實的面孔, 通過我們 和一些來自 Image Metrics 的伙伴開發的一系列電腦繪圖技術。 我們將嘗試製作 一位名叫 Emily O'Brien 的女演員的面孔,她就在那兒。 那實際上是一個完全依據她的面孔,由電腦渲染產生的結果。 在演講的最後,我們將會看到她動起來。
The way that we did this is we tried to start with Emily herself, who was gracious enough to come to our laboratory in Marina Del Rey, and sit for a session in Light Stage 5. This is a face-scanning sphere, with 156 white LEDs all around that allow us to photograph her in a series of very controlled illumination conditions. And the lighting that we use these days looks something like this. We shoot all of these photographs in about three seconds.
我們嘗試的方式是由 Emily 她本身開始, 她非常親切地來到我們的實驗室。 在瑪麗安德爾灣,並坐到了我們的設備「光台五號」(Light Stage 5) 當中。 這是一個臉部掃描球體,有 156 個白色的發光二極管, 能讓我們在一系列嚴格控制的照明條件下, 對她進行拍照。 我們當時使用的照明方式看起來是這樣的。 我們在三秒鐘之內拍攝出這裡全部的照片。
And we basically capture enough information with video projector patterns that drape over the contours of her face, and different principle directions of light from the light stage, to figure out both the coarse-scale and the fine-scale detail of her face. If we zoom in on this photograph right here, we can see it's a really nice photograph to have of her, because she is lit from absolutely everywhere at the same time to get a nice image of her facial texture.
我們基本上採集到了足夠的資訊, 能夠通過圖像投影模擬出她的面部輪廓, 以及在燈光舞台不同角度的照明條件下, 來計算出她面孔粗略的 以及精細的面部細節模型。 如果我們放大這張照片的這個地方, 我們可以很清晰地看到她臉部的細節, 因為她的臉部同時從各個角度被完全地照亮了, 因此得到一份非常優質的她的臉部紋理的相片。
And in addition, we've actually used polarizers on all the lights -- just like polarized sunglasses can block the glare off of the road, polarizers can block the shine off of the skin, so we don't get all those specular reflections to take this map. Now, if we turn the polarizers around just a little bit, we can actually bring that specular reflection of the skin back in, and you can see she looks kind of shiny and oily at this point. If you take the difference between these two images here, you can get an image lit from the entire sphere of light of just the shine off of Emily's skin.
再加上,我們實際上在所有燈光上應用了偏光鏡 就如同偏光的太陽眼鏡可以阻擋 馬路上的強烈反強光, 偏光鏡可以阻擋皮膚上出現的光澤, 所以在這張圖中,我們沒有採集到那些球面反射。 現在,如果我們稍微調整一下偏光鏡, 我們就可以將球面反射 帶回到皮膚上面, 並且,你會看到她的面部看起來有些閃耀和油光。 如果你取得這兩張圖像的差別, 你可以得到的一個被球形光線照亮的 Emily 的皮膚的相片。光照效果
I don't think any photograph like this had ever been taken before we had done this. And this is very important light to capture, because this is the light that reflects off the first surface of the skin. It doesn't get underneath the translucent layers of the skin and blur out. And, as a result, it's a very good cue to the detailed shape of the skin-pore structure and all of the fine wrinkles that all of us have, the things that actually make us look like real humans.
我認為沒有任何人拍過像這樣的照片, 我們是第一個這麼做的。 採集這種光線是非常重要的, 因為這是皮膚最表層所反射出來的光線, 它不會跑到皮膚的透明層下面去, 然後造成模糊不清。 因此,它是一個 用來獲得皮膚細緻毛孔結構 以及我們都有的皮膚全部的褶皺細節的很好線索, 實際上,是這些東西讓我們看起來像真實人類。
So, if we use information that comes off of this specular reflection, we can go from a traditional face scan that might have the gross contours of the face and the basic shape, and augment it with information that puts in all of that skin pore structure and fine wrinkles. And, even more importantly, since this is a photometric process that only takes three seconds to capture, we can shoot Emily in just part of an afternoon, in many different facial poses and facial expressions.
如果,我們使用來自高光反射的資訊, 我們可以從傳統的臉部掃描開始, 來得到臉部的粗略輪廓和基本形狀, 再加上大量 皮膚的毛孔結構和紋理特徵的資訊。 更重要的是, 因為這是基於照片建模的流程而我們只採集了三秒鐘的照片資料, 我們可以用一下午的時間 來拍愛蜜麗 (Emily), 拍攝不同的面部姿態和表情變化。
So, here you can see her moving her eyes around, moving her mouth around. And these we're actually going to use to create a photo-real digital character. If you take a look at these scans that we have of Emily, you can see that the human face does an enormous amount of amazing things as it goes into different facial expressions. You can see things. Not only the face shape changes, but all sorts of different skin buckling and skin wrinkling occurs. You can see that the skin pore structure changes enormously from stretched skin pores to the regular skin texture. You can see the furrows in the brow and how the microstructure changes there. You can see muscles pulling down at flesh to bring her eyebrows down. Her muscles bulging in her forehead when she winces like that.
這裡,你可以看到她轉動眼睛,嘴巴的開合。 這些,實際上都會被我們用來創造數位虛擬的寫實人物。 來看一下這些我們對愛蜜麗 (Emily) 進行的掃描, 你會看到我們人類臉部表情是千變萬化的, 在不同的臉部表情變換之中, 你會看到,不僅僅是臉部形態發生變化, 各種各樣的皮膚的弧線和皮膚上的褶皺也會出現。 你可以看到皮膚毛孔結構也有著巨大的變化, 從被拉伸的皮膚毛孔, 到普通的皮膚質地。 你可以看到前額眉骨那裡細微的變化, 你可以看到在她皺眉時候肌肉的收縮。 在她畏懼的時候,她前額的肌肉會這樣地膨脹。
In addition to this kind of high-resolution geometry, since it's all captured with cameras, we've got a great texture map to use for the face. And by looking at how the different color channels of the illumination, the red and the green and the blue, diffuse the light differently, we can come up with a way of shading the skin on the computer. Then, instead of looking like a plaster mannequin, it actually looks like it's made out of living human flesh.
對於這樣高分辨率的幾何體, 因為這些都通過照相機採集,我們有她面部表情足夠的貼圖。 通過顏色和亮度不同的頻道, 紅綠藍色彩頻道, 受光照不同的散射, 我們可以通過電腦總結出為皮膚添加陰影的方式。 然後,不同於看起來如同石膏像的假人, 她看起來如同人類一樣鮮活。
And this is what we used to give to the company Image Metrics to create a rigged, digital version of Emily. We're just seeing the coarse-scale geometry here. But they basically created a digital puppet of her, where you can pull on these various strings, and it actually moves her face in ways that are completely consistent with the scans that we took. And, in addition to the coarse-scale geometry, they also used all of that detail to create a set of what are called "displacement maps" that animate as well. These are the displacement maps here. And you can see those different wrinkles actually show up as she animates.
這就是我們 為 Image Metrics 公司 所製作的寫實的數位虛擬人物 Emily。 這裡,我們看一下粗糙的幾何模型, 但他們已經對她創建了一些基本的數據程式, 你可以拖曳這些不同的座標繩, 它實際上運動的方式就會依照 我們掃描出的數據那樣。 並且,對於這個基本模型, 他們也使用了全部的細節 來創建一系列的「置換貼圖」, 同樣可以對其進行動畫設置。 這裡是一些置換貼圖, 你可以看到不同的褶皺出現在她的動畫之中。
So the next process was then to animate her. We actually used one of her own performances to provide the source data. So, by analyzing this video with computer vision techniques, they were able to drive the facial rig with the computer-generated performance. So what you're going to see now, after this, is a completely photo-real digital face. We can turn the volume up a little bit if that's available.
所以,接下來的步驟是對她進行動畫設置, 我們實際上應用她自身的表演作為源數據。 通過計算機視覺技術來分析這些影片, 他們可以使用電腦合成的表演, 來驅動表情的變化。 所以,我們將看到,通過這一切處理, 是一個完全寫實的數位化臉部。 我們可以把聲音開大一點,如果可以的話。
Emily: Image Metrics is a markerless, performance-driven animation company. We specialize in high-quality facial animation for video games and films. Image Metrics is a markerless, performance-driven animation company. We specialize in high quality facial animation for video games and films.
Emily: Image Metrics 是一家追求高品質的動畫公司。 我們致力於為電動和電影 製作高品質的臉部表情動畫。 Image Metrics 是一家追求高品質動畫的公司。 我們致力於為電動和電影 製作高品質的臉部表情動畫。
Paul Debevec: So, if we break that down into layers, here's that diffuse component we saw in the first slide. Here is the specular component animating. You can see all the wrinkles happening there. And there is the underlying wireframe mesh. And that is Emily herself.
保羅•德貝維奇:因此,如果我們把它分解到各個層,這是散射部分, 我們在第一張投影片裡面看到的。 這是高光部分的動畫。 你們可以看到各處皮膚褶皺的出現。 這是原始的線框結構層。 那是 Emily 她自己。
Now, where are we going with this here? We've gone a little bit beyond Light Stage 5. This is Light Stage 6, and we're looking at taking this technology and applying it to whole human bodies. This is Bruce Lawmen, one of our researchers in the group, who graciously agreed to get captured running in the Light Stage. And let's take a look at a computer-generated version of Bruce, running in a new environment. And thank you very much. (Applause)
現在,我們接下來會做些什麼? 我們已經有了高於「光台五」的裝置設備。這是「光台六」。 我們期望將這一技術 應用到人的全身。 這是布魯斯•勞曼,我們研發團隊中的一位, 非常樂意配合我們,採集他在「光台」中跑步的數據。 接著讓我們看一下電腦合成的, 在一個全新的環境中跑步的,布魯斯的影像。 感謝大家。 (掌聲)