My passions are music, technology and making things. And it's the combination of these things that has led me to the hobby of sound visualization, and, on occasion, has led me to play with fire.
我的熱情 是音樂,科技,以及創造東西。 而且就是這些東西的組合 促使我對聲音的圖像化產生興趣, 而且,有時,促使我去玩火.
This is a Rubens' tube. It's one of many I've made over the years, and I have one here tonight. It's about an 8-foot-long tube of metal, it's got a hundred or so holes on top, on that side is the speaker, and here is some lab tubing, and it's connected to this tank of propane. So, let's fire it up and see what it does. So let's play a 550-herz frequency and watch what happens.
這是魯本斯管.近幾些年來我做的其中一個, 今天晚上呢我也帶來了一個. 這東西是一支大約8英尺長的金屬管, 管上差不多有一百多個小孔, 那邊有個喇叭,而這邊呢 有一些實驗用的管子,而且是和這個裝滿丙烷 的容器連接的. 那麼,我們現在就把它點燃看看會發生什麼事. 嗯...我們先試試550赫茲的頻率吧 看看會有什麼效果.
(Frequency)
(頻率)
Thank you. (Applause) It's okay to applaud the laws of physics, but essentially what's happening here -- (Laughter) -- is the energy from the sound via the air and gas molecules is influencing the combustion properties of propane, creating a visible waveform, and we can see the alternating regions of compression and rarefaction that we call frequency, and the height is showing us amplitude. So let's change the frequency of the sound, and watch what happens to the fire.
謝謝.(觀衆鼓掌) 確實值得讚揚物理定律的美, 但是實際發生的是 (笑聲) 聲音的能量透過空氣和氣體分子 來影響丙烷的燃燒性質, 因而產生一個可見的波形, 而且我們能看到這些一下子壓縮 又接著膨脹的變化區段,我們把它叫頻率, 而這個高度告訴我們的是振幅. 好吧.那讓我們改變一下聲音的頻率, 然後再看看火會發生什麽變化
(Higher frequency)
(更高的頻率)
So every time we hit a resonant frequency we get a standing wave and that emergent sine curve of fire. So let's turn that off. We're indoors. Thank you. (Applause)
所以每當我們調到共鳴頻率時,會有駐波產生 而那個像正弦函數的火焰就會出現. 好吧我們還是把這關了.我們在室內呢. 謝謝. (觀衆鼓掌)
I also have with me a flame table. It's very similar to a Rubens' tube, and it's also used for visualizing the physical properties of sound, such as eigenmodes, so let's fire it up and see what it does.
我今天還帶來了一個火焰桌. 它跟魯本斯管挺像的,而且它還能用來 將聲音的物理性質圖像化, 比如說特徵模式,那我們把它點燃吧 然後看看會發生什麽.
Ooh. (Laughter) Okay. Now, while the table comes up to pressure, let me note here that the sound is not traveling in perfect lines. It's actually traveling in all directions, and the Rubens' tube's a little like bisecting those waves with a line, and the flame table's a little like bisecting those waves with a plane, and it can show a little more subtle complexity, which is why I like to use it to watch Geoff Farina play guitar.
喔~(笑聲) 好吧.現在,當這個桌子受到到壓力, 此時我注意到聲音(波形)不是以完美的直線在傳遞 實際上是在各個方向傳遞, 而魯本斯管有點像是用一條線將這些波形分兩半 而這個火焰桌有點像 用一個面將這些波分兩半, 而它可以呈現一些微妙的複雜性,這就是爲啥 我喜歡用它去看Geoff Farina彈吉他.
(Music)
(音樂)
All right, so it's a delicate dance. If you watch closely — (Applause) If you watch closely, you may have seen some of the eigenmodes, but also you may have seen that jazz music is better with fire. Actually, a lot of things are better with fire in my world, but the fire's just a foundation. It shows very well that eyes can hear, and this is interesting to me because technology allows us to present sound to the eyes in ways that accentuate the strength of the eyes for seeing sound, such as the removal of time.
不錯,這是個曼妙的舞蹈. 如果你看的近一點--(觀衆鼓掌). 如果你仔細看,你可以發現 一些特徵模式,不過你還可以發現 爵士樂在火焰的陪襯下就感覺不錯. 實際上,我的世界裡,好多東西在火焰的陪襯下就不錯, 不過火焰只不過是個陪襯. 可以將耳朵能聽見的顯示得很清楚, 我對這個很感興趣 因爲科技允許我們去用眼睛來感受聲音 從而加強我們眼睛接受聲音的敏銳度 比如時間的轉移.
So here, I'm using a rendering algorithm to paint the frequencies of the song "Smells Like Teen Spirit" in a way that the eyes can take them in as a single visual impression, and the technique will also show the strengths of the visual cortex for pattern recognition. So if I show you another song off this album, and another, your eyes will easily pick out the use of repetition by the band Nirvana, and in the frequency distribution, the colors, you can see the clean-dirty-clean sound that they are famous for, and here is the entire album as a single visual impression, and I think this impression is pretty powerful.
那麽這裏呢,我用透視演算法去畫出 歌曲"Smell Like Teen Spirit"的頻率 我們的眼睛因而可以將它們讀入 成爲一種獨一無二的直覺印象,這技術 也能印證視覺皮質對圖案辨認的能力 也能印證視覺皮質對圖案辨認的能力 如果我給你這首專輯裏面另外一首歌, 再另外一首,你的眼睛會輕易地捕捉到 樂隊Nirvana(在音樂中)所使用的重複性, 以及在頻率分布,顔色方面, 你可以發現這個一會兒清晰一會兒吵雜的聲音 那種讓他們成名的音樂特色, 這是個以獨特的直覺印象呈現的整張專輯, 我認爲這種印象十分的強大.
At least, it's powerful enough that if I show you these four songs, and I remind you that this is "Smells Like Teen Spirit," you can probably correctly guess, without listening to any music at all, that the song a die hard Nirvana fan would enjoy is this song, "I'll Stick Around" by the Foo Fighters, whose lead singer is Dave Grohl, who was the drummer in Nirvana. The songs are a little similar, but mostly I'm just interested in the idea that someday maybe we'll buy a song because we like the way it looks.
至少,強大到 如果我給你看四首歌, 而且我提醒你這首是"Smells Like Teen Spirit", 在不聽音樂的情況下,你可能會猜到這首歌, 在不聽音樂的情況下,你可能會猜到這首歌, Nirvana的死忠歌迷所喜愛的這首歌, 由Foo Fighters所製作的“I’will Stick Around”, 他們的主唱是Dave Grohl, 他是Nirvana的鼓手. 這些歌曲都有點相似, 不過主要是 我覺得有一想法蠻有趣的,就是有一天我們買這首歌 是因爲我們喜歡它顯現的方式
All right, now for some more sound data. This is data from a skate park, and this is Mabel Davis skate park in Austin, Texas. (Skateboard sounds) And the sounds you're hearing came from eight microphones attached to obstacles around the park, and it sounds like chaos, but actually all the tricks start with a very distinct slap, but successful tricks end with a pop, whereas unsuccessful tricks more of a scratch and a tumble, and tricks on the rail will ring out like a gong, and voices occupy very unique frequencies in the skate park.
好吧, 再多呈現一點聲音數據, 這是從滑冰公園所擷取的數據, 而這是Mabel Davis滑冰公園 在德州的奧斯丁.(滑板的響聲) 你聽到的聲音是從八個 在公園附近障礙上的麥克風擷取來的, 聽起來有點混亂,不過實際上 所有的特技從一個明顯拍擊聲開始, 不過成功的特技最後會砰一聲結束, 而沒表演好的特技 還會摻雜更多的刮傷和翻滾的聲音, 而在欄杆上的特技會發出一個像是敲鑼的聲音, 而這些聲音在這個公園都有著獨特的頻率,
So if we were to render these sounds visually, we might end up with something like this. This is all 40 minutes of the recording, and right away the algorithm tells us a lot more tricks are missed than are made, and also a trick on the rails is a lot more likely to produce a cheer, and if you look really closely, we can tease out traffic patterns. You see the skaters often trick in this direction. The obstacles are easier.
所以,我們要是想把這些聲音圖像化, 我們應該要得到像這樣的東西. 這是40分鐘的記錄, 從這演算法我們立即發現 沒做成功的特技比成功的特技還要多, 而且在這些欄杆上的特技很有可能會 帶來更多的歡呼聲,如果你非常仔細看的話, 我們可以找出關於(滑冰公園內)流量的圖案. 你會看到溜冰的人通常朝這個方向,(這方向的)障礙比較少.
And in the middle of the recording, the mics pick this up, but later in the recording, this kid shows up, and he starts using a line at the top of the park to do some very advanced tricks on something called the tall rail. And it's fascinating. At this moment in time, all the rest of the skaters turn their lines 90 degrees to stay out of his way. You see, there's a subtle etiquette in the skate park, and it's led by key influencers, and they tend to be the kids who can do the best tricks, or wear red pants, and on this day the mics picked that up.
然後在這個記錄的中央, 麥克風把這個記錄下來, 但是在這記錄的後面,這個小孩出現, 他在公園最高處沿著一條新的路線開始溜冰 做一些難度高的特技 稱作高欄特技. 而這相當有看頭.在這個時候, 其餘所有溜冰的人都把他們的路線轉了90度讓路給他. 其餘所有溜冰的人都把他們的路線轉了90度讓路給他. 你可以看到,在這個公園裡有個約定俗成的禮儀 是由頗具影響力的人主導的, 而他們傾向是作高難度動作的小孩, 或者是穿紅褲子的,而這天麥克風就記錄下這現象.
All right, from skate physics to theoretical physics. I'm a big fan of Stephen Hawking, and I wanted to use all eight hours of his Cambridge lecture series to create an homage. Now, in this series he's speaking with the aid of a computer, which actually makes identifying the ends of sentences fairly easy. So I wrote a steering algorithm. It listens to the lecture, and then it uses the amplitude of each word to move a point on the x-axis, and it uses the inflection of sentences to move a same point up and down on the y-axis.
好吧,從溜冰物理到理論物理. 我是Stephen Hawking的大粉絲, 我想用他在劍橋八個小時的講座來表示我對他的敬意 我想用他在劍橋八個小時的講座來表示我對他的敬意. 現在呢,在這堂講座裡,他借助於電腦作演講, 這樣確實比較容易分辨句子是否結束 因此我寫了個操縱算法. 它一邊聽講座,然後一邊利用 每個詞的振幅在x軸方向上來回移動點, 它還利用句子的音調變化 針對同一點在Y軸方向上下移動.
And these trend lines, you can see, there's more questions than answers in the laws of physics, and when we reach the end of a sentence, we place a star at that location. So there's a lot of sentences, so a lot of stars, and after rendering all of the audio, this is what we get. This is Stephen Hawking's universe.
然後這些指明趨勢的線,你可以看到在物理世界中 未知比已知多, 每當我們聽完一個句子, 我們就在(那個句子對應到的)那個地方放一顆星星. 所以呢很多句子就產生很多星星, 播完所有的語音檔後,我們就得到了這個 這就是Stephen Hawking的宇宙.
(Applause)
(觀眾鼓掌)
It's all eight hours of the Cambridge lecture series taken in as a single visual impression, and I really like this image, but a lot of people think it's fake. So I made a more interactive version, and the way I did that is I used their position in time in the lecture to place these stars into 3D space, and with some custom software and a Kinect, I can walk right into the lecture. I'm going to wave through the Kinect here and take control, and now I'm going to reach out and I'm going to touch a star, and when I do, it will play the sentence that generated that star.
全部是在劍橋的八個小時的講座 以一個獨一無二的直覺印象呈現, 我相當喜歡這個圖案. 不過很多人認為它是假的. 所以我做了個互動版本, 而且我所使用的方式是利用他們在時間軸上出現的位置 將這些星星放置在3D空間裡(對應位置), 同時利用一些客製化軟體和一個Kinect, 這樣我就能”走進”課堂中. 我要握著這個Kinect 作揮手動作 然後作一些操控,接著我伸手去 抓一顆星星,當我這樣做的時候, 它會播出(這個星星所對應到的)這句話. 它會播出(這個星星所對應到的)這句話.
Stephen Hawking: There is one, and only one, arrangement in which the pieces make a complete picture.
Stephen Hawking:這裡有一個,獨一無二的(星星位置)分布 (星星所在的)各個位置構成一個完整的圖案.
Jared Ficklin: Thank you. (Applause) There are 1,400 stars. It's a really fun way to explore the lecture, and, I hope, a fitting homage.
Jared Ficklin:謝謝 (觀眾鼓掌) 這裡面一共有1400顆星星 這是個很有意思的方式去探索這個講座, 我希望(這是表示)合適敬意的方式.
All right. Let me close with a work in progress. I think, after 30 years, the opportunity exists to create an enhanced version of closed captioning. Now, we've all seen a lot of TEDTalks online, so let's watch one now with the sound turned off and the closed captioning turned on.
好的.讓我們以接下來的作品來結束今天的演講. 我認為,30年後,我們有可能 去創造沒字幕的特殊版本. 現在,我們在網上可以看到好多TED演講, 因此呢讓我們將聲音關掉去看一個吧 當然我們把字幕打開
There's no closed captioning for the TED theme song, and we're missing it, but if you've watched enough of these, you hear it in your mind's ear, and then applause starts. It usually begins here, and it grows and then it falls. Sometimes you get a little star applause, and then I think even Bill Gates takes a nervous breath, and the talk begins.
TED片頭曲是沒有字幕, 因此我們經常錯過這部分,不過你已經看得夠多了, 你可以在你心裡聽到這片頭曲, 接著呢開始聽到掌聲. 通常(掌聲)是這樣開始,先漸強,最後漸弱. 但有時候你僅會有一點的掌聲, 而且我認為即使Bill Gates也緊張地深吸一口氣, 然後開始演講.
All right, so let's watch this clip again. This time, I'm not going to talk at all. There's still going to be no audio, but what I am going to do is I'm going to render the sound visually in real time at the bottom of the screen. So watch closely and see what your eyes can hear.
好吧,那麼就讓我們再看一次這個片段. 這次,我不準備說話了. 依然是沒有聲音, 不過這回我要做的就是讓這個聲音圖像化, 並顯示在銀幕的最底下 不過這回我要做的就是讓這個聲音圖像化, 並顯示在銀幕的最底下 所以呢 更靠近一點看,看看你的眼睛能”聽”到什麼.
This is fairly amazing to me. Even on the first view, your eyes will successfully pick out patterns, but on repeated views, your brain actually gets better at turning these patterns into information. You can get the tone and the timbre and the pace of the speech, things that you can't get out of closed captioning. That famous scene in horror movies where someone is walking up from behind is something you can see, and I believe this information would be something that is useful at times when the audio is turned off or not heard at all, and I speculate that deaf audiences might actually even be better at seeing sound than hearing audiences. I don't know. It's a theory right now. Actually, it's all just an idea.
這個對我來說非常神奇 即使看第一次,你的眼睛也會成功的 擷取一些圖案,不過在重複看之後, 你的腦子將圖案轉化為信息能力方面會變得更好 你的腦子將圖案轉化為信息能力方面會變得更好 你可以感覺到說話人的語氣和音色 以及演講的速度, 這些是你從字幕中得不到的東西. 恐怖電影中的經典畫面 也就是當有人從後面走來時 變成了你可以看到的東西, 因此我相信這個東西會成為 當聲音被關掉的時候的一個有用的工具 或者根本無法聽到的時候,我還認為耳聾的觀眾 會比看正常的聽眾更擅長看聲音 會比看正常的聽眾更擅長看聲音 我也不太清楚.目前為這只是個理論. 實際上,這只不過是個想法.
And let me end by saying that sound moves in all directions, and so do ideas. Thank you. (Applause)
我結束前我說這樣一句話吧:聲音會向所有的方向移動, 想法也會。 謝謝 (觀眾鼓掌)