Thinking: Directions of Commonsense Reasoning Research

I don't think that the applications developed here at the media lab have fully taken the advantage of the commonsense reasoning capability of computers right now.

There are, in my point of view, two major styles of applications base on OMCS. One is more practical, down-to-earth, kind of work, and the other is, mostly by Hugo, a more creative style. The former is more similar to traditional computer science research projects that it is convinsingly practicable. The later, oppositely, shows the possiblities of solving new problems that are people generally don't believe to be solvable (But it only shows some "possible" ways toward the solution, instead of something readily useful)

I think there are no applications taking fully advantage of the power of OMCS, in the sense that that are they inspired from two schools of thoughts: the computing devices, and computational linguistics. I don't see any of them coming up deeply from sociological or psychological thinking, despite that commonsense is really about us humans, abour our lives.

So I'm deciding to gain more nutrition from ideas in the sociological/psychological domains, and stand from those point of views to see the whole commonsense computing thing. It's not gonna work for me to focus solely on natural language processing or other pure computer science stuffs, cuz those are simply tools, and do no good to coming up with ideas that have true impacts.

I think filmmaking is a good domain for me, actually. But it's not enough. To me it's only a part of the projects that should be going on. People have thought too much about devices for social interactions, and I don't think it's gonna bring much difference if I stick to those social devices as well. So right now it's probably good for me to check out other domains that is truely interesting, standing from a human's point of view.

The commonsense-based film-editing agent could certainly be something meaningless to users as well. What I'm doing right now is to find what people really want when they tell stories - maybe a computer program, maybe not - but it's gonna be cool even if I find that they don't. It's just a process of looking into humans, and looking into what supply that computers can give at the same time. Commonsenes reasoning is for building applications. And applications come from users' needs. Wouldn't it be wierd if we keep focusing on the question "what application can I build for commonsense reasoning technology?" instead of looking into humans?

延伸閱讀

腦力激盪工廠

昨天下午想了一個新idea
在電話上面顯示很多泡泡
泡泡裡面有字
建議你還有什麼什麼話題你可能會想提的
後來跟幾個人討論 也許整個設計會改掉
但是從隨意塗鴉到想出來的過程 還挺有意思

延伸閱讀

Building a Story's Threads based on Interesting Concepts

The Contour system connects the images or video footages using keywords: the more keywords shared between two images, the higher relation between them will be. So when the viewer sorf of "create" a new "thread" by assigning a couple of keywords, the images related to these keywords will first be selected, and those relating to these chosen images will be selected as well, based on the spreading activaion technique.

Based on this idea, what I'm thinking about is that, maybe threads in a story can be constructed based on keywords that are found by our new system automatically. Suppose that the system finds that a particular keyword, or more specifically, a particular concept, is shared by some shots in the collection. And, the system can determine whether it's an interesting concept worth treating as a "thread" according to some criteria based on ConceptNet. Threads can thus be created even if they don't express particular intentions such as conflict resolution. I think it would probably be the most basic way of building threads, since in documentary films one cannot guarantee there will always be those high-level, interesting plot like conflict resolution or negotiation.

What is a so-called interesting concept? Something I can think of right now is a particular concept that evokes, or at least involves in, different affect states in different granules or shots. I wonder if the reasoning algorithm behind Hugo's EmpathyBuddy's could be used as some detecting engine. It doesn't need to be perfect at recognizing the inherent complex emotions within the shots, but only suggests an idea what kind of role the shot might play, with respect to the overall flow of the story thread.

Or maybe something I should think about before determining the tool to use is, "What are the criteria that script writers use to design the storylines?"

延伸閱讀

Turning into a Filmmaker

今天下午 我背著DV走出Lab 準備到Sloan School
去拍去年MIT 50K大賽的前三名 還有時代基金會從台灣帶來的學生們
一起參與的創業計畫展示/討論會
(當然重點不是他們的創業計畫 而是Glorianna要我拍的故事 發生在生活中的各種大小事情)
下午陽光很好 我和我的影子一邊走 一邊有種奇怪的感覺
我很高興的要去跟大家哈拉 吃免費的點心 又有台灣來的認識的朋友可以見面
這麼開心的事 居然是我的功課?
一直以來功課好像都應該是坐在電腦前面愁眉苦臉完成的
了不起 是覺得螢幕裡頭的3D動畫很炫好了
但是不管怎樣 不會有這種到外面玩耍 沒有負擔的
(如果就電機系或資訊系而言)

我在想 所以我是來對地方了 The MIT Media Lab
在這裡可以同時拍片 同時彈吉他 同時跟小朋友玩耍 同時寫程式燒電路板寫論文
而且其實不只是可以 甚至是必須
我真的要變成一個拍片剪片的人 一天到晚想到底要怎麼拍才好 怎麼剪才好
才有辦法知道導演或剪接師真正需要的是什麼
才能寫出那種program 讓電腦知道怎樣在剪接的時候做個幫手
Glorianna是個真正的Filmmaker 也是computer scientist
很高興能夠受到大師的教導

延伸閱讀

MIT的第一堂課

雖然上禮拜三就開學了 但是我的課都是星期二的課
所以昨天 才算是第一天正式上課
早上六點半就起床 因為十一點跟Glorianna要見面
我怕什麼都沒準備就去找老師很不好
所以硬是把剩下一半的博士論文看完 在見面前五分鐘想出了一個粗糙的idea
帶去講給她聽

我們聊了半個鐘頭 就結束了我這禮拜的第一門課
Glorianna是個超nice的...怎麼講.. 老奶奶
我是因為知道她快要退休了 所以很急著想修到她的課
才不顧眾多新奇有趣的課 自己一個人跟她做independent study
她聽了我講的idea 問了我幾個問題
但因為是最後幾分鐘隨便亂想出來的東西
所以我幾乎不太能回答她的問題
她卻也很有耐心地 講話慢條斯理地
告訴我她的想法 然後給我這禮拜的功課
要我看一篇博士論文 一堆video
然後想清楚 我要拍什麼故事 然後仔細斟酌地去用這捲只有一個鐘頭長的DV帶
去拍

走出她的office時 突然還覺得挺exciting的
一方面是因為 真的開始了 傳說中的Media Lab
另外一方面可能也因為 我要開始拍東西了
如果是對於有興趣的東西 完全是新手 的那種感覺
其實還不賴

下午的課是Henry的"Commonsense Reasoning and Interactive Applications"
我們在三樓的 我之前一直完全不知道有這麼一個地方的 小會議室上課
我想起了Peggy在e貓掉進未來湯那本書裡面
畫出來的很可愛的圖 紀錄第一次上課的時候 教室裡哪個位子坐了哪個人
所以我也拿了筆 隨手把圍著大桌子的座位畫在本子上
Peggy當初只畫了六個人 結果這次我畫了十多個
我想到Francis說這門課有很多人修 可能十多個真的是所謂的多吧 我猜

然後 我也想到她在書裡寫的的第一次上課的緊張/期待/興奮
雖然因為Henry是我自己的老闆
在很多sponsors參觀的場合 我已經聽過他講的東西了 甚至也有他的投影片檔案
所以已經差不多知道這節課會幹嘛
但是還是覺得蠻新鮮的
Henry這門課是跟Hugo一起開的
Hugo雖然是PhD學生 但是在Media Lab待超過六年的他 某種程度上不太被當學生看待
我想他自己也完全覺得自己是同事 而不是學生吧
講話很快很快的Hugo 永遠有講不完的新理論
人長得不錯帥 又能一年發五六篇長篇的paper 能夠理解為什麼能有源源不絕的自信
我一邊聽著他接著Henry的話批哩啪拉講個不停
一邊感覺好像回到上次來interview的時候認識他的時候的感覺
覺得自己矮了一截 又想要打敗他的感覺
很希望自己也可以英文講得這麼快這麼快想都不用想 的感覺

延伸閱讀

Multilinear Storytelling

Adobe has provided the powerful video-editing tool, Premiere, which shifts video editing from linear to so-called "non-linear" processes. Video editors don't need to sequence the video shots linearly anymore. They can keep jumping from one point to another during the editing process, and reduce the time moving all those shots back-and-forth.

However, Premiere, as well as most video-editing tool to-date, facilitates people to solve only half of the problem of modern video editing. The story delivered through the video media is not edited in a non-linear, or more precisely, multi-linear, way with some particular tool. The stories are mostly created and represented linearly with text, which is a linear form itself. Only the profound directors or editors can organize the complicated multi-linear story structure and create aesthetically good video with such a tool.

In other words, to help people to edit the video content, it is now the time to help people to edit their stories.

"Multilinear Storytelling" means that, as in many of the Hollywood films nowdays and other storytelling forms, stories tend to be told in a multi-thread fashion. The story goes on as each of the intertwined lines in the story evolves, and as the corelations among these lines vary. I am trying to apply commonsense reasoning into such a tool that helps people to create and organize multi-linear stories (in the video form) more efficiently and enjoyably. I believe the project website is gonna lauch soon, and I think it's something interesting to look forward to.

延伸閱讀