Thursday, September 30, 2010

S. Chandrasekhar and his attempts to get away from Astrophysics

I went back to Nobel Laureate S. Chandrasekhar’s biography a few months back after 17 years. Chandra written by Kameshwar Wali was a birthday gift I gave to my wife a few months after she joined me in Buffalo, New York after our wedding. If you have given books as gifts to folks at home, you must be aware of the hidden agenda. I don’t remember who started reading the book first, but I was definitely the first one to finish it. It was the first biography I read where the hero of the story co-operated with the author in a frank manner and the author has done a splendid job with the narration. Why did I go back the book now?

I had a specific objective – to re-read the story of “Chandrasekhar Limit” – the idea that languished for several decades before gaining acceptance by the scientific community eventually leading to a Nobel. It is a masala story with lots of twists and turns. It even has a villain, Prof. Eddington then doyan of Astrophysics, who launched a campaign to kill Chandra’s idea. But that would be a topic of a separate article. In this second reading of the book one of things that struck me was Chandra’s two serious attempts in getting away from Astrophysics. Why did Chandra try to do that? And what happened in each attempt? Let’s see briefly in this article.

Chandra’s career in Astrophysics was more of an accident. By the time Chandra landed in Cambridge he had already published a paper based on the work of Fowler, a Fellow of the Royal Society. Chandra had encountered Fowler’s paper while browsing the newly arrived journals in university library in Madras. In fact, Fowler helped him secure admission at Cambridge. So it was natural for him to start his PhD work under Fowler in Astrophysics.

1930s were hey days of theoretical Physics as a totally new foundation known as quantum mechanics was being built. Chandra soon realized that working in Astrophysics was more like being on the periphery. The real action was in pure Physics. In the meantime, Fowler left Cambridge on sabbatical and Dirac who had nothing to do with Astrophysics became Chandra’s official guide. Chandra consulted Dirac who advised him to visit Niels Bohr and co in Copenhagen. Folks at Copenhagen indeed turned out to friendlier and it is here Chandra made his first serious attempt at pure physics. He wrote a paper titled “On the statistics of Similar Particles” and sent it to Dirac in Cambridge. Bohr had already given his nod. But pretty soon Dirac found an error in his paper. “My paper sent to the Proceedings of the Royal Society is WRONG. That is all” Chandra wrote to his father in Nov 1932. Over four months of work was down the drain. Pressure started building up to submit a thesis. Chandra went back to Astrophysics.

Chandra’s second attempt at quitting Astrophysics happened just after he finished PhD in 1933 and was elected to trinity fellowship. Could this be the time to shift to pure Physics or pure Mathematics? The dilemma ended soon when he got an advice from a senior Fellow Harold Davenport, “You can’t teach an old dog new tricks. Why don’t you continue in astrophysics? You got the fellowship on that account. There is enough time to do mathematics later.”

Fortunately for Astrophysics, Chandra not only stayed with it but made a significant contribution in bringing Astrophysics part of main stream physics. Today 10% of the 12th standard Physics syllabus my wife teaches is Astrophysics. Incidentally, we will be hearing a lot more of Chandra in the coming months as this year is Chandra’s birth centenary year. In fact, Kameshwar Wali will be giving a talk at a conference on Chandra at Indian Institute of Astrophysics, Koramangala, Bangalore in December. I hope to attend it.

Sunday, September 19, 2010

My most favorite YouTube video and the marvels & the flaws of intuition

Chances are high your most favorite YouTube video is very different from mine in its form and content. Mine is neither a funny clip nor a music video although I enjoy watching both kinds of clips. But here is what is perhaps common between your favorite video and mine. Like you, I have watched it a dozen times so far. But more importantly I have listened to the audio extract of the video several dozen times by now. I listened to it in the gym a couple of days back and I listened to it before going to bed last night. It is titled “Explorations of the mind” and it is an hour-long lecture given at University of California, Berkley by the father of behavioral economics and Nobel Laureate Prof. Daniel Kahneman of Princeton. Who is Kahneman? And what is this video all about? And why do I like it so much? Let me briefly describe in this article.

Kahneman was born in 1934 (same age as my dad) to Jewish parents of Lithuanian origin and spent his childhood in France where his parents had migrated in 1920s. He says in his bio at the Nobel Prize site, “I had grown up intellectually precocious and physically inept”. His school PE teacher felt to pass him means to stretch his extreme tolerance. Kahneman made up for what he lacked in his physique by writing essays in a notebook like, “What I write of what I think”. Kahneman became a psychologist and together with Emos Tversky formulated a theory known as Prospect Theory. This theory turned upside-down a long standing belief among economists that human decision making is a rational process and eventually earned him a Nobel in 2002.

This video is titled “Explorations of the mind: Intuition: the marvels and the flaws”. The video can be divided into three parts. Part 1 is where the speaker is introduced (0-4:20). In part 2 Kahneman sets the context and defines the question he is planning to address for the rest of the talk (4:20-20:10). Part 3 (20:10-55:05) is where he presents a simple and useful model that addresses the question.

What is the question Kahneman addresses in this talk? It is as follows – On the one hand we encounter marvels of intuition when we see world class chess players, basketball players, fire fighters consistently making accurate decisions in a “blink” of an eye (check out Malcolm Gladwell’s Blink for stories on this). On the other hand, we have “experts” like the Chief Economists of the International Monetary Fund (IMF) Olivier Blanchard who on September 2, 2008 said, “If the price of oil stabilizes, I believe we can weather the financial crisis at limited cost in terms of real activity”. Within two weeks some of the biggest financial companies in the world collapsed (check out Nassim Taleb’s Black Swan for stories of these kinds). Why is it that the human intuition works beautifully in certain areas like sports and doesn’t work in some other areas like long term forecasting?

Why do I find this question so interesting? I have a deep-rooted fascination for the learning process. And I believe this question is at the heart of this process. I encounter people all the time who have spent decades managing projects but have stopped learning a while ago. Surprisingly, they believe that with every passing year, they are learning more. But actually they are not. The talk sheds light on what happens when we learn or stop learning.

Wednesday, September 1, 2010

Vinay Deolalikars proof of P=NP and the power of "web-test"

There are several ways of making a million dollars. One way suggested by Warren Buffett is to start with a billion dollars and invest them in an airline company. Unfortunately, for most of us this is not a practical way. There is another way which surprisingly needs no investment. Well, it needs a pencil and a paper which you can borrow from your friend. And of course, it needs your time which you can steal from your employer like Einstein did at Swiss Patent Office. Or better still, like Srinivas Ramanujam, you can make your dreams work for you. But here is the catch – you should be able to solve one of the six open problems in mathematics called the “Millennium Problems” defined by the Clay Mathematics Institute. The original list contained seven problems but one was solved by Grigory Perelman of Moscow in 2002. Three weeks back, on August 8, an HP researcher Vinay Deolalikar put up a draft of a proof sketch for one of the six Millennium Problems called “P=NP” on the Internet. Will Vinay get his million dollars?

I am supposed to have studied ABC of complexity theory in my undergraduate and graduate schools. But I pretty much don’t remember anything except for what “P” and “NP” means. Class P contains problems like sorting and searching which have fast computer programs. The other class NP contains problems which don’t have fast programs yet but if someone gives us an answer, we can write a program that can check its correctness quickly. Nobody has been able to mathematically prove whether P is equal to NP or not equal to NP. The problem is around 40 years old and was proposed by Steven Cook in 1971.

For an abstract subject like complexity theory, Prof. Richard Lipton writes amazingly lucid blog titled Godel’s lost letter and P=NP. In the months of May, June and July Richard wrote 8 blogs every month and averaged 17, 20 and 33 comments respectively on each blog entry. That’s very high for an abstract mathematical topic like complexity theory. But something even more extraordinary happened earlier this month. From Sunday 8th August till next Sunday 15th August Richard wrote 6 blogs on topics related to Vinay Deolalikar’s proof. Here is the list of blogs and the number of comments on each blog:

8-Aug A proof that P is not equal to NP (188 comments)

9-Aug Issues in the proof that P not equal to NP (121 comments)

10-Aug Updates on Deolalikar’s proof that P not equal to NP (184 comments)

11-Aug Deolalikar responds to issues about his P not equal to NP proof (129 comments)

12-Aug Fatal flaws in Deolalikar’s proof? (311 comments)

15-Aug P=NP “Proof” is one week old (257 comments)

That’s an average of 198 comments, roughly 8 times his average over the past 3 months. Some of the comments came from students and amateurs. But many comments came from what Richard called “The group”: Timothy Gowers, Gil Kalai, Ken Regan, Terence Tao, and Suresh Venkatasubramanian a powerful collection of top experts in the field. The Group found out three major gaps in the proof which Vinay has worked on and sent a final version for publication to a journal. Only time will tell whether his proof can fill the gaps but at this point odds are stacked against him.

Following sample comment gives us an idea of what Richard’s blog meant to readers:

I also want to thank Dick Lipton for the monumental effort. I am very thankful not only for the things already said (things learned, putting N != NP in the spotlight, etc) but also for something else that many researchers here take for granted: this was a public showing of the rigorous peer-review process that many people, specially aspiring researchers, non graduate students and amateur scientists, are unfamiliar with. For somebody attending a good university in his/her country of origin but where top notch research is not conducted, this whole exercise provided a glimpse about what’s like doing cutting edge work. Hopefully that might be a motivation to pursue a career in math or science later on.

Peer review in scientific publishing is undergoing a shift. “web-test” is becoming a powerful alternative to traditional “it-may-take-a-year” kind of review. Will review of this kind happen also in literature & social sciences? I guess so. Shakespeare Quarterly a 60-year old prestigious humanities journal is going to open its review to the web. “web-test” is still in its infancy but holds a great promise.

I would like to thank my friend D. Sivakumar for sending me the link to Richard’s blog. It is because of Siva that I can claim – I have a friend who is an expert in complexity theory!