Watson, come quickly I need you!

I watched the Jeopardy IBM Challenge this week and was quite impressed. For those who missed it, IBM's AI (artificial Intelligence) computer "Watson" played Jeopardy against two of the show's all time winners. Watson is a natural language processing system, which means it "understands" and "uses" English sentences for input and output.

I put the terms in quotes because we don't have a clear idea what it means to understand things or to use language. Certainly Watson did a good job of parsing the sentences fed to it (electronically - it does not do speech to text) and a good job speaking (although some of its pronunciation left a lot to be desired). From the sentence fed to it, it would determine what it was supposed to figure out much of the time and come up with a correct answer form its database of knowledge (no, it was not allowed to use the Internet).

It cleaned the clocks of the other contestants, but it won like the machine beat John Henry - by mechanical means. What I mean by that is that it could push the button faster than they could. Throughout the tournament you could see the contestants furiously trying to beat it to the button push and losing, and being frustrated by not being able to answer a question they knew the answer to.

Watson, on the other hand, routinely pushed the button first, unless it didn't know the answer. And the answers it didn't know were telling. Things dealing with "book learning" it knew very well, but it had "no common sense" as my kids put it. One interesting thing that had Alex Trebek and others puzzled were the unusual amounts it bet in the daily doubles and final Jeopardy. Generally a human will pick a number that will give him or her a "round" number or will pass another opponent. It seemed as though Watson took the maximum amount times some predicted probability of winning ignoring the scores of its opponents. Which makes sense, because I'm not sure if Watson knew the scores of its opponents. It did not know what their answers were, which became clear in the first game when it repeated a wrong answer its opponent had already given.

That's not to demean the incredible achievement of a machine that can play Jeopardy at all, and the hard work and talent that went into such an effort. On the contrary, as a computer scientist I find the project fascinating. I just thought the victory was cheapened by the buzzer speed. Had it been more "human" the contest would have been more about intellect and less about mechanisms.

Posted in: computing,technology