Elementary, my dear Watson: Jeopardy Blues

Post by **vison** » Wed Feb 16, 2011 3:46 pm

I'm really surprised no one else has started this thread. The last 2 evenings an IBM "supercomputer" has played Jeopardy and displayed the most absolutely astonishing ability to do so.

This is a really big deal. Watson seems to be able to understand and speak "natural language".

http://www.google.ca/search?q=Computer+ ... CCcQqAIwAA

Post by **yovargas** » Wed Feb 16, 2011 4:02 pm

I really wanted to watch this. Is it up online anywhere? I thought there's no way it could compete against good humans but I guess I was wrong! We will soon be made irrelevant so enjoy being necessary for as long as you can.

Post by **axordil** » Wed Feb 16, 2011 4:34 pm

So long as the computer is answering questions and not formulating them I'm not too worried.

Post by **Primula Baggins** » Wed Feb 16, 2011 5:14 pm

I forgot this was on and caught only the end of last night's game. I'll be watching tonight.

Really, though, this is not that disturbing to me. I guess I'll have to watch it play more than Final Jeopardy to judge, but this computer was programmed specifically to understand Jeopardy questions, and it was not, I'm guessing, an easy task. But once that's accomplished, of course a computer with a human's understanding of the questions is going to wipe the floor with human players. It's got a bigger stock of accurate knowledge than any human brain could contain (but that's what computers are for, that's not some super power), and it can access it more quickly than any human can (again, what computers are for).

We invented them; they're our tools. Yes, I wrote a science fiction trilogy where intelligent machines had destroyed Earth before the story started, but in fact I don't think we're going to need computer assistance for that one.

Post by **nerdanel** » Wed Feb 16, 2011 5:24 pm

I thought this was going to be about the fact that the computer apparently misidentified Toronto as an American city last night? I don't know much about it, but someone posted a link on FB. It does call into question the conclusion that a well-programmed computer will automatically trump human intelligence.

Post by **Primula Baggins** » Wed Feb 16, 2011 5:33 pm

It was a good question to stump a computer, because it called for a sort of non-hierarchical tracing of information across several categories to arrive at the correct answer. Both humans got it, and apparently right away. The computer's answer was "Toronto??????" meaning it was very unsure. I would guess it couldn't come up with any answer that fit the necessary criteria.

Post by **Túrin Turambar** » Wed Feb 16, 2011 5:40 pm

It failed on the last question ('which U.S. city has it's major airport named after a WWII hero and it's second airport named after a WWII battle?' or something close to that) but got most of the others. Including questions where it had to use the title of the category to arrive at the answer. I was amazed that it could get arcane clues about fictional characters (eg. 'Wanted for general evil-ness. Last seen in Mordor').

Post by **vison** » Wed Feb 16, 2011 6:23 pm

I am not troubled or frightened by this machine, but I am impressed by it.

I wonder if PART of its advantage is speed. I would expect that the builders allowed for that - Watson had an actual button, but the signal to press that button might be quicker in a computer's wiring than in a human's. The guys that were playing were the best Jeopardy players ever and I am sure they could have answered the questions just as well, but it seemed to me that Watson MIGHT have had a speed advantage.

It was very weird to see it come up with "Toronto" when it was told that it wanted a US city. And it also seemed to me that it answered more difficult or more intuitive questions.

But it was EXCEEDINGLY cool. The last game is tonight and I would say, watch it.

It took me back to the early 60s when all the computer geeks were sure that true artificial intelligence was only a matter of time. Jack Davies and Rick Blasius, are you still alive and geeking?

Post by **Hurin** » Wed Feb 16, 2011 7:10 pm

Vison is right. I don't think there is any question that Watson has a speed advantage in buzzing in. The two guys playing against Watson are fantastic Jeopardy players and I'm sure they knew almost all the correct answers (or "questions"). But Watson managed to buzz in ahead of them at least 80% of the time (or so it seemed). With really good players, the ability to time the buzzer is extremely important (see Ken Jennnings's comments in the Wikipedia article on Watson). If I were given a half second advantage in buzzing in and could therefore answer all the questions I knew before my opponents could, I think I could beat Ken Jennings and Brad Rutter myself, even though they would know more of the overall answers than I would. Since Watson seems to know at least as many of the answers as the human players (if not more), he is winning easily because somehow he is able to buzz in faster.

Post by **yovargas** » Wed Feb 16, 2011 7:13 pm

As someone who had some (very basic) computer programming experience, this stuff is truly mind-boggling to me. The thing with Jeopardy is that it uses a lot of word-play, puns, associations, ect. so it truly needs an algorithm for parsing out real language and I doubted that we were anywhere near being able to parse out such nuanced problems (have you ever tried to get one of those voice-operated phone systems to understand anything???

). It is nearly as baffling to me as the software out there now that can recognize songs from "hearing" it. Neither of those seem possible to me in the 0s and 1s world of computers.

Post by **WampusCat** » Wed Feb 16, 2011 7:40 pm

I watched both shows and also was impressed by Watson's ability to answer complex questions. It was even more obvious when you saw the times he missed answers.

I found myself wondering about his wagering, though. In Final Jeopardy, for example, he bet less than a thousand dollars, even though he could have wagered much more without risk. A human competitor would have bet a dollar less than the difference between his score and twice his opponent's score. Wouldn't that have been programmed into Watson? Or was there a "let's not humiliate the humans more than is necessary" subprogram?

Post by **Túrin Turambar** » Wed Feb 16, 2011 7:57 pm

On his first double jeopardy he bet modestly but still a reasonable amount. On his second he bet small, but that was in a category where he’d already gotten an answer wrong. He was very cautious in final jeopardy, but then he had a big lead and was playing for best-of-two games anyway.

Post by **Maria** » Wed Feb 16, 2011 8:02 pm

An article about this says:

IBM frequently mentions creating a doctor's assistant, a tool that will help suggest potential diagnoses to the doctor who will make a final decision and bear the brunt of any second guessing during a malpractice lawsuit. Humans, it seems, are still necessary for some things.

That sounds like a pretty cool use of the technology. It might make up for the live docs who don't keep up with latest advances in their fields.

Post by **SirDennis** » Thu Feb 17, 2011 1:33 am

SPOILER ALERT

When Ken jumped ahead of Watson in today's episode I was gripped by a sudden expectation that Watson would sprout LASER cannons and vapourize him.

Post by **Dave_LF** » Thu Feb 17, 2011 2:39 pm

Artificial intelligence has been with us for some time, it just doesn't look like people expected it to look. It turns out that many mental tasks which are trivially easy for humans are incredibly difficult for the sorts of computers we design, and vice-versa (so have no fear; the machines may take your job, but they will still need you there to solve captchas for them!

). Watson seems to represent a major accomplishment, but I do wonder whether he is capable of applying his skills generally, or if his brain is just a giant, hard-coded rules-engine full of special cases that's good for playing Jeopardy and not much else.

Reading about him on Wikipedia, it sounds like a great deal of his success can be attributed to parallelism. This probably shouldn't be surprising. Although we're not usually aware of it, the human brain is massively parallel. When you're able to solve problems of balance and recognition "without thinking", it usually means some unconscious neural circuit solved it for you while your awareness was focused somewhere else. Watson has 2880 separate processor cores at his disposal, and can therefore consider more than 2000 separate possibilities simultaneously. Expect future breakthroughs in artificial intelligence to follow this pattern--networks of computers backed by huge data stores coordinating their efforts to solve complex problems.

Post by **yovargas** » Thu Feb 17, 2011 3:16 pm

An observation after watching the games last night - we may have been able to build a better Jeopardy player but Brad Rutter is much better looking than Watson so, for now, we're still good for something.

Post by **Túrin Turambar** » Thu Feb 17, 2011 5:28 pm

Personally, I think that Watson's win has to be as big news as Deep Blue's victory over Gary Kasparov was. Obviously he still has a lot of limitations, but that was an amazing feat nonetheless. Particularly his answer for final jeopardy, where he needed to work backwards (not 'which book inspired Bram Stoker's most famous novel?' but 'which author was inspired by whatever the name of the book was?').

Post by **Primula Baggins** » Thu Feb 17, 2011 5:38 pm

I liked Ken Jennings' little note following his Final Jeopardy answer: "I for one welcome our new computer overlords."

We watched, and it was impressive. However, some clues were naturals for the computer, such as giving a state's exact area in square miles as one of the clues. Not one that would help a typical human, but made for a computer.

"Smarter" computers are better tools for us.

Post by **yovargas** » Thu Feb 17, 2011 5:42 pm

Primula Baggins wrote:I liked Ken Jennings' little note following his Final Jeopardy answer: "I for one welcome our new computer overlords."

(This was an awesome Simpsons reference which, IMO, should have gotten Ken the $1M.

I LOL'd!

)

Post by **Túrin Turambar** » Thu Feb 17, 2011 5:54 pm

The other funny thing was watching the two other responses that Watson considered but passed over. My favourites were the #2 to 'which computer key is where the heart is?' (delete key) and 'what does a bad workman always blame?' (Yogi Berra).