WSJ Blogs

Real-time commentary and analysis from The Wall Street Journal
Ideas Market

Why I Lost to Watson

    • hah

    • Would you be up for a rematch with the electronic question delivery taken out of the picture? I'd like to see Watson upgraded with OCR and voice recognition, so all players work on the same input. Do you agree that a victory with those rules would be much more credible?

    • I was thinking exactly the same as Stettin. Make Watson process the question via OCR and voice recognition and then see if it can still beat the humans to the buzzer. Heck - too make it totally fair. Watson should also be no bigger than a human male, should be in a android body, and should have to find it's way from "home" to the contest on it's own. It should have to walk, stand, catch a cab, and even press the button with an andriod arm and hand. Once it beats the human's with all these conditions. Then it is an impressive achievement.

    • @Stettin
      I agree. Plus, I have a hard time taking any contestant seriously who doesn't throw in an occasional racial slur or well timed "suck it Trabek".

    • @deepthough...... One step at a time. This is only the beginning.

    • @deepthough

      I think we should allow some sort of built-in rocket propulsion for the Watson-droid getting to the contest. Also it must be required to gloat upon giving a correct response and mope after an incorrect one.
      I'm thinking something from Ace Ventura for the gloat, and something from Wall-E for the mope.

    • Don't forget, you didn't just lose to Watson, you lost to Ken too.

    • Sore loser!

    • @zegoat, he is neither sore, nor a loser.

    • I understand that he was third place, but we need consider when WATSON buzzed before Brad. If it was just WATSON and Brad competing for the buzz (which could have happened many time), then that tells an entirely different story, because that money could have gone to Brad.

    • Way to try to down play the machine. The machine beat you becuase it was faster that says something right there. When the questions are up on the board the human can read them, correct. So Watson can read them faster that is an advantage 50% of humans have over each other. How about you stop complaining and let IBM have their victory. I read Brad say that it took years for the machine to learn all this information, so what Brad learned it in a day, im sorry I think it took him his whole life to get up to the same knowledge that Watson had in a few years. The ability to take an answer and get the clue right is pretty substantial for a computer and well to beat two of the smartest at their own game is pretty impressive. The 2 human players could have buzzed first anytime they wanted to, I am sure of it.

    • You're still the champ in my book. Watson might be able to answer faster, but it will never have a cool goatee like you do. Way to go Brad! You're still number 1.

    • Finally we learn the rules on when contestants can buzz. They have to wait for Trebek to FINISH reading the clue. Watson can't hear so it is getting an electronic notification of this event while Ken and Brad had to try to time his voice. This explains their facial reactions, particularly Ken's. He is hitting the buzzer too soon - ahead of Watson and Brad but before Trebek has finished reading.

      I hope the Jeopardy producers got enough loot from IBM to make up for the decline in viewership after this promotion.

      No, make that "there's no way they got enough"....if the lessons of Deep Blue vs Kasparov extend to TV game shows. How do I buy stock in "Wheel of Fortune"?

    • @deepthough I was thinking more of a next year rematch like Deep Blue had versus Garry Kasparov. I fully expect your scenario to be possible within my lifetime, however one year is pushing it. With new advances in nanotechnology and AI, we really might have to worry about being replaced by machines much smaller than our clunky human bodies.

    • I don't understand what Brad might mean by 'hunt[ing] for the Daily Doubles'. No clue-selection 'strategy' is any more or any less likely to find a Daily Double. If I have gone sequentially down the first three clues in Column C, the fourth clue in that column is just as likely to contain the Daily Double as is any other clue on the board. As far as finding the Daily Double goes, I don't improve my chances any by abandoning Column C.

      It's a pretty simple instance of the Gambler's Fallacy. And it's the same reason why, if I've tossed a coin for Heads 15 times in a row, that I still stand a 50/50 chance of throwing Heads on the next toss. The probability of each instance is statistically independent of past results.

      He'd have a point if clues 'hinted' at the presence of a Daily Double further down the column. In such a hypothetical case, I might improve my chances at finding one if I hadn't 'detected' such a 'hint' earlier in the column. But that's not the way the board works.

    • @Ross, actually, the indication for contestants is not Alex's voice, per se. The outer border of the clue board lights up when contestants are able to buzz in. So they have as precise and unmistakable an indication as Watson does, but obviously their reflexes are nowhere near as reliable as a computer's.

    • Steven,

      What he meant by hunting for Daily Doubles is not column-based. It's row-based. If you watch Jeopardy, you'll notice that the Daily Double is rarely in the first 2 row in each column. Statistically, it shows up in the 4th row more often than any other. Watson was programmed to hunt for the Daily Doubles in this way. By choosing clues in the 3rd, 4th, and 5th row, and skipping the first 2 rows, it increased the chances of finding the Daily Double.

      If the Daily Double were randomly placed, you would be correct. But it is not randomly placed. Just watch Jeopardy for a few weeks, and you'll see what Brad is talking about.

    • @Pat: I think you're right; I have never seen a daily double above the 3rd row.

    • One question I have regarding the operation of WATSON was this: If WATSON submits an incorrect answer, was it provided with the proper response after all players had responded? (Whether Ken or Brad answered correctly? Or if no player submits a correct answer, was WATSON provided with the correct response as the human players are by Mr. Trebeck?) If so, how did WATSON utilize this information? If WATSON was not provided with a correct answer, this could significantly effect the system's ability to resolve correct answers as the catagories progress.

    • The superficial comments here clearly demonstrate the lack of understanding of how remarkable Watson is. Yes, the computer is fast, but the fact that Watson could answer at all is more interesting, considering the difficutlies in making sense of natural speech. This is not about Brad's rather delicate ego, or how fast one can push a buzzer - this is about yet another small step forward in putting huge volumes of information at the fingertips of thinkers.

    • As a Jeopardy contestant in the 1984-85 season, I have always maintained "it's all about the buzzer!"

      We have a Jeopardy contestant group on LinkedIn and I encourage all former Jeopardy contestants to join us.

    • @sj25, this was one of the enhancements made-they changed the rules and fed Watson the answer from the other contestant. for example, when the human answered incorrectly, unitl this programming error was changed, Watson might have given the same wrong answer. Provided the correct answer, Watson learned valuable clues about a category. Provided the incorrect answer, Watson was able to eliminate that chioce from its list of possible answers.

    • I agree with Howard. This game was not to prove man is inferior to a machine. But to showcase to the world what Watson can do. Ken and Brad are the best and only humans will be, atleast for the foreseeable future.

    • @sj25 this was one of the enchancements made - they changed the rules to give Watson the answer of the other contestant. If the answer was correct, Watson was able to use that response to "learn" something about the category. If the answer given by one of the other contestants was wrong, Watson was able to eliminate that answer from its list of possible answers. This was an important aspect of Watson being able to win.

    • Being first, being right, that the "name of the game!" And this: "Watson was lucky, too: as impressive as the machine is, it still encounters categories that it can’t handle." Well, yes, OK, but with WATSON's $77,000 of chump change and the championship, and Mr. self-proclaimed “Humble Pie” in 3rd, WATSON was the winner . . . actually, the real winner was the cooler-than-Apple buzz surrounding IBM, as the Company gleefully recruits at top engineering schools, all while receiving thousands upon thousands of unsolicited job aps from folks wanting to work at IBM, especially on the WATSON team. How 'bout a rematch, and WATSON will have one circuit tied behind its digital back - hahahaha.

    • What if we eliminate the Buzzer, wouldn't the game be fairer?

    • I think we should eliminate the buzzer AND the questions. I don't think it could get more fair than that.

    • Stephen, IBM did lots of statistical studies on Daily Doubles. They're much more likelynto pop up in the more expensive categories. So by calling for the cheap categories, the player has little chance to land a daily doublemand risks losing control of the board. Stettin, no player, human or machine, listens to Alex reading the question. That is simply processing time. hearing would have saved Watson (ie IBM) the embarrassment of repeating Ken's incorrect response. But the team didn't think equipping Watson with voice recognitionnwas worth the time and effort.

    • Lost jobs for $2000 Alex...

      A: This company has outsourced more American jobs (for other companies) than any other over the last 10 years?
      Q Who is IBM?

    • Without a doubt, IBM’s Watson is a triumph in the area of man machine interactions. But can it really "think", "draw conclusions", or make a decision based upon "a gut feeling"? Would it be any "smarter" if it had ten thousand micro processors or another terabyte of RAM? Or would it just be faster? Can every nuance of natural language be represented by a mathematical formula? Can an algorithm make people laugh? A few years ago Netflix sponsored a contest, offering a one million dollar grand prize, if anyone could improve their movie recommendation algorithm by more than ten percent. No one in the entire world was able to... Perhaps mathematics has reached its theoretical limit in imbuing a machine with intelligence, or something that tries to mimic it. Maybe we need newer and better technologies, or a combination thereof to build machines that are truly smart.

      Although IBM has always excelled in the areas of sales and marketing, and has mastered the art of "showmanship" (the Jeopardy episodes were taped in an IBM television studio), what we really have to ask ourselves is how many jobs are there in the U.S. today for trivia fanatics? And how much more profitable can a corporation become by augmenting the intellectual capabilities of these few trivia experts? There are some start-ups in the United States, like nTeligence Corporation in Cary N.C., that are actively applying machine intelligence to solve mission critical business issues. The company that can tackle the real pressing intellectual problems that organizations face today, will ultimately become the undisputed leader in the race to build machines that can think.

    • I propse we send Watson down to Gitmo. Watson can interrogate the detainees in the Jeopardy! format. Think of it, an absolutely tireless interrogator. Will it be considered torture?

    • "Due to Watson’s advantage in buzzer timing, the IBM programmers rightly saw the Daily Doubles as the humans’ best opportunity to win, and had Watson take them off the board before we could potentially double our money."

      How does Watson know where they are? Or is it just a presumption that they usually are higher-value question, and so, Watson picked them first?

    • I believe Brad is in error on one point. According to several sources Watson only has the text of the question released to him at the moment they finish reading it. Being a computer professional that was one of the reasons I was so in awe of IBM's accomplishment in creating Watson. So, if indeed that is true, then actually the humans had an advantage over Watson, providing they could read faster than the host could recite. The same explanatory TV programs showed how Watson has a relay and solenoid to push the exact same pushbutton switch the human contestants employ. While there is no doubt there would be no hesitation by Watson once it reaches a conclusion, I'd say that this advantage would be offset by the extra tenth of a second or so the humans likely have to peruse before Watson even receives the question. Therefore I suspect the honors are about equal - as they should be.

    • Personally I am considering what the development of WATSON can give us the tools to extend the SUB-HUMAN-INTELLIGENCE to be built into relatively compact computers which could function as (better) language translations, voice recognition, and then produce an edited printed letter with a voice input from an illiterate person, etc. There is no need for "procreating" machines with human emotions and humors, which we already have too many of them(humans) in the world.

    • Some of the defenders of the computer and critics of the humans need to relax.

      Identifying the differences between the computer and human advantages (and disadvantages) is not only generally interesting, but also provides clues to improving both the computer's capability as well as the layman's understanding of where we are in artificial intelligence.

      For example, will computers ever be able to replicate emotion and its effect on decision-making, good and bad?

      I enjoyed the light summary provided in this article and did not at all smell sour grapes but good old fashion chutzpah.

    • Is it just me, or does this sound somewhat like sour grapes?

      Beaten by the buzzer? I didn't see Watson buzzing in and using and extra 1/2 second to a second thinking time before answering...

    • Um, I may be wrong.. But according to the NOVA Special about Watson, that aired a week before the games were televised.. Watson's robotic buzzer pressing device was set to a delayed reaction time, which was intentionally matched to the reaction speed of good Jeopardy! players.
      I can't remember the exact number they stated, but I believe it was between 100 and 200 milliseconds. Which sounds about right for the reaction speed of a human.

      Besides, there were several times when both players STILL managed to beat Watson to the buzzer, and no human would have been able to beat a 10 millisecond reaction speed, if that was true.
      ESPECIALLY in the final round, both Brad and Ken beat Watson to the buzzer many times, even when Watson was extremely sure of his answers.

    • Can someone say "butthurt"?

    • OkinKun, thanks for the clarification. Could Brad or anyone else confirm this? This seems like the million dollar question here.

      But generally, I'd stay any task that can be untrusted to machines is not a task you should be trying to get good at. Probably why I really liked to play chess, or any trivia games. Sports are another matter - while it is conceivable they could build a robot to win Federer, it would not diminish the fun of playing tennis even a little bit. What about Medicine? Could you build a computerized doctor, probably yes, and probably it would be a good thing, probably would solve the high cost of medical care as well.

      Whats my occupation? I not telling you.

    • sorry my post got totally garbled the first time.


      But generally, I’d stay any task that can be trusted to machines is not a task you should be trying to get good at. Probably why I never really liked to play chess, or any trivia games. Sports are another matter – while it is conceivable they could build a robot to win Federer, it would not diminish the fun of playing tennis even a little bit. What about Medicine? Could you build a computerized doctor, probably yes, and probably it would be a good thing, and probably this would solve the high cost of medical care as well.

      Whats my occupation? I not telling you. But I dont believe I could be replaced by computers, haha, hmm... oh wait...nooooooo....

    • Jennings beat you too... :)

    • Mr. Rutter is not the first contestant to point out the buzzer advantage. Another contestant used his right index finger to press the buzzer in his left hand. He beat the other contestants consistently with that tactic.

    • Maybe the machine should have been given a "human slow" and slightly random reaction time (but only in the case where it has already computed the answer by the time the question finishes). Winning because of better reflexes does not seem to be a very good test of A.I. or knowledge system performance. A computer always reacts faster than a human. There is no intelligence in that and no news. Of course the Watson might have won anyway. Anyway good publicity IBM! The Watson was at least a genius in that regard!

    • Let me first say that Ken did score higher than Brad when they both lost to Watson. However as I see it Brad is still the all-time Jeopardy champion. He is the player with the fewest losses. Brad has only lost once, last week when Watson beat both he and Ken. Jennings has now been beaten three times. One time during regular Jeopardy, a second time when Brad beat him in the most recent tournament of champions and then a third time last week to Watson. Ken made his name by competing in innumerable games in which he played against players who had never played the game before. He has never played other champions and won the game. In the last tournament of champions Ken only had to play in the final round while everyone else had to compete in an elimination tournament. Correct me if I am wrong but I believe the only time Jennings ever played against seasoned champions was the last round of the champion tournament in which Brad beat him? Brad has made his mark the hard way by consistently beating Jeopardy champions and he has done that in in every tournament of champions he has been in. Brad has only one loss, to Watson. Ken has three losses. Go Brad!!!

    • The only time Ken played against seasoned champions was at the Ultimate Tournament of Champions in which he got "ruttered" in all three games. Although Ken's run on Jeopardy was quite impressive, Brad is the all-time Jeopardy champion having won three tournaments and his only loss being to Watson. Way to go Brad!!

    • Man, everybody has a hard bromance for Brad... lol

    • @Stettin,

      Forcing Watson to use OCR/voice recognition would hardly make any difference at all. OCR, especially, is an essentially instantaneous operation for even a desktop computer. Watson was fed text answers the same as a human blind contestant would receive. The IBM engineers and Jeopardy staff knew this, and that's why they didn't impose such a requirement on Watson - it wouldn't "prove" anything interesting about what machines are capable of. The point of Watson was the answer processing algorithm.

    • Don't sweat it Brad. You never had a chance. Watson had a huge unfair advantage at buzzing in, not to mention a head start on processing the clues. It got a signal when the buzzers are enabled and of course a computer can respond much more quickly than a human. That's never been a contest. It should have
      been given a delay that made it more comparable to human response times. At the very least it should have had a sensor to respond to the buzzer light as humans have to. It turned into more of an IBM infomercial than a real competition. I think IBM is struggling to remain relevant in today's tech world with Microsoft, Apple, Google, etc and this was a a way to get the masses (especially those that buy stock) to remember IBM still exists.

      The innovative/impressive thing about the computer/software on Jeopardy is
      the language processing. The Jeopardy clues can be very ambiguous and
      indirect, often intentionally so. So far in the computer age we've
      had to be very specific in how we interact with computers and that has
      been a major limitation in their applications. This would open up
      computers to a lot more applications. YOu can see how this could be
      applied to having computers do telephone-based jobs like technical
      support, information, sales, etc. It's too bad IBM just turned this
      into a publicity stunt/infomercial instead of a fair competition of
      language processing and reasoning.

    • @Truth

      IBM has and still remains quite relevant in today's age and Watson is one such example. It's too bad you have to bring in such negative thoughts and opinions, instead of stepping back and seeing how this event has caused people to start talking about how we can use such knowledge and processing power to areas such as health care, which will ultimately improve the quality of life, reduce health care costs, and most importantly save lives. I saw this less of an IBM Server commercial, and more about technology that spurs hallway and lunch discussions about how advancement in technology can be used for real world applications that truly make a difference in society.

    • Good Job Brad. Well done.

    • Eventually humans, with their electronic "helpers", will build a machine that truly rivals human intelligence. What blows my mind is that a few years after that achievement the next model will probably be a billion times smarter than a human.

    • A few clarifications as I understand it from some discussions with the developers:
      1. RIGHT AND WRONG ANSWERS: Watson did not get other contestants wrong answers, such that it could eliminate them from its list--this was evidenced by Watson repeating the same wrong answer. Watson was only fed the correct answer after the question had been answered by another contestant or the moderator.
      2. BUZZING IN:As I understand it, Watson estimates the reading time of the question such that it knows about how much thinking time it has before needed to decide whether to buzz in or not--very similar to what a human contestant could do by listening. It would then only buzz in if its confidence level was high enough AND the light came on. Its main advantage in buzzing in was that it would never jump the gun, buzzing in before the question was read, and hence never incurred the 250ms lockout that occurs if you jump the gun.
      3. WAGERS: The oddball values for the wagers was not some higher-order/high faluting calculation thing--it was a suggestion from one of the developers purely for entertainment--the general size of the wage was based on theory of the game, but the least-significant digit randomization was just for grins.

    • I hope Brad's article was mostly tongue in cheek. If not then I'm sorry he didn't get it. Watson's triumph was not machine over man, much less machine over Brad. Jeopardy is a game. It was all about what the machine might be capable of in the future in terms of assisting man (even Brad) in leveraging vast amounts of information to the betterment of the human condition in ways only man can do - with ever better tools to assist in the pursuit of same.

Add a Comment


About Ideas Market


    • Ideas Market on Facebook