 Another big milestone for AI, a competition between a former debating world champion and a computer. So many people are here working on this production. Many of the previous demonstrations that we have seen were the realm of games. And games have very well defined set of rules. We could do simple things like play games, but you couldn't do complex things like having a debate. There is so many things that are connected together, and so many things need to work smoothly. I am actually a bit stressed. I feel the team didn't have enough time to practice. So I have to ask you, who do you think is going to win tonight? Well, I'm partial today, of course. What we really want is to create systems that work with us so that we can make better decisions. Ladies and gentlemen, here we go, Project Debater. Greetings Harish. I suspect you've never debated a machine. Welcome to the future. I started my amateur debating career back in high school. The teacher for history made the mistake of returning the exams and saying, if you want to appeal about your score, I'm willing to hear your thoughts. And obviously nobody seriously considered to do that except me. Okay, I understand that you think that this is my score. I have a different view. Let's argue about that in writing. Later on, I started to think about writing comedy sketches. I started to develop a sitcom called Puzzle. Oddly enough, the final episode of that sitcom is entirely focused on competitive debates. They filmed 16 episodes and it was a complete failure. This is how I found myself doing computer science and artificial intelligence. For centuries, people have always thought that games is a great way to measure your intelligence. When I play chess, I want to prove to myself I'm smarter than the other side. If you take the position that you have to hand code a rule for every situation, it is insanely hard. What you now do is you give them examples and they learn from those examples and make their own rules compared to having to craft all the rules by hand. Games have been important for AI, but games differ from debate because the classical game is a set of equations that's fully understood. You can draw the rules of chess. You cannot draw the rules of conversation. That's impossible. This is Jeopardy! The IBM Challenge. And now we all get to find out how well Watson will perform. We're looking at the top of the screen. We're looking for Bram Stoker. And we find... Who is Bram Stoker and the wager? Hello! And we have a new champion, Watson. Just a couple of days after the Jeopardy! event, an email was sent to the entire research division asking us what should be the next AI Grand Challenge. I was intrigued. So I offered my office mate at the time to spend an hour and brainstorm together. And towards the end of the hour, at some point, I suggested this notion of developing a machine that will be able to debate humans. So we submitted the proposal in a single slide. They told us this is impossible and this is not a very good plan. I think this is a characteristic of a Grand Challenge because if it is very clear to you that this is doable, then probably this is not a Grand Challenge. And if it is very clear to you that this is not doable, then probably you shouldn't start. It should be somewhere in between, but you need to take into consideration that perhaps you will fail. I didn't hear anything for a few months. Then February 2012, Aya Sofer used this chat service in IBM to text me. And she said, did you hear? And I said, no. What? And she said, it was selected. The beta was selected as the next Grand Challenge. I was very excited and I wrote her back. Aya, I want to thank you for all the support along the way. And she said, don't thank me yet. Thank you very much. Both teams in this debate, both sides of this debate agree that women have been discriminated against. I'm Harish Natarajan. And my debating record is that I won the European Championships in 2012, the Oxford Intervarsity and the Cambridge Intervarsity. I have twice won the largest open tournament in the world, the LSE Open. And certainly, even if I'm not necessarily the best debater in the world, I'll certainly be very high up most people's list. What I think debating forces you to do is you just to defend and justify your own opinions, but defend and justify often the very opposite opinions as well. And I think that process was so valuable because I think it makes us interrogate the other side of the coin. It makes us interrogate the beliefs and the views which we hold. One of the things that happens when you take a grand challenge is that you have to slice it up into question that each of them is less grand. From the beginning, one thing you would need is a massive collection of newspaper and journal articles on the order of 10 billion sentences. Then the debater system needed the capability to automatically identify the exact boundaries of the claim within the billions of sentences. And figure out if those arguments are supporting your side of the debate or the other side. This is something that is very easy for humans to do based on our knowledge about concepts in the world. And it is very complicated for machines to do. Now you need to arrange a meaningful narrative. After the opening statement, now there's rebuttal speech. This is being able to listen to an opponent and respond accordingly. Those were the main challenges that we had in front of us. And if one of these steps is not working properly, you will end up with a useless speech. Total effort and timeline. After we were selected, it was very hard for us to make significant progress. Only four months after we started to work for the topic physical education should be mandatory in high school, the debater system detects the following claim in Wikipedia. Several studies have shown that lizards display no benefit from exercise. This is going to sound ridiculous, okay? They're going to ask us about physical exercise, whether it should be mandatory or not. Then how are we going to understand that, you know, these claims are actually referring to lizards and not to high school students? It will be very hard to get even close to a meaningful debate between this system and an expert human debater. First time I heard about Project Debater was one of those small articles probably back in 2014 where they talked about this debating Watson project. And at the time it suddenly seemed like a pipe dream. Because humans are rarely just convinced by evidence and logical, rational argument. It's about really connecting with them in one of many different ways, some of which aren't as rational as others. Welcome to today's debate on the motion. First generation immigrants should encourage their children to fully integrate into majority culture. In the world today, what we see is people scared of immigrants, scared from integration because... What I do is I coach debaters and I tell them, this is what you need to do in order to improve your culture. This is what you need to do in order to improve. And usually a human debater, it just kind of their brain understands what I'm saying and they can convert that into being a better debater. Sometimes not, but usually. You did two things very well in this debate, okay? You took the stand of... This is about... And in this case, I'm saying it to the programmers and they're saying, okay, how do we take that learning process that happens inside people's heads and convert that into some sort of algorithm? And that's where the magic happens. It seems to me that this is a motion that basically for the government it's impossible to defend. Well, you can agree in general that you can ban... Like any nicotine product to begin with. What I'm saying is that if you will debate me, not as an expert debater, I will make this point. Is there an answer to that? Being able to show to the world that you can take a computer program and teach it not just to play a game that has a very specific set of rules and very specific ways of winning, but something that is so difficult for humans to learn and so subjective in how you see it and how you hear arguments and how you respond to arguments, I think that that's... It's incredible. When I joined the team, it was not at a level where it would be competitive. My automatic analysis indicates that Jarre suggested that the abolition of the entire fossil fuel field is going to create and the traumatic effects of that. Lastly, however, considering relevant data suggests some information that may contradict this claim. I first heard about IBM Debater because they were asking for volunteers to record speeches for them. Basically, they wanted to build up this database of speeches from people with a lot of debating experience and then train the computer using that database. We should not ban smoking. We should ban smoking. On our side of this debate, we're going to explain... We should not ban nuclear weapons. Despite the obvious risks that these weapons pose... I think at first, Project Debater would sometimes give arguments that weren't necessarily for its side. Such mandatory minimum sentences lead to over-crowded prisons, and rightfully so. These were the main points I wanted to raise. The types of arguments, the connections between arguments, the way that we sometimes tell jokes or sometimes we're more serious, those are all things that they've pulled in for Project Debater. I often compare Project Debater to my kids and their development because I guess my oldest was born about the same time the program was born. In terms of personality, they'd be, I guess, on par, right? My seven-year-old has a great personality. They do develop that quite early. It's scary, but also nice to see that Project Debater is developing a personality as well. I don't know whether to call my opponent naive, old-fashioned, or just a romantic, but this way or that, my opponent is wrong. I remember the debater, she called him naive, and he was really hurt. Suddenly he's laughed at, not only by a machine, but the audience are now bursting out laughing. My name is Asaf Gavron. I'm an author most of the days. I'm in charge of writing what we call generic texts, little bits and little phrases that are put together by the software, metaphors or quotes, opening sentences or jokes. We have debated in the past, but never in front of a real audience. I am curious to see how well you will perform on the big stage when the cameras are rolling and the stakes are high. Good luck. We had a lot of discussions about who is the debater. My friend, you are speaking at the extremely fast rate of 203 words per minute. Don't hurry, we have plenty of time. Please slow down and present your arguments calmly. We never write the text to a specific debate. We don't know what the debate will be. No, the magic of debater is that it can argue on any topic thrown at it under the sun. So we write a general text that could fit some topics and then the system has to identify them and use the right ones in the right time. Climate change is what allows them to poison the minds of young kids. We know the earth is warming. Will there be more droughts? Yes. How definitive is the evidence? Is there any room for debate? This is the Natasha Hall Show on CJAD 800. We are speaking with Dr. Aharonov. She is the global manager for IBM's artificial intelligence project, Debater. Thanks for being here. How nuts is that it could take artificial intelligence to teach us to understand other human perspectives better? I think that's where I should be going. Why? Why do we need robots to teach us how to debate? Ultimately, making a decision involves how you weigh your decisions. Ultimately, making a decision involves how you weigh your different values. It's not all about information and I think that AI is not going to do for you. But it can give you all the information so you can make a better decision. You caught me in one of the most tense days ever. Flying in three days. Obviously, we had the bug that we found last week, naturally. You see results that are just not consistent. I'm going to run this tomorrow, I'm going to get a different result. And you have no idea where it came from. It was really what's called a ghost bug. It just didn't make any sense at all. This was testing how much we can stretch our nerves. So that was fixed by a patch to Java and then we had to rerun everything and go from there. But it's fixed. Test again. Go ahead and just play something constant and I'll bring it in slowly. Test again, I can wake up. Test again, I can wake up. Test again, I can wake up. It's a single debate. It's just one debate. So in a single debate, many things can happen. And Harish is a much better debater than any debater we faced in the past. Let me introduce John Donvan. I host a debate program, Intelligence Squared US. Our goal has always been to raise the level of public discourse. This truly is a first for us, the first time that an artificial intelligence, namely Project Debater, will be on our stage arguing with a human being and may the best debater win. So let's get started. Let's first meet and applaud our debaters. First arguing for the resolution tonight will be IBM Project Debater. Representing the rest of us, please come to the stage, Harish. So to reveal now the resolution of the evening is this. We should subsidize preschool. We should subsidize preschool. We are going to ask you to vote before and after the arguments using your mobile phone to decide whether or not you want to vote. Using your mobile phone to tell us where you stand on this position and to tell us whether you were persuaded or not by one side or the other. Project Debater actually has a gender. She will be arguing for the resolution. Ladies and gentlemen, here we go, Project Debater. Greetings Harish. I have heard you hold the world record in debate competition wins against humans. But I suspect you've never debated a machine. Welcome to the future. I will argue that we should subsidize preschools. For decades, research has demonstrated that high quality preschool is one of the best investments of public dollars resulting in children who fare better on tests and have more successful lives than those without the same access. In December 2015, researchers at Duke University found that students who enroll in preschool are 39% less likely to be placed in special education programs. A statistical summary of studies by the National Institute for Early Education Research found that high quality preschool can create long term academic and social benefits for individuals and society far exceeding costs. While I cannot experience poverty directly and have no complaints concerning my own standards of living, I still have the following to share. The Centers for Disease Control and Prevention reports that universal full day preschool creates significant economic savings in health care as well as decreased crime, welfare dependence and child abuse. Why don't we examine the evidence and the data and decide accordingly? Thank you for listening. Well thank you very much everybody. It's a pleasure to be here for this historic event. There was a lot of information in that speech and lots of facts and lots of figures. The problem though is the reality of subsidizing preschools is one which does not deal with the underlying problems in society. I'm not sure if you massively increase the number of people going to preschool, they're all going to be the ones going to the high quality preschools. Mr. Natarajan suggested that preschools should not be subsidized because this will reduce their quality. If anything, the opposite is true. One of many reasons is that subsidizing attracts more skilled and qualified people to the field, improving the quality of preschools for all. Preschool is pushing that child to learn in a competitive environment at the age of three or four when you realize you aren't necessarily as talented as someone else. That huge psychological damage for many children may mean that preschool is actively harmful. My opponent claimed that preschools are harmful. I believe my argument suggested that the benefits outweigh the potential disadvantages. You cannot fund everything. I think this is simply empirically true and you have to make choices and you have to make trade-offs. Many middle class parents and many people from upper incomes already send their children to preschool. You're taking money from all taxpayers to help those individuals who are already often the best off. To be clear, my intention is not to leave a suitcase full of money for everyone to grab at will. We are talking about a limited, targeted, and helpful mechanism. I don't think the project debater has helped those individuals. She identifies as the most important, but in reality has hurt them. Thank you and that concludes round three and the argument phase of this debate. We're on our way to making history here. We would like to ask you now to complete that process by using your phones and choose your position where you stand now that you've heard the arguments from both sides. At Intelligence Squared, we deliver victory to the side whose numbers have moved up the most in percentage point terms. Harish Natarajan, his first vote was 13 percent, his second vote was 30 percent, he pulled up 17 percentage points. That is it. Harish Natarajan arguing against the resolution, subsidized preschool, declared our winner. Congratulations to them. Oh, and we have the second vote. Which of the two debaters better enriched your knowledge? Let's see what that number is. Project debater. Better enriched the knowledge of the audience on that side. So a little bit of a split decision. This is amazing and I think regardless we made history tonight because project debater held her own. It's fine to disagree. It is not fine not to listen and not to really try to understand the other side. We see artificial intelligence technologies everywhere, right? We use that in our cars and we use that to select music and we use that when we try to buy something. What about discussions between us? Thanks for joining us here in Geneva for the AI for Good Global Summit 2019. I think the debater is really just the beginning. It touches on a very fundamental problem, understanding the pros and cons for taking better decisions. So for example a government which is considering a policy and the citizens that are going to be affected by this policy. I don't think it will replace the decision makers but it will help them. Greetings all. The citizens of Lugano have contributed more than 1000 arguments in support of the claim that we should further explore the development of autonomous vehicles. The following is a summary of their thoughts. The crowd raised four issues. The first will demonstrate how autonomous cars reduce accidents.