David Ferrucci: IBM Watson, Jeopardy & Deep Conversations with AI | Lex Fridman Podcast #44
following is a conversation with David Ferrucci he led the team that built Watson the IBM question-answering system that beat the top humans in the world at the game of Jeopardy for spending a couple hours of David I saw a genuine passion not only for abstract understanding of intelligence but for engineering it to solve real-world problems under real-world deadlines and resource constraints where science meets engineering is where brilliant simple ingenuity emerges people who work adjoining it to have a lot of wisdom earned two failures and eventual success David is also the founder CEO and chief scientist of elemental cognition a company working to engineer AI systems that understand the world the way people do this is the artificial intelligence podcast if you enjoy it subscribe on YouTube give it five stars and iTunes support it on patreon or simply connect with me on Twitter Alex Friedman spelled Fri D M a.m. and now here's my conversation with David Ferrucci your undergrad was in biology with a with an eye toward medical school before you went on for the PhD in computer science so let me ask you an easy question what is the difference between biological systems and computer systems in your when you sit back look at the Stars and think philosophically I often wonder I often wonder whether or not there is a substantive difference and I think the thing that got me into computer science and artificial intelligence was exactly this presupposition that if we can get machines to think or I should say this question this philosophical question if we can get machines to think to understand to process information the way do we do so if we can describe a procedure or describe a process even if that process where the intelligence process itself then what would be the difference so from philosophical standpoint I'm not trying to convince that there are there is I mean you can go in the direction of spirituality you can go in the direction of a soul but in terms of you know what we can what we can experience from an intellectual and physical perspective I'm not sure there is clearly there implement there are different implementations but if you were to say as a biological information processing system fundamentally more capable than one we might be able to build out of silicon or or some other substrate I don't I don't know that there is how distant do you think is the biological implementation so fundamentally they may have the same capabilities but is it really a far mystery where a huge number of breakthroughs are needed to be able to understand it or is that something that for the most part in the important aspects echoes are the same kind of characteristics yeah that's interesting I mean I so you know your question presupposes that there's this goal to recreate you know what we perceive is biological intelligence I'm not I'm not sure that's the I'm not sure that that's how I would state the goal I mean I think that studying the goal good so I think there are a few goals I think that understanding the human brain and how it works is important for us to be able to diagnose and treat issues for us to understand our own strengths and weaknesses both intellectual psychological and physical so neuroscience and on sending the brain from that perspective has a there's a clear clear goal there from the perspective of saying I want to I want to I want to mimic human intelligence that one's a little bit more interesting human intelligence certainly has a lot of things we Envy it's also got a lot of problems too so I think we're capable of sort of stepping back and saying what do we want out of it what do we want out of an intelligence how do we want to communicate with that intelligence how do we want to behave how do we want it to perform now of course it's it's somewhat of an interesting argument because I'm sitting here as a human with a biological brain and I'm critiquing this trends and weaknesses of human intelligence and saying that we have the capacity just the capacity to step back and say gee what what is intelligence is what do we really want out of it and that even in and of itself suggests that human intelligence is something quite amiable that it could you know it can it can it can introspect that it could introspect that way and the flaws you mentioned the flaws the human self yeah but I think I think that flaws that humans wholeness house is extremely prejudicial and bias and the way it draws many inferences do you think those are sorry to interrupt you think those are features or are those bugs do you think the the prejudice the forgetfulness the fear what other flaws list them all what love maybe that's a flaw you think those are all things that can be get gotten getting in the way of intelligence or the essential components of and well again if you go back and you define intelligence as being able to sort of accuracy accurately precisely rigorously reason develop answers and justify those answers in an objective way yeah then human intelligence has these flaws and that it tends to be more influenced by some of the things you said and it's and it's largely an inductive process meaning it takes past data uses that to predict the future very advantageous in some cases but fundamentally biased and prejudicial in other cases because it's gonna be strongly influenced by its priors whether they're whether they're right or wrong from some you know objective reasoning perspective you're gonna favor them because that's those are the decisions or those are the paths that succeeded in the past and I think that mode of intelligence makes a lot of sense for when your primary goal is to act quickly and and and survive and make fast decisions and I think those create problems when you want to think more deeply and make more objective and reasons that decisions of course humans capable of doing both they do sort of one more naturally than they do the other but they're capable of doing both you're saying they do the one that responds quickly in it more naturally right because that's the thing you kind of need to not be eaten by the Predators in the world for example but I mean better than we've we've learned to reason through logic we've developed science we train people to do that I think that's harder for the individual to do I think it requires training and you know and and and teaching I think we are human - certainly is capable of it but we find more difficult and then there are other weaknesses if you will as you mentioned earlier it's just memory capacity and how many chains of inference can you actually go through without like losing your way so just focus and so the way you think about intelligence and we're really sort of floating this philosophical slightly but I think you're like the perfect person to talk about this because we'll get to jeopardy and beyond that's like an incredible one of the most incredible accomplishments in AI in the history of AI but hence the philosophical discussion so let me ask you've kind of alluded to it but let me ask again what is intelligence underlying the discussions we'll have with with jeopardy and beyond how do you think about intelligence is it a sufficiently complicated problem being able to reason your way through solving that problem is that kind of how you think about what it means to be intelligent so I think of intelligence to primarily two ways one is the ability to predict so in other words if I have a problem what's gonna can I predict what's going to happen next whether it's to you know predict the answer of a question or to say look I'm looking at all the market dynamics and I'm going to tell you what's going to happen next or you're in a in a room and somebody walks in and you're going to predict what they're going to do next or what they're going to say next doing that in a highly dynamic environment full of uncertainty be able to lots of lockdown the more the more variables the more complex the more possibilities the more complex but can I take a small amount of prior data and learn the pattern and then predict what's going to happen next accurately and consistently that's a that's certainly a form of intelligence what do you need for that by the way you need to have an understanding of the way the world works in order to be able to unroll it into the future all right thank you one thing is needed to predict depends what you mean by understanding IIIi need to be able to find that function and this is very much like what function deep learning does machine learning does is if you give me enough prior data and you tell me what the output variable is that matters I'm going to sit there and be able to predict it and if I can predict you predict it accurately so that I can get it right more often than not I'm smart if I do that with less data and less training time I'm even smarter if I can figure out what's even worth predicting I'm smarter meaning I'm figuring out what path is gonna get me toward a goal what about picking a goal so again well that's interesting about picking our goal sort of an interesting thing I think that's where you bring in what do you pre-programmed to do we talked about humans and humans a pre-programmed to survive so sort of their primary you know driving goal what do they have to do to do that and that that could be very complex right so it's not just it's not just figuring out that you need to run away from their ferocious tiger but we survive in social context as an example so understanding the subtleties of social dynamics becomes something that's important for surviving finding a mate reproducing right so we're continually challenged with complex sets of variables complex constraints rules if you will that we we or patterns and we learn how to find the functions and predict the things in other words represent those patterns efficiently and be able to predict what's going to happen that's a form of intelligence that doesn't really record that doesn't really require anything specific other than ability to find that function and and predict that right answer it's certainly a form of intelligence but then when we when we say well do we understand each other in other words do would you perceive me as as intelligent beyond that ability to predict so now I can predict but I can't really articulate how I'm going to that process what my underlying theory is for predicting and I can't get you to understand what I'm doing so that you can follow you can figure out how to do this yourself if you hadn't if you did not have for example the right pattern matching machinery that I did and now we have potentially have this breakdown where in effect I'm intelligent but I'm sort of an alien intelligence relative to you you're intelligent but nobody knows about it or I can see the I can see the output knowing so so you're saying let's to separate the two things one is you explaining why you were able to predict the future and and the second is me being able to like impressing me that you're intelligent me being able to know that you successfully predicted the future do you think that's well it's not a pressing you item intelligent in other words you may be convinced that I'm intelligent in some form so high well because of my ability to predict so I would imagine that wow wow you're right all here you're you're right more times than I am you're doing something interesting that's a form that's a form of intelligence but then what happens is if I say how are you doing that and you can't communicate with me and you can't describe that to me now I'm a label you a savant I mean I may say well you're doing something weird and it's and it's just not very interesting to me because you and I can't really communicate and and so now this is interesting right because now this is you're in this weird place where for you to be recognized as intelligent the way I'm intelligent then you and I sort of have to be able to communicate and then my we start to understand each other and then my respect and my my appreciation my ability to relate to you starts to change so now you're not an alien intelligence anymore yours you're our human intelligence now because you and I can communicate and so I think when we look at when we look at when we look at animals for example animals can do things we can't quite comprehend we don't quite know how they do them but they can't really communicate with us they can't put what they're going through in our terms and so we think of them in sort of low there are these alien intelligences and they're not really worthless so what we're worth we don't treat them the same way as a result of that but it's it's hard because who knows what you know what's going on so just a quick elaboration on that the explaining that you're intelligent the explaining the the reasoning the one end to the prediction is not some kind of mathematical proof if we look at humans look at political debates and discourse on Twitter it's mostly just telling stories so you usually your task is sorry that your task is not to tell an accurate depiction of how you reason but to tell a story real or not that convinces me that there was a mechanism by which you ultimately that's what a proof is I mean even a mathematical proof is is that because ultimately the other mathematicians have to be convinced by your proof otherwise in fact they're been that the measurement success yeah yeah there have been several proofs out there where mathematicians would study for a long time before they were convinced that it actually proved anything right you never know if it proved anything until the community of mathematicians decided that it did so I mean so it's but it's it's a real thing yeah and and that's sort of the point right is that ultimately on you know this notion of understanding us understanding something there's ultimately a social concept in other words you I have to convince enough people that I I did this in a reasonable way I did this in a way that other people can understand and and replicate and that make sense to them so we're very human Houghton's is bound together in that way we're bound up in that sense we sort of never really get away with it until we can consider convince others that our thinking process you know make sense did you think the general question of intelligence is then also social constructs so if we task asked questions of an artificial intelligence system is this system intelligent the answer will ultimately be a socially constructed I think I think so I so I think you're making to be a mess I'm saying we can try to define intelligence in this super objective way that says here here's this data I want to predict this type of thing learn this function and then if you get it right often enough we consider you intelligent but that's more like a stepfather that I think it I think it is it doesn't mean it's useful if it could be incredible useful it could be solving a problem we can't otherwise solve and can solve it more reliably than we can but then there's this notion of can humans take responsibility for the decision that you're that you're making can we make those decisions ourselves can we relate to the process that you're going through and now you as an agent whether you're a machine or another human frankly are now obliged to make me understand how it is that you're arriving at that answer and allow me I mean me or the obviously a community or a judge of people to decide whether or not whether or not that makes sense and by the way that happens with the humans as well you're sitting down with your staff for example and you ask for suggestions about what to do next and someone says well I think you should buy and I think you should buy this much or would have or sell or whatever it is or I think you should launch the product today or tomorrow or launch this product versus that product whatever decision may be and you ask why and the person so I just have a good feeling about it and it's not you're not very satisfied now that person could be you know you might say well you've been right you know before but I'm gonna put the company on the line can you explain to me why I should believe this and that explanation may have nothing to do with the truth just them and all them convinced the wrong yes they'll be wrong she's got to be convincing but it's ultimately got to be convinced and that's why I'm saying it's we're bound together right our intelligences are bound together in that sense we have to understand each other and and if for example you're giving me an explanation I mean this is a very important point right you're giving me an explanation and I'm and I and I and I have iton I'm not good and then I'm not good at reasoning well and being objective and following logical paths and consistent paths and I'm not good at measuring and sort of computing probabilities across those paths what happens is collectively we're not going to do we're not going to do well how hard is that problem the second one so we I think will talk quite a bit about the the first on a specific objective metric benchmark performing well but being able to explain the steps the reasoning how hard is that probably that's I think that's very hard I mean I think that that's um well it's hard for humans the thing that's hard for humans as you know may not necessarily be hard for computers and vice-versa so sorry so how hard is that problem for computers I think it's hard for computers and the reason why are related to or saying that it's also hard for humans is because I think when we step back and we say we want to design computers to do that one of the things we have to recognize is we're not sure how to do it well I'm not sure we have a recipe for that and even if you wanted to learn it it's not clear exactly what data we use and what judgments we use to learn that well and so what I mean by that is if you look at the entire enterprise of science science is supposed to be at a bad objective reason and reason right so we think about who's the most intelligent person or group of people in the world do we think about the savants who can close their eyes and give you a number we'd think about the think tanks or the scientists of the philosophers who kind of work through the details and write the papers and come up with the thoughtful logical proves and use the scientific method and I think it's the latter and my point is that how do you train someone to do that and that's what I mean by it's hard how do you what's the process of training people to do that well that's a hard process we work as a society we work pretty hard to get other people to understand our thinking and to convince them of things now we could for so weighed them obviously talked about this like human flaws or weaknesses we can persuade through persuade then through emotional means but to but to get them to understand and connect to and follow a logical argument is difficult we try it we do it we do it as scientists we try to do it as journalists we know we try to do it as you know even artists in many forms as writers as teachers we go to a fairly significant training process to do that and then we could ask what why is that so hard but it's hard and for humans it takes a lot of work and when we step back and say well step back and say well how do we get a machine - how do we get a machine to do that it's a vexing question how would you begin to try to solve that and maybe just a quick pause because there's an optimistic notion in the things you're describing which is being able to explain something through reason but if you look at algorithms that recommend things that we look at next well there's Facebook Google advertising based companies you know their goal is to convince you to buy things based on anything so that could be reason because the best of advertisement is showing you things that you really do need and explain why you need it but it could also be through emotional manipulation the algorithm that describes why a certain reason a certain decision was was made how hard is it to do it through emotional manipulation and why is that a good or a bad thing so you've kind of focused on reason logic really showing in a clear way why something is good one is that even a thing that us humans do and and and - how do you think of the differences in the reasoning aspect and the emotional manipulation well they you know so you call it emotional manipulation but more objectively is essentially saying you know thing you know there are certain features of things that seem to attract your attention I'm gonna kind of give you more of that stuff manipulation is a bad word yeah I mean I'm not saying it's good right or wrong is it it works to get your attention and it works to get you to buy stuff and when you think about algorithms that look at the patterns of the you know patterns of features that you seem to be spending your money on and is there going to give you something with a similar pattern so I'm going to learn that function because the objective is to get you to click on and/or get you to buy and or whatever it is I don't know I mean that it is like it is what it is I mean that's what the algorithm does you can argue whether it's good or bad it depends what your you know what your what your goal is I guess this seems to very useful for convincing telling us the thing for convincing humans yeah it's good because you gives again this goes back to how does a human you know what is the human behavior like how does a human you know brain respond to things I think there's a more optimistic view of that too which is that if you're searching for certain kinds of things you've already reasoned that you need them and these these algorithms are saying look that's up to you the reason whether you need something or not that's your job you know you you met you may have an unhealthy addiction to this stuff or you may have a reasoned and thoughtful explanation for why it's important to you and the algorithms are saying hey that's like whatever like that's your problem all I know is you're buying stuff like that you're interested in stuff like that could be a bad reason could be a good reason that's up to you I'm gonna show you more of that stuff and so and I and I and I think that that's it's not good or bad it it's not reason or not reason the algorithm is doing what it does which is saying you seems to be interested in this I'm going to show you more that stuff and I think we're seeing it's not just in buying stuff but even in social media you're reading this kind of stuff I'm not judging on whether it's good or bad I'm not reasoning at all I'm just saying I'm gonna show you other stuff with similar features and you know and like and that's it and I wash my hands from it and I say that's all you know that's all what's going on you know there is you know people are so harsh on AI systems so one the bar of performance is extremely high and yet we also asked them to in the case of social media to help find the better angels of our nature and help make a better society so what do you think about the role of it that so that agrees you that's that's the interesting dichotomy right because on one hand we're sitting there and we're sort of doing the easy part which is finding the patterns we're not building the systems not building a theory that it's consumable and understandable other humans that could being explained and justified and and so on one hand to say oh you know AI is doing this why isn't doing this other thing well those other things a lot harder and it's interesting to think about why why why it's harder and because you're interpreting you're interpreting the data in the context of prior models in other words understandings of what's important in the world what's not important what are all the other abstract features that drive our decision-making what's sensible what's not sensible what's good what's bad what's moral what's valuable what is it where is that stuff no one's applying the interpretation so when I when I see you clicking on a bunch of stuff and I look at these simple features the raw features the features that are there in a data like what words are being used or how long the material is more other very superficial features what colors are being used in the material like I don't know why you're clicking on the stuff you're looking or if it's products what the price of what the price is or what the categories or stuff like that and I just feed you more of the same stuff that's very different than kind of getting in there and saying what does this mean what the stuff you're reading like why are you reading it what assumptions are you bringing to the table are those assumptions sensible is the miss the material make any sense does it does it lead you to thoughtful good conclusions again there's judgment this interpretation judgment involved in that process that isn't really happening in in in the AI today that's harder right because you have to start getting at the meaning of this of the of the stop of the content you have to get at how humans interpret the content relative to their value system and deeper thought processes so that's what meaning means is not just some kind of deep timeless semantic thing that the statement represents but also how a large number of people are likely to interpret so that's again even meaning is a social construct it's so you have to try to predict how most people would understand this kind of statement yeah meaning is often relative but meaning implies that the connections go beneath the surface of the artifact so if I show you a painting it's a bunch of colors in a canvas what does it mean to you and it may mean different things at different people because of their different experiences it may mean something even different to the artist to who painted it as we try to get more rigorous with our communication we try to really nail down that meaning so we go from abstract art to precise mathematics precise engineering drawings and things like that we're really trying to say I want to narrow that that space of possible interpretations because the precision of the communication ends up becoming more and more important and so that means that I have to specify and I think that's why this becomes really hard because if I'm just showing you an artifact and you're looking at it superficially whether it's a bunch of words on a page or whether it's you know brushstrokes on a canvas or pixels on a photograph you can sit there and you can interpret lots of different ways at many many different levels but when I want to when I want to align our understanding of that I have to specify a lot more stuff that's actually not in it not directly in the artifact now I have to say well how you were how are you interpreting this image and that image and what about the colors and what do they mean to you what's what perspective are you bringing to the table what are your prior experiences with those artifacts what are your fundamental assumptions and values what what is your ability to kind of reason to chain together logical implication as you're sitting there and saying well if this is the case then I would conclude this and if that's the case then I would conclude that and it so your reasoning processes and how they work your prior models and what they are your values and your assumptions all those things now come together into the interpretation getting in sick of that is hard and yet humans able to intuit some of that without any pre because they have the shared experience me and we're not talking about shared two people have any shares know me as a society that's correct we have this shared experience and we have similar brains so we tend to Institute in other words part of our shared experiences are shared local experience like we may live in the same culture we may live in the same society and therefore we have similar education we have similar what we like to call prior models about the world prior experiences and we use that as a think of it as a wide collection of interrelated variables and they're all bound to similar things and so we take that as our background and we start interpreting things similarly but as humans we have it we have a lot of shared experience we do have similar brains similar goals similar emotions under similar circumstances because we're both humans so now one of the early questions you ask well how is biological and you know computer information systems fundamentally different well one is you know one is come you means come with a lot of pre-programmed stuff yeah a ton of program stuff and they were able to communicate because they have a lot of it because they share that stuff do you think that shared knowledge if it can maybe escape the hardware question how much is encoded in the hardware just the shared knowledge in the software the the history the many centuries of wars and so on that came to today that shared knowledge how hard is it to encode and did you have a hope can you speak to how hard is it to encode that knowledge systematically in a way that could be used by a computer so I think it is possible to learn to form machine to program machine to acquire that knowledge with a similar foundation in other words in a similar interpretive interpretive foundation for processing that knowledge but what do you mean by that so in other in other words foundation we view the world in a particular way and so in other words we we have i if you will as humans we have a frame reference for bringing the world around us so we have multiple frameworks for interpreting the world around us but if you're interpreting for example social political interactions you're thinking about what there's people there's collections and groups of people they have goals the goals largely built around survival and quality of life that are their fundamental economics around scarcity of resources and when when humans come and start interpreting a situation like that because you've brought you've grown up like historical events they start interpreting situations like that they apply a lot of this a lot of this this fundamental framework for interpreting that well who are the people what were their goals what users did they have how much power influence that they have over the other like this fundamental substrate if you will for interpreting and reasoning about that so I think it is possible to in view a computer with that that stuff that humans like take for granted when they go and sit down and try to interpret things and then and then with that with that foundation they acquire they start acquiring the details the specifics in any given situation are then able to interpret it with regard to that framework and then given that interpretation they can do what they can predict but not only can they predict they can predict now with an explanation that can be given in those terms in the terms of that underlying framework that most humans share now you could find humans that come in interpret events very differently than other humans because they're like using a different different framework you know movie matrix comes to mind where you know they decided the humans were really just batteries and that's how they interpreted the value of humans as a source of electrical energy so but um but I think that you know for the most part we we have a way of interpreting the events or do social events around us because we have to share at framework it comes from again the fact that we're we're similar beings that have similar goals similar emotions and we is we can make sense out of these these frameworks make sense to us so how much knowledge is there do you think so it's you said it's possible well there's all its tremendous amount of detailed knowledge in the world there you know you can imagine you know effectively infinite number of unique situations and unique configurations of these things but the the knowledge that you need what I refer to as like the frameworks for you for interpreting them I don't think I think that's those are finite you think the frameworks I'm more important than the bulk of them now so it's like framing yeah because the frameworks do is they give you now the ability to interpret and reason and to interpret and reasoning to interpret and reason over the specific in ways that other humans would understand what about the specifics you know who acquired the specifics by reading and by talking to other people and so mostly actually just even if we can focus on even the beginning the common-sense stuff the stuff that doesn't even require reading or animalistic requires playing around with the world or something just being able to sort of manipulate objects drink water and so on all does that every time we try to do that kind of thing in robotics or AI it seems to be like an onion you seem to realize how much knowledge is really required to perform you in some of these basic tasks do you have that sense as well and if so how do we get all those details are they written down somewhere idea they have to be learned through experience so I think when like if you're talking about sort of the physics the basic physics around us for example acquiring information about for acquiring how that works yeah I think that I think there's a combination of things going I think there's a combination of things going on I think there is like fundamental pattern matching like what were you talking about before where you see enough examples enough data about something you start assuming that and with similar input I'm going to predict similar outputs you don't can't necessarily explain it at all you may learn very quickly that when you let something go it falls to the ground that's a that's a sickness is horribly explained that but that's such a deep idea if you let something go like they do gravity I mean people were letting things go and counting on them falling well before they understood gravity but that seems to be a that's exactly what I mean is before you take a physics class or the or study anything about Newton just the idea that stuff falls to the ground and they be able to generalize that other all kinds of stuff falls to the ground it just seems like a non if without encoding it like hard coding it in it seems like a difficult thing to pick up it seemed like gift of Allah of different knowledge to be able to integrate that into the framework sort of into everything else so both know that stuff falls to the ground and start to reason about social political discourse so both like the very basic and the high-level reasoning decision-making I guess my question is how hard is this problem and sorry to linger on it because again and we'll get to it for sure as well Watson with jeopardy did its take on a problem that's much more constrained but has the same hugeness of scale at least from the outsider's perspective so I'm asking the general life question of to be able to be an intelligent being and reason in the in the world about both gravity and politics how hard is that problem so I think it's solvable okay now beautiful so what about what about time travel okay convinced not as convinced yet okay no I said I I think it is I mean I I took it as solvable I mean I think that it's alert it's versatile it's about getting machines to learn learning is fundamental and I think we're already in a place that we understand for example how machines can learn in various ways right now our learning our learning stuff is sort of primitive in that we haven't sort of taught machines to learn the frameworks we don't communicate our frameworks because of our shared in some cases we do but we don't annotate if you will all the data in the world with the frameworks that are inherent or underlying our understanding instead we just operate with the data so if we want to be able to reason over the data in similar terms in the common frameworks we need to be able to teach the computer or at least we need to program the computer to require to have access to and acquire learn the frameworks as well and connect the frameworks to the data I think this I think this can be done I think we can start I think machine learnings for example with enough examples can start to learn these basic dynamics will they relate the necessary to gravity not unless they can also acquire those theories as well and put the experiential knowledge and connected back to the theoretical knowledge I think if we think in terms of these class of architectures that are are designed to both learn the specifics find the patterns but also acquire the frameworks and connect the data to the frameworks if we think in terms of robust architectures like this I think there is a path toward getting there jeez in terms of encoding architectures like that do you think systems they were able to do this will look like and you know that works or representing if you look back to the eighties and nineties of the expert systems so more like graphs the systems that are based in logic able to contain a large amount of knowledge where the challenge was the automated acquisition of that knowledge the I guess the question is when you collect both the frameworks and the knowledge from the data what do you think that thing will look like yeah so I mean I think think is asking a question they look like neural networks is a bit of a red herring I mean I think that they they will they will certainly do inductive or pattern match based reasoning and I've already experimented with architectures that combine both that use machine learning and neural networks to learn certain classes of knowledge in other words to find repeated patterns in order or in order for it to make good inductive guesses but then ultimately to try to take those learnings and and marry them in other words connect them to frameworks so that it can then reason over that in terms of their humans understand so for example at elemental cognition we do both we have architectures that that do both but both those things but also have a learning method for acquiring the frameworks themselves and saying look ultimately I need to take this data I need to interpret it in the form of these frameworks so they can reason over it so there is a fundamental knowledge representation like what you saying like these graphs of logic if you will there are also neural networks that acquire certain class of information they then they they and align them with these frameworks but there's also a mechanism to acquire the frameworks themselves yes so it seems like the idea of framework requires some kind of collaboration with humans absolutely so do you think of that collaboration as well and unless to be clear let's be clear only for the for the express purpose that you're designing you you're designing machine designing and intelligence that can ultimately communicate with humans in terms of frameworks that help them understand things right so so now to be really clear you can create you can independently create an a machine learning system and an intelligent intelligence that I might call an alien's elegans that does a better job than you with some things but can't explain the framework to you that doesn't mean is it might be better than you at the thing it might be that you cannot comprehend the framework that it may have created for itself that is inexplicable to you that's a reality but you're more interested in a case where you can I I am yeah I per might sort of approach to AI is because I've set the goal for myself I want machines to be able to ultimately communicate understanding with human I want to meet would acquire and communicate acquire knowledge from humans and communicate knowledge to humans they should be using what you know inductive machine learning techniques are good at which is to observe patterns of data whether it be in language or whether it be in images or videos or whatever to acquire these patterns to induce the generalizations from those patterns but then ultimately work with humans to connect them to frameworks interpretations if you will that ultimately make sense to humans of course the machine is gonna have the strength egg it has the richer or longer memory but that you know it has the more rigorous reasoning abilities the deeper reasoning abilities so be it interesting you know complementary relationship between the human and the machine do you think that ultimately needs explained ability like a machine so if we look we study for example Tesla autopilot a lot or humans I don't know if you've driven the vehicle or are aware of what is it so you basically the human and machine are working together there and the human is responsible for their own life to monitor the system and you know the system fails every few miles and so there's there's hundreds of there's millions of those failures a day and so that's like a moment of interaction DC yeah that's exactly right that's a moment of interaction where you know the the the machine has learned some stuff it has a failure somehow the failures communicated the human is now filling in the mistake if you will or maybe correcting or doing something that is more successful in that case the computer takes that learning so I believe that the collaboration between human and machine I mean that's sort of a permanent example of sort of a more another example is where the machine is literally talking to you and saying look I'm I'm reading this thing I know I know that like the next word might be this or that but I don't really understand why I have my gas can you help me understand the framework that supports this and then can kind of take acquire that take that and reason about it and reuse it the next time it's reading to try to understand something not on not unlike a human student might do I mean I remember like when my daughter was the first great in she was had a reading assignment about electricity and you know somewhere in in the text it says and electricity is produced by water flowing over turbines or something like that and then there's a question that says well how was electricity created and so my daughter comes to me and says I mean I could you know created and produced or kind of synonyms in this case so I can go back to the text and I can copy by water flowing over turbines but I have no idea what that means like I don't know how to interpret water flowing over turbines and what electricity even is I mean I can get the answer right by matching the text but I don't have any framework for understanding what this means at all and framework really I mean it's a set of not to be mathematical but axioms of ideas that you bring to the table and interpreting stuff and then you build those up somehow you build them up with the expert that there's a shared understanding of what they are Sheriff it's the social network that us humans do you have a sense that humans on earth in general share a set of like how many frameworks are there I mean it depends on how you bound them right so in other words how big or small like their their individual scope but there's lots and there are new ones I think they're I think the way I think about is kind of an a layer I think that the architectures are being layered in that there's there's a small set of primitives that allow you the foundation to build frameworks and then there may be you know many frameworks but you have the ability to acquire them and then you have the ability to reuse them I mean one of the most compelling ways of thinking about this is or reasoning by analogy where I could say oh wow I've learned something very similar you know I never heard of this I never heard of this game soccer but if it's like basketball in the sense that the goals like the hoop and I have to get the ball in the hoop and I have guards and I have this and I have that like we're weird is the where where are the similarities and where the difference is and I have a foundation now for interpreting this new information and then the different groups like the Millennials will have a framework and then and then well that you never you know yeah well Kratz and Republicans well I Neal's nobody wants that framework well I mean I think understands it right I mean you're talking about political and social ways of interpreting the world around them and I think these frameworks are still largely largely similar I think they differ in maybe what some fundamental assumptions and values are now from a reasoning perspective like the ability to process the framework of Magna might not be that different the implications of different fundamental values or fundamental assumptions in those framework frameworks may reach very different conclusions so from so from a social perspective that conclusions may be very different from an intelligence perspective I you know I just followed where my assumptions took me yeah the product the process itself would look similar but that's a fascinating idea that frameworks really helped carve how a statement will be interpreted I mean having a Democrat and the Republican framework and read the exact same statement and the conclusions that you derive would be totally different from an ad respective is fascinating what we would want out of the AI is to be able to tell you that this perspective one perspective one set of assumptions is going to lead you here in other setups as luncheons is gonna leave you there and to and in fact you know to help people reason and say oh I see where I see where our differences lie yeah you know I have this fundamental belief about that I have this fundamental belief about that yeah that's quite brilliant from my perspective and NLP there's this idea that there's one way to really understand a statement but there probably isn't there's probably an infinite number of ways then just as well well there's a lot finding on there's lots of different interpretations and the you know the the broader you know the broader to the the contents the richer it is and so you know you you and I can have very different experiences with the same text obviously and if we're committed to understanding each other we start and that's the other important point like if we're committed to understanding each other we start decomposing and breaking down our interpretation towards more and more primitive components until we get to that point where we say oh I see why we disagree and we try to understand how fundamental that disagreement really is but that requires a commitment to breaking down that interpretation in terms of that framework in a logical way otherwise you know and this is why I like I think of a eyes is really complementing and helping human intelligence to overcome some of its biases and its predisposition to be persuaded by you know buys but more shallow reasoning in the sense that like we get over this idea well I you know you know I'm right because I'm a Republican or I'm right because I'm democratic and someone labeled this is democratic point of view or it has the following keywords in it and and if the machine can help us break that argument down and say wait a second you know what do you really think about this right so essentially holding us accountable to doing more critical thinking to sit and think about that as fast that's I love that I think that's really empowering use of AI for the public discourse it's completely disintegrating currently I don't know as we learn how to do it on social medias right so one of the greatest accomplishments in the history of AI is Watson competing against in a game of Jeopardy against humans and you were a lead in that accrue at a critical part of that let's start the very basics what is the game of Jeopardy the game for us humans human versus human right so it's to take a question and answer it actually no but it's not right it's really not it's really it's really to get a question and answer but it's what we call a factoid questions so this notion of like it's it really relates to some fact that everything few people would argue whether the facts are true or not in fact most people what and jeopardy kind of counts on the idea that these these statements have factual answers and and the idea is to first of all determine whether or not you know the answer which is sort of an interesting twist so first of all understand the question you have to understand the question what is it asking and that's a good point because the questions are not asked directly right they're all like the way the questions are asked is nonlinear it's like it's a little bit witty it's a little bit playful sometimes it's a it's a little bit tricky yeah they're asked and exactly in numerous witty tricky ways exactly what they're asking is not obvious it takes it takes an experienced humans a while to go what is it even asking right and it's sort of an interesting realization that you have was a missus Oh what's the Jeopardy is a question answering Shou and there's a go like I know a lot and then you read it and you're you're still trying to process the question and the champions have answered and moved on there's like there's three questions ahead at the time you figured out what the question even met so there's there's definitely an ability there to just parse out what the question even is so that was certainly challenging it's interesting historically though if you look back at the jeopardy games much earlier you know 63 yeah and I think the questions were much more direct it weren't quite like that they got sort of more and more interesting the way they asked them that sort of got more and more interesting and subtle and nuanced and humorous and witty over time which really required the human to kind of make the right connections and figuring out what the question was even asking so yeah you have to figure out the questions even asking then you have to determine whether or not you think you know the answer and because you have to buzz in really quickly you sort of have to make that determination as quickly as you possibly can otherwise you lose the opportunity buzz in you've been going before you really know if you know the answer I think well I think a lot of humans will will assume they'll they'll look at the look at their process of very superficially in other words what's the topic what are some key words and just say do I know this area or not before they actually know the answer then they'll buzz in and then I'll buzz in and think about it it's interesting what humans do now some people who know all things like Ken Jennings or something or the more recent big jeopardy player that knows about that though just assume they know although jeopardy and I'll just pose it you know Watson interestingly didn't even come close to knowing all of Jeopardy right Watson even at the peak even at that's been yeah so for example I mean we had this thing called recall which is like how many of all the Jeopardy questions you know how many did could we even find like find the right answer for like anywhere like could we come up with if we look you know we had up a big body of knowledge some of the order of several terabytes I mean from from a web scale was actually very small but from like a book scales talking about millions in bucks right so the equivalent millions of books and cyclopædia is dictionaries books it's a ton of information and you know for I think was 80 only 85% was the answer anywhere to be found hmm so you're ready down you're ready down at that level just to get just to get started right so and so was important to get a very quick sense of do you think you know the right answer to this question so we have to compute that confidence as quickly as we possibly could so it's in effect to answer it and at least you know spend some time essentially answering it and then judging the confidence that we you know that that our answer was right and in deciding whether or not we were confident enough to buzz in and that would depend on what else was going on in the game it could because it was a risk so like if you're really in a situation where I have to take a gas I have very little to lose then you'll buzz in with less confidence so that was the counter for the the financial standings of the different competitors cracks yeah how much of the game was laughs how much time was left and where were you were in the standings things like that what how many hundreds of milliseconds that we're talking about here do you have a sense of what is we targets because we yeah was the targeted so I mean we targeted answering and under three seconds and buzzing it so the decision to buzz in and then the actual answering are those two yes there were two there were two different things in fact we had multiple stages whereas like we would say let's estimate our confidence which which is sort of a shallow answering process and then ultimate and then ultimately decide to buzz in and then we may take another second or something it's kind of go in there and and do that but by and large we're saying like we can't play the game we can't even compete if we can't on average answer these questions and around three seconds or less so you stepped in so there's this there's these three humans playing a game and you stepped in with the idea that IBM Watson would be one of replaced one of the humans and compete against two can you tell the story of Watson taking on this game sure seems exceptionally difficult yeah so the story was that um it was or it was coming up I think the 10-year anniversary of a big blue an optical deep blues IBM wanted to do sort of another kind of really you know fun challenge public challenge that can bring attention to IBM research and the kind of cool stuff that we were doing I had been working in an AI at IBM for some time I had a team doing what's called open domain factoids question-answering which is you know we're not gonna tell you what the questions are we're not even gonna tell you what they're about can you go off and get accurate answers to these questions and it was an area of AI research that I was involved in and so it was a big Pat it was a very specific passion of mine language understanding and always always been a passion of mine one sort of narrow slice on whether or not you could do anything was language was this notion of open domain and meaning I could ask anything about anything factoids meaning it essentially had an answer and and you know being able to do that accurately and quickly so that was a research area that might even already been in and so completely independently several you know IBM exactly there's like what are we gonna do what's the next cool thing to do and Ken Jennings was on his winning streak this was like whatever was 2004 I think was on his win winning streak when someone thought hey that'd be really cool um if the computer can play jeopardy and so this was like in 2004 they were shopping this thing around and everyone who's telling the the research execs no way like this is crazy and we had some pretty you know senior people know if you'll understand the others crazy and he'll come across my desk and I was like but that's kind of what what I'm really interested in doing and but there was such this prevailing sense of this is nots we're not going to risk IBM's reputation on this we're just not doing it and this happened in 2004 it happened in 2005 at the end of 2006 it was coming around again and I was coming off of a I was doing that the open domain question-answering stuff but I was coming off a couple other projects I had a lot more time to put into this and I argued that it could be done and I argue it would be crazy not to do this can I you could be honest at this point so even though you argued for it what's the confidence that you had yourself privately that this could be done it was we just totally told the story of how you tell stories to convince others how confident were you what was your estimation of the problem at that time so I thought it was possible and a lot of people thought it was impossible I thought it was possible a reason why I thought it was possible is because I did some brief experimentation I knew a lot about how we were approaching on open domain factoids question asked me we have been doing it for some years I looked at the Japanese stuff I said this is going to be hard for a lot of the points that you mentioned earlier hard to interpret the question hard to do it quickly enough hard to compute an accurate confidence none of this stuff had been done well enough before but a lot of the technologies were building with the kinds of technologies that should work but more to the point what was driving me was I was an IBM research I was a senior leader in IBM Research and this is the kind of stuff we were supposed to do we were basically supposed to the moonshot this is I mean we were supposed to take things and say this is an active research area it's our obligation to kind of if we have the opportunity to push it to the limits and if it doesn't work to understand more deeply why we can't do it and so I was very committed to that notion saying folks this is what we do it's crazy not not to do this is an active research area we've been in this for years why wouldn't we take this Grand Challenge and and push it as hard as we can at the very least we'd be able to come out and say here's why this problem is is way hard here's what we've tried and here's how we failed so I was very driven as a scientist from that perspective and then I also argued based on what we did a feasibility study oh why I thought it was hard but possible and I showed examples of you know where it succeeded where it failed why it failed and sort of a high level architecture approach for why we should do it but for the most part that at that point the execs really were just looking for someone crazy enough to say yes because for several years at that point everyone has said no I'm not willing to risk my reputation and my career you know on this thing clearly you did not have such fears okay I did not say you died right in and yet for what I understand it was performing very poorly in the beginning so what were the initial approaches and why did they fail well there were lots of hard aspects to it I mean one of the reasons why prior approaches that we had worked on in the past um failed was because of because the questions were difficult difficult to interpret like what are you even asking for right very often like if if the question was very direct like what city you know or what you know even then it could be tricky but but you know what city or what person was often when it would name it very clearly you would know that and and if there was just a small set of them in other words we're gonna ask about these five types like it's gonna be an answer and the answer will be a city in this state or a city in this country the answer will be a person of this type right like an actor or whatever it is but turns out that in jeopardy there were like tens of thousands of these things and it was a very very long tale meaning you know that it just went on and on and and so even if you focused on trying to encode the types at the very top like there's five that were the most let's say five of the most frequent you still cover a very small percentage of the data so you couldn't take that approach of saying I'm just going to try to collect facts about these five or ten types or twenty types or fifty types or whatever so that was like one of the first things like what do you do about that and so we came up with a an approach toward that and the approach to look promising and we we continue to improve our abilities to handle that problem throughout the project the other issue was that right from the outside I said we're not going to I committed to doing this in three five years so we did in four so I got lucky um but one of the things that that putting that like stake in the ground was I and I knew how hard the language of the standard problem was I said we're not going to actually understand language to solve this problem we are not going to interpret the question and the domain of knowledge the question refers to in reason over that to answer these questions were obviously we're not going to be doing that at the same time simple search wasn't good enough to confidently answer with this you know a single correct answer first others like brilliant that's such a great mix of innovation in practical engineering three three four eight so you're not you're not trying to solve the general NLU problem you're saying let's solve this in any way possible oh yeah no I was committed to saying look we're gonna solving the open the main question answering problem we're using jeopardy as a driver for that management hard enough big benchmark exactly and now we're how do we do it we're just like whatever like just figure out what works because I want to be able to go back to the académica scientific community and say here's what we tried here's what work here's what didn't work I don't want to go in and say oh I only have one technology hammer and only gonna use this I'm gonna do whatever it takes I'm like I'm gonna think out of the box do whatever it takes one um and I also Baloo's another thing I believed I believe that the fundamental NLP technologies and machine learning technologies would be would be adequate and this was an issue of how do we enhance them how do we integrate them how do we advance them so I had one researcher and came to me who had been working on question answering with me for a very long time who had said we're gonna need Maxwell's equations for question-answering and I said if we if we need some fundamental formula that breaks new ground and how we understand language we're screwed yeah we're not gonna get there from here like we I am not counting I am that my assumption is I'm not counting on some brand new invention what I'm counting on is the ability to take everything that has done before to figure out a an architecture on how to integrate it well and then see where it breaks and make the necessary advances we need to make and sold this thing works yeah push it hard to see where it breaks and then patch it up I mean that's how people change the world and that's the you know mosque approaches Rockets SpaceX that's the Henry Ford and so on a lot and and I happen to be and in this case I happen to be right but but like we didn't know right but you kind of have to put a second or so how you gonna run the project so yep and backtracking to search so if you were to do what's the brute force solution what what would you search over so you have a question how would you search the possible space of answers look web search has come a long way even since then but at the time like you know you first of all I mean there are a couple of other constraints around the problems interesting so you couldn't go out to the web you couldn't search the Internet in other words the AI experiment was we want a self-contained device device if devices as big as a room fine it's as big as a room but we want a self-contained advice contained device you're not going out the internet you don't have a life lifeline to anything so it had to kind of fit in a shoebox if you will or at least the size of a few refrigerators whatever it might be see but also you couldn't just get out there you couldn't go off Network right to kind of go so there was that limitation but then we did it but the basic thing was go go do what go do a web search the problem was even when we went and did a web search I don't remember exactly the numbers but someone the order of 65% at a time the answer would be somewhere you know in the top 10 or 20 documents so first of all that's not even good enough to play Jack pretty you know the words even if you could pull the avian if you could perfectly pull the answer out of the top 20 documents top 10 documents whatever was which we didn't know how to do but even if you could that do that your you'd be at and you knew it was Ryan Lizza we've had enough confidence in it right so you have to pull out the right answer you have you depth of confidence it was the right answer and and then you'd have to do that fast enough to now go buzz in and you'd still only get 65% of them right with nine doesn't even put you in the winner's circle winner's circle you have to be up over 70 and you have to do it really quick and you do really quickly but now the problem is well even if I had somewhere in the top 10 documents how do I figure out where in the top 10 documents that answer is and how do i compute a confidence of all the possible candidates so it's not like I go in knowing the right answer and I have to pick it I don't know the right answer I have a bunch of documents somewhere in there's the right answer how do i as a machine go out and figure out which ones right and then how do I score it so and now how do I deal with the fact that I can't actually go out to the web first of all if you pause and then just think about it if you could go to the web do you think that problem is solvable if you just pause on it just thinking even beyond jeopardy do you think the problem of reading text defined where the answer is but we saw we solved that and some definition of solves given the Jeopardy challenge how did you do it forever so how did you take a body of work and a particular topic and extract the key pieces of information so what so now forgetting about the the huge volumes that are on the web right so now we have to figure out we did a lot of source research in other words what body of knowledge is gonna be small enough but broad enough to answer Jeffrey and we ultimately did find the body of knowledge that did that I mean it included Wikipedia and a bunch of other stuff so like encyclopedia type of stuff I don't know if you use Mary's different types of semantic resources unlike wordnet and other types of Mantic resources like that as well as like some web crawls in other words where we went out and took that content and then expanded it based on producing statistical see you know statistically producing sees using those sees for other searchers searches and then expanding that so using these like expansion techniques we went out and had found enough content and we're like okay this is good and we even up and totally and you know we had a threat of resources always trying to figure out what content could we efficiently include I mean there's a lot of popular cut like what is the church lady well I think was one of the end hey yeah what we ready I guess that's probably an encyclopedia so it's a pepino is that but then we would but then we would take that stuff when we would go out and we would expand in other words we go find other content that wasn't in the core resources and expanded you know the amount of content will grew it by an order of magnitude but still so again from a web scale perspective this is very small amount of content it's very select we then we then took all that content so we we pre analyzed the crap out of it meaning we we we parsed it you know broke it down into all this individual words and then we did semantic static and semantic parses on it you know had computer algorithms that annotated it and we in that we indexed that in a very rich and very fast index so we have a relatively huge amount of you know let's say the equivalent of for the sake of argument two to five million bucks we've now analyzed all that blowing up at size even more because now with all this metadata and we then we richly indexed all of that and in by way in a giant in-memory cache so Watson did not go to disk so the infrastructure component there if you just speak to it how tough it I mean I know mm maybe this is 2089 you know that that's kind of a long time ago right how hard is it to use multiple machines Olivia how hard is the infrastructure part of the hardware component we used IBM we so we used IBM hardware we had something like I figured exactly but 2,000 to 3,000 cores completely connected so had a switch were you know every CPU was connected to every other scene they were sharing memory in some kind of way Lauren up close shared memory right and all this data was pre analyzed and put into a very fast indexing structure that was all all all in all in memory and then we took that question we would analyze the question so all the content was now pre analyzed so if I so if I went and tried to find a piece of content it would come back with all the metadata that we had pre computed how do you shove that question how do you connect the the big stuff with the meta the the big knowledgebase of the metadata and that's indexed to the simple little witty confusing question right so therein lies you know the Watson architects right so we would take the question we would analyze the question so which means that we would parse it and interpret it a bunch of different ways we try to figure out what is it asking about so we would come we had multiple strategies to kind of determine what was it asking for that might be represented as a simple string and character string or was something we would connect back to different semantic types that were from existing resources so anyway the bottom line is we would do a bunch of analysis and the question and question analysis had to finish and had to finish fast so we do the question analysis because then from the question analysis we would now produce searches so we would and we had built using open source search engines we modified them we had a number of different search engines we would use that had different characteristics we went in there and engineered and modified those search engines ultimately to now take our question analysis produce multiple queries based on different interpretations of the question and fire out a whole bunch of searches in parallel and they would produce combate with passages so this is these are passive search algorithms they would come back with passages and so now you let's say you had a thousand passages now for each passage you you parallel eyes again so you went out and you paralyze those paralyze the search each search would now come back with a whole bunch of passages maybe you had a total of a thousand or five thousand different passages for each passage now you don't figure out whether or not there was a candidate it would call it candidate answer in there so you had a whole bunch of other a whole bunch of other algorithms that would find candidate answers possible answers to the question and so you had candidate answers jet cold candidate answers generators a whole bunch of those so for every one of these components the team was constantly doing research coming up better ways to generate search queries from the questions better ways to analyze the question better ways to generate candidates and speed so better is accuracy and speed cracked so right and speed and accuracy for the most part we're separated we handle that sort of in separate ways like I focus purely on accuracy and to an accuracy are we ultimately getting more questions and producing more accurate confidences and they had a whole nother team that was constantly analyzing the workflow to find the bottlenecks and then if you're getting out of both parallel eyes and drive the algorithm speed but anyway so so now think of it like you have this big fan out now right because you have you had multiple queries now you have now you have thousands of candidate answers for each candidate answer you're gonna score it so you're gonna use all the data that built up you're gonna use the question analysis you can use how the query was generated you're going to use the passage itself and you're going to use the candidate answer that was generated and you're gonna score that so now we have a group of researchers coming up with scores there are hundreds of different scores so now you're getting a fan at it again from however many candidate answers you have to all the different scorers so if you have a 200 different scores and you never a thousand candidates now you have two thousand scores and and so now you got to figure out you know how do I now rank these rank these answers based on the scores that came back and I want to rank them based on the likelihood that there are correct answer to the question so every score was its own research project what do you mean by score so is that the annotation process of basically human being saying that this this answer do you think you think of if you want to think of it what you're doing you know if you want to think about what a human would be doing human would be looking at a possible answer they'd be reading the you know Emily Dixon Dickinson they've been reading the passage in which that occurred they'd be looking at the question they'd be making a decision of how likely it is that Emily Dixon Dickinson given this evidence in this passage is the right answer to that quad got it so that that's the annotation task that Stan Johnson scoring task so but scoring implies zero to one kind of trite continuance is not a binary no give it a score give it a zero yeah exactly so it's what humans did give different scores so that you have to somehow normalize and all that kind of stuff that deal with all that depends on what your strategy is we both we could be relative to it could be we actually looked at the raw scores as well standardized scores because humans are not involved in this humans are not involved sorry so I mean I'm misunderstanding the the the process here this is passages where is the ground truth coming from grass root there's only there were answers to the questions so it's end to end it's end to end so we also I was always driving and and performance a very interesting a very interesting you know engineering approach and ultimately scientific and researcher personal always driving in 10 now that's not to say we wouldn't make hypotheses that individual component performance was related in some way to n10 performance of course we would because people would have to build individual components but ultimately to get your component integrates with the system you had to show impact on end-to-end performance question-answering performance as there's many very smart people work on this and they're basically trying to sell their ideas as a component that should be part of the system that's right and and they would do research on their component and they would say things like you know I'm going to improve this as a candidate generator I'm going to improve this as a question score or as a passive scorer I'm going to proved as or as a parser and I can improve it by two percent on its component metric like a better parse or better candidate or a better type estimation or whatever it is and then I would say I need to understand how the improvement on that computer metric is going to affect the end-to-end performance if you can't estimate that and can't do experiments to demonstrate that it doesn't get in that's like the best run AI project I've ever heard that's awesome okay what breakthrough would you say like I'm sure there's a lot of day to day break this but it was there like a breakthrough that really helped improve performance like wait what people began to believe or is it just a gradual process well I think it was a gradual process but one of the things that I think gave people confidence that we can get there was that as we fouled as as we follow this procedure of different ideas build different components plug them into the architecture run the system see how we do do the error analysis start off new research projects to improve things and the and and and the very important idea that the individual component work did not have to deeply understand everything that was going on with every other component and this is where we we leverage machine learning in a very important way so while individual components could be statistically driven machine learning components some of them were your wrist ik some of them were machine learning components the system has a whole combined all the scores using machine learning this was critical because that way you can divide and conquer so you can say okay you work on your candidate generator or you work on this approach to answer scoring you work on this approach to type scoring you work on this approach to passage search or the passive selection and so forth but when we you just plug it in and we had enough training data to say now we can we can train and figure out how do we weigh all the scores relative to each other based on the predicting the outcome which is right right or wrong on jeopardy and we had enough training data to do that so this enabled people to work independently and to let the machine learning do the integration beautiful so that yeah the machine learning is doing the fusion and then it's a human orchestrated ensemble that's right friend approaches as a great still impressive they were able to get it done a few years that not obvious to me that it's doable if I just put myself in that mindset but when you look back at the Jeopardy challenge again when you're looking up at the stars what are you most proud of looking back at those days I'm most proud of my um my commitment and my team's commitment to be true to the science to not be afraid to fail that's beautiful because there's so much pressure because it is a public event this is a public show that you were dedicated to the idea that's right do you think it was a success in the eyes of the world it was a success by your I'm sure exceptionally high standards is there something you regret you would do differently it was a success it was a success for our goal our goal was to build the most advanced open domain question-answering system we went back to the old problems that we used to try to solve and we did dramatically better on all of them as well as we beat jeopardy so we wanted jeopardy so it was it was a success it was I worried that the world would not understand that has success because it came down to only one game and I knew statistically speaking this can be a huge technical success and we could still lose that one game and that's a whole nother theme of this of the journey but it was a success it was not a success in natural language understanding but that was not the goal yeah that was but I would argue I understand what you're saying in terms of the science but I would argue that the inspiration of it right the they not a success in terms of solving natural language understanding there was a success of being an inspiration to future challenges absolutely drive future efforts what's the difference between how human being compete in jeopardy and how Watson does it that's important in terms of intelligence yeah so thats that actually came out very early on in the project also in fact I had people who wanted to be on the project who were early on who has sort of approached me once I committed to do it had wanted to think about how humans do it and they were you know from a cognition perspective like human cognition and how that should play and I would not take them on the project because another assumption or another stake I put in the ground was I don't really care are you into this at least in the context of this prior need to build in the context to this project in NLU and in building an AI that understands how it needs to alter that communicate with humans I very much care yeah so wasn't that I didn't care in general in fact as an AI scientist I care a lot about that but I'm also a practical engineer and I committed to getting this thing done and I wasn't gonna get distracted I had to kind of say look if I'm gonna get this done and when it charts this path and this path says we're gonna engineer a machine that's gonna get this thing done and we know what search and NLP can do we have to build on that foundation if I come in and take a different approach and start wondering about how the human mind might or might not do this I'm not going to get there from here in the time and you know in the timeframe I think that's a great way to lead the team but now there's done and then one when you look back right so analyse what's the difference sexy right so so I was a little bit surprised actually to discover over time as this would come up from time to time and would reflect on it that and and talking to Ken Jennings a little bit and hearing Ken Jennings talk about it about how he answered questions that it might have been closer to the way humans answer questions than I might have imagined previously because humans are probably in the game of Jeopardy at the level of Ken Jennings probably also cheating their weight into winning right now one else is shallow they're doing that fast as possible they're doing shallow analysis so they are very quickly analyzing the question and coming up with some you know key you know key vectors or cues if you will and they're taking those cue they're very quickly going through like their library of stuff not deeply reasoning about what's going on and then sort of like a lots of different like what we call these these scores which kind of score that in a very shallow way and then say oh boom you know that's what it is and and so it's interesting as we reflected on that so we may be doing something that's not too far off from the way humans do it but we certain certainly didn't approach it by saying you know how would you even do this now in an elemental cognition like the project I'm leading now we asked those questions all the time because ultimately we're trying to do something that is to make the the the intelligence in the machine and the intelligence of the human very compatible well compatible in the sense they can communicate with one another and they can reason with this shared understanding so how they think about things and how they build answers how they build explanations becomes a very important question to consider so what's the difference between this open domain but cold constructed question answering or jeopardy and more something that requires understanding for shared communication with humans and machines yeah well this goes back to the interpretation of what we were talking about before anyway jeopardy the systems on trying to interpret the question and that's not interpreting the content that's reasoning and with regard to any particular framework I mean it's it is parsing it and like parsing the contents and using grammatical cues and stuff like that so if you think of grammar as a human framework in some sense and as that but when you get into the richer semantic frameworks what are people how do they think what motivates them what are the events that are occurring and why are they occurring and what causes what else to happen and and and when it where are things in time and space and it's like when you started thinking about how humans formulate and structure the knowledge that they acquire in their head and wasn't doing any of that what do you think are the essential challenges of like free flowing communication free flowing log versus question-answering even with the framework of the interpretation dialogue yep do you see free-flowing dialogue as a fundamentally more difficult than question answering even with shared so dialogue is as important in number of different ways I mean it's a challenge so first of all when I think about the machine that when I think about a machine that understands language and ultimately can reason in an objective way that can take the information that it perceives through language or other means and connects it back to these frameworks reason and explain itself that system ultimately needs to be able to talk to humans or I needs to be able to interact with humans so in some sentence to dialogue that doesn't mean that it it that like sometimes people talk about dialogue and they think you know how do humans talk how do you montork talk to each other in a casual conversation then you could mimic casual conversations we're not trying to mimic casual conversations we're really trying to produce the machine as goal is it is to help you think and help you reason about your answers and explain why so instead of like talking to your friend down the street about having a smoke having a small talk conversation with your friend down the street this is more about like you would be communicating to the commuter computer on Star Trek we're like what do you want to think about like what do you want to reason about I'm going to tell you the information I have I'm gonna have to summarize it I'm gonna ask you questions you're gonna answer those questions I'm gonna go back and forth with you I'm gonna figure out what your mental model is I'm gonna I'm gonna now relate that to the information I have and present it to you in a way that you can understand it and we could ask follow-up questions so it's that type of dialogue that you want to construct it's more structured it's more goal oriented but it needs to be fluid in other words it can't it can't it has to be engaging and fluid it has to be productive and not distracting so there has to be a model of the words the machine has to have a model of how humans think through things and discuss them so basically a productive rich conversation unlike this part yes but what I'd like to think it's more similar to this pocket as in joking I'll ask you about humor as well actually but what's the hardest part of that because it seems we were quite far away as a community from thats though to be able to so one is having a shared understanding as i think a lot of the stuff you said with frameworks is quite brilliant but just creating a smooth discourse yeah it feels clunky right now well which aspects of this whole problem you just specified all having a productive conversation is the hardest and that were or maybe maybe any aspect of it you can comment on because it's so shrouded in mystery so I think do this you kind of have to be creative in the following sense if I were to do this is purely a machine learning approach and someone said learn how to have a good flue in structured knowledge acquisition conversation I'd go out and say okay I have to collect a bunch of data of people doing that people reasoning well having a good structured conversation that both acquires knowledge efficiently as well as produces answers and explanations as part of the process and you struggle I don't know elect a day to collect the data because I don't know how much data is like that I think okay okay so this one there's a human but also even if it's out there say was out there how do you like alright so I think I think like an accessible right so I think any like any problem like this where you don't have enough data to represent the phenomenon you want to learn in other words you want you if you have enough data you could potentially learn the pattern in an example like this it's hard to do it this is the you know Susie sort of a human sort of thing to do what you recently came out IBM was the debate or projects and surest thing right because now you had you do have these structured dialogues these debate things where they did use machine learning techniques to generate the you know generate these debates dialogues are a little bit tougher in my opinion than generating a a structured argument where you have lots of other structural arguments like this you could potentially annotate that data and you could say this is a good response a bad response in a particular domain here I have to be responsive and I have to be opportunistic with regard to what is the human saying what so I'm goal-oriented and saying I want to solve the problem I want to acquire the knowledge necessary but I also have to be opportunistic and responsive to what the human is saying so I think that it's not clear that we could just train on a body of data to do this but we could bootstrap it in other words we can be creative and we could say what do we think what do we think the structure of a good dialogue is that does this well and we can start to create that if we can if we can create that more programmatic programmatically at least to get this process started and I can create a tool that now engages humans effectively I could start both I could start generating data I could start with the human learning process and I can update my machine but I can also start the automatic learning process as well but I have to understand what features to even learn over so I have to bootstrap the process a little bit first and that's a creative design task that I could then use as input into a more automatic learning task this is some creativity and bootstrapping all right what elements of conversation do you think you would like to see so one of the benchmarks for me is humor right that seems to be one of the hardest if you end to me the biggest contrast is from Watson so one of the greatest sketches of comedy sketches of all time right is the SNL celebrity jeopardy with uh with with Alex Trebek and Sean Connery and Burt Reynolds and so on with uh with the Sean Connery commentating on Alex Trebek smile there a lot so and I think all of them are in the negative point what's why so they're clearly all losing in terms of the game of Jeopardy but they're winning in terms of comedy so what do you think about humor in this whole interaction in the dialogue that's productive or even just whatever what human represents to me is it the same idea that you're saying about a framework because humor only exists within a particular human framework so what do you think about humor what do you think about things like humor that connect to the kind of creativity you mentioned that's needed I think there's a couple things going on there so I I I sort of feel like and I might be too optimistic this way but I think that there are we did a little bit about with with this and with puns and in jeopardy we literally sat down and said well you know how do puns work and you know it's like wordplay and you could formalize these things so I think there's a lot aspects of humor that you could formalize you could also learn new Murr you could just say what do people laugh at and if you have enough again if you have enough data to represent the phenomenon you know might be able to you know weigh the features and figure out you know what humans find funny and what they don't find funny you might the Machine might not be able to explain why the my buddy unless we unless we sit back and think about that more formally I think again I think you do a combination of both and I'm always a big proponent that I think you know robust architectures and approaches are always a little bit combination of us reflecting and being creative about how things are structured and how to formalize them and then taking advantage of large data and doing learning and figuring how to combine these two approaches I think there's another aspect of human to human though which goes to the idea that I feel like I can relate to the person telling the story telling the person telling the story and I think that's that's a interesting theme in the whole AI theme which is do I feel differently when I know it's a robot and when I know when I imagine there's a row but is not conscious the way I'm conscious when they imagine the robot does not actually have the experiences that I experience do I find it you know funny or do because it's not as related I don't imagine that the person is relating it to it the way I relate to it I think this also you see this in in the arts and in entertainment where like you know sometimes you have savants who are remarkable at a thing whether it's sculpture it's music or whatever but the people who get the most attention are the people who can't who can evoke a similar emotional response who can get you to emote right about the way they in other words who can basically make the connection from the artifact from the music of the painting of the sculpture to the to the emotion and get you to share that emotion with them and then and that's when it becomes compelling so they're communicating at a whole different level they're just not communicating the artifact they're communicating their emotional response to the artifact and then you feel like oh wow I can relate to that person I can connect to that I can connect to that person so I think humor has that has that aspect as well so the idea that you can connect to that person person being the critical thing but we're also able to anthropomorphize objects pretty robots and AI systems pretty well so we're almost looking to make them human there may be from your experience with Watson maybe you can comment on did you consider that as part well obviously the problem of Jeopardy doesn't require int the promotoras ation but nevertheless well there was some interest in doing that and I've that's an that's another thing I didn't want to do so I didn't want to distract from the from the actual scientific test nights so you're absolutely right I mean humans do anthropomorphize and and without necessarily a lot of work I mean just put some eyes in a couple of eyebrow movements and you're getting humans to react emotionally and I and I think you can do that so I didn't mean to suggest that that that connection can't cannot be mimicked I think that connection can be mimicked and can get you to can produce that emotional response I just wonder though if you're told what's really going on if you know that the machine is not conscious not having the same richness of emotional reactions and understanding that doesn't really share the understanding but is essentially just moving inside brow or drooping its eyes or making them big or whatever it's doing that's getting the emotional response will you still feel it interesting I think you probably would for a while and then when it becomes more important that there's a deeper under depreciate understanding it may run flat but I don't know I'm pretty I'm pretty confident that it will the majority of the world even if you tell them how no matter well it will not matter especially if the Machine herself says that she is cautious that's very possible so you the scientists that made the machine is saying that this is how the algorithm works everybody will just assume you're lying and that there's a conscious being there so you're deep into the science fiction shop you're on right now but yeah I think it's actually psychology I think it's not science fiction I think it's reality I think it's a really powerful one that will have to be exploring in the next few decades it's a very interesting element of intelligence so what do you think we've talked about social constructs of intelligences and and frameworks and the way humans kind of interpret information what do you think is a good test of intelligence in your view so there's the Alan Turing with the Turing test Watson accomplished something very impressive with Jeopardy what do you think is a test that would impress the heck out of you that you saw that a computer could do they say this is crossing a kind of threshold that's that gives me pause in a good way expectations for a are generally high what does high look like by the way so not the threshold test as a threshold what do you think is the destination what do you think is the ceiling I think machines will in many measures will be better than us will become more effective in other words better predictors about a lot of things and then then then ultimately we can do I think where they're gonna struggle is what we talked about before which is relating to communicating with and understanding humans in deeper ways and and so I think that's a key point like we can create the super parrot what I mean by the super parrot is given enough data a machine can mimic your emotional response can even generate language that will sound smart and what someone else might say under similar circumstances look how its paws on that like that's a super parrot right so given similar circumstances moves its face its faces in similar ways changes its tone of voice in similar ways produce the strings of language that you know would similar that a human might say not necessarily being able to produce a logical interpretation or understanding that would ultimately satisfy a critical interrogation or a critical understanding I think you guys describe me in a nutshell I think I think philosophically speaking you could argue that that's all we're doing as human beings to war so I was gonna say it's very possible you know humans do behave that way too and so upon deeper probing and deeper interrogation you may find out that there isn't a shared understanding because I think humans do both like humans are statistical language model machines and and and they are capable reasoner's you know they're they're both and you don't know which is going on right so and I think it's I think it's an interesting problem we talked earlier about like where we are in our social and political landscape can you distinguish some who can string words together and sound like they know what they're talking about from someone who actually does can you do that without dialogue without integrity of a programming dialogue so it's interesting because humans are really good at in their own mind justifying or explaining what they hear because they project their understanding on onto yours so you could say you could put together a string of words and and someone will sit there and interpret in a way that's extremely biased this is the way they want to interpret it they want to assuming you're an idiot and they'll true put it one way they've all seen you're a genius and interpreted another way that suits their needs so this is tricky business so I think the answer your question as AI gets better and better at better and better mimic you we create the super parrots we're challenged just as we are with we're challenged with humans do you really know what you're talking about do you have a meaningful interpretation a powerful framework that you could reason over and justify your answers justify your predictions and your beliefs why you think they make sense can you convince me what the implications are you know can you so can you reason intelligently and make me believe that those um the implications of your prediction and so forth so what happens is it becomes reflective my standard for judging your intelligence depends a lot on mine but you're saying that there should be a large group of people with a certain standard of intelligence that would be convinced by this particular AI system then there should be by I think one of the depending on the content one of the problems we have there is that if that large community of people are not judgment judging it with regard to a rigorous standard of objective logic and reason you still have a problem like masses of people can be persuaded the Millennials yeah to turn them turn their brains off right okay sorry I have nothing against the warning I just so you you're a part of one of the great benchmarks challenges of AI history what do you think about alpha zero open AI five alpha star accomplishments on video games recently which are also I think at least in the case of go without fagala now for zero playing go was a monumental accomplishment as well what are your thoughts about that challenge I think it was a giant lamare I I think it was phenomenal I mean as one of those other things nobody thought like solving go was gonna be easy particularly because it's again it's hard for particularly hard for humans our team is to learn how for humans to excel at and so it was up another measure a measure of intelligence it's very cool I mean it's very interesting you know what they did I mean and I loved how they solved like the data problem which again they bootstrapped it and got the machine to play itself to generate enough data to learn from I think that was brilliant I think that was great and and and of course the result speaks for itself I think it makes us think about again it is okay what's intelligence what aspects of intelligence are important can the can the go machine help me make me a better go player is it an alien intelligence it was is am I even capable of like again if we if we put in very simple terms it found the function we found the go function can I even comprehend the go function can I talk about the go function can i conceptualize the go function like whatever it might be so one of the interesting ideas of that system is it plays against itself right yeah but there's no human in the loop there so like you're saying it could have by itself created an alien intelligence how torta torta gorrik imagine you're sentencing you're judging you're sentencing people or you're setting policy or you're you know you're making medical decisions and you can't explain you can't get anybody to understand what you're doing or why so it's it's it's an interesting dilemma for the applications of AI do we hold AI to this accountability that says you know humans have to be humans have to be able to take responsibility you know for for the decision in other words can you explain why you would do the thing well you will use get up and speak to other humans and convince them that this was a smart decision is the AI enabling you to do that can you get behind the logic that was made there do you think sorry to link on this point because it's a there's a fascinating one that's a great goal for AI do you think it's achievable in many cases or do you okay there's two possible worlds that we have in the future one is where AI systems do like medical diagnosis or things like that would drive a car without ever explaining to you why it fails when it does that's one possible world then we're okay with it or the other where we are not okay with it and we really hold back the technology from getting to good before it gets able to explain which of those worlds are more likely do you think and which are concerning to you or not I think the reality is it's gonna be a mix you know I'm not trying a problem with that I mean I think there are tasks that perfectly fine with machines show a certain level of performance and that level of performance is already better it is already better than humans so for example I don't know that I get tape driverless cars if driverless cars learn how to be more effective drivers than humans but can't explain what they're doing but bottom line statistically speaking there you know ten times safer than humans I I don't know that I care I think when we we have these edge cases when something bad happens and we want to decide who's liable for that thing and who made that mistake in what we do about that and I think in those those educators are interesting cases and now do we go to designers of the AI and the I says I don't know if that's what it learned to do and it says well you didn't train it properly you know you you were you were negligent in the training data that you gave that machine like how do we drive down and realize oh so I think those are I think those are interesting questions so the optimization problem there sorry is to create a system that's able to explain the lawyers away there you go um I think that uh uh I think it's gonna be interesting I mean I think this is where technology and social discourse are gonna get like deeply intertwined and how we start thinking about problems decisions and problems like that I think in other cases it becomes more obvious where you know it's I got like why did you decide to give that person you know a longer sentence or or to deny them parole again policy decisions or why did you pick that treatment like that treatment up killing that guy like why was that a reasonable choice to make so so and people are gonna demand explanations now there's a reality though here and the reality is that it's not I'm not sure humans are making reasonable choices when they do these things they are using statistical hunches biases or even systematically using statistical averages to make Osmonds is what happened my dad if you saw that target gave about that but you know I mean they decided that my father was brain dead he had went into cardiac arrest and it took a long time for the ambulance to get there and wasn't not resuscitated right away and so forth and they came they told me he was brain dead and why was he brain dead because essentially they gave me a purely statistical argument under these conditions with these four features 98% chance he's brain dead innocent but can you just tell me not inductively but deductively go there and tell me his brain stopped functioning is the way for you to do that and they and and their the the protocol in response was no this is how we make this decision I said this is adequate for me I understand the statistics and I don't have you know there's a two percent chance he's so like I just don't know the specifics I need the specifics of this case and I want the deductive logical argument about why you actually know he's brained it so I wouldn't sign that do not resuscitate and I don't know it was like they went through lots of procedures as a big long story but the bottom was a fascinating story by the way but how I reasoned and how the doctors reasoned through this whole process but I don't know somewhere around 24 hours later or something he was sitting up that would zero bushido brain damage any what lessons do you draw from that story that experience that the data that they're you that the data that's being used to make sophistical inferences doesn't adequately reflect the phenomenon so in other words you're getting shit Ramsar you're getting stuff wrong because you're your model is not robust enough and you might be better off not using statistical inference and statistical averages in certain cases when you know the models insufficient and that you should be reasoning at about the specific case more logically and more deductively and hold yourself responsible to hold yourself accountable to doing that and perhaps AI has a role to say the exact thing we just said which is perhaps this is a case you should think for yourself you should reason deductively so it's hard it's it's so it's hard because it's hard to know that you know you'd have to go back and you'd have to have enough data to essentially say and this goes back to how do we this goes back to the case of how do we decide whether the AI is good enough to do a particular task and regardless of whether or not it produces an explanation so um and and what standards do we hold right for that so um you know if you look at you you look more broadly for example as my father as a metal kick medical case the medical system ultimately helped him a lot throughout his life without it he probably would have died much sooner so overall sort of you know work for him and sort of a net in that kind of way actually I don't know that's fair um but it maybe not in that particular case but overall like oh the medical system overall that's more given a system overall you know was doing more more good than bad now is another argument that suggests that wasn't the case but for the for the sake of argument let's say like that's let's say a net positive and I think you have to sit there and there and take take that into consideration now you look at a particular use case like for example making this this decision have you done enough studies to know how good that prediction really is right and how you have you done enough studies to compare it to say well what if we what if we dug in and in a more direct you know let's get the evidence let's let's do the deductive thing and not use the statistics here how often would that have done better right you just so you have to do this studies to know how good the AI actually is and it's complicated because depends how fast you have to make decision so if you have to make the decision superfast do you have no choice right if you have more time right but if you're ready to pull the plug and this is a lot of the argument that I had was a doctor I said what's he gonna do if you do it what's gonna happen to him in that room if you do it my way you know if you do well he's gonna die anyway so let's do it my way though I mean it raises questions for our society to struggle with as was the case with your father but also when things like race and gender start coming into play when when certain when when judgments are made made based on things that are complicated in our society at least in this course and it starts you know I think I think I'm safe to say that most of the violent crimes committed by males so if you discriminate based you know as a male versus female saying that if it's a male more likely to commit the crime so this is one of my my very positive and optimistic view views of why the study of artificial intelligence the process of thinking and reasoning logically and statistically and how to combine them is so important for the discourse today because it's causing a regardless of what what state AI device devices are or not it's causing this dialogue to happen this is one of the most important dialogues that in my view the human species can have right now which is how to think well yeah how to reason well how to understand our own cognitive biases and what to do about them that has got to be one of the most important things we as as as a species can be doing honestly we are reached we've created an incredibly complex society we've created amazing abilities to amplify noise faster than we can play amplifies signal we are challenged we are deeply deeply challenged we have you know big segments of the population getting hit with enormous amounts of information do they know how to do critical thinking do they know how to objectively objectively reason do they understand what they are doing nevermind with their AI is doing this is such an important dialogue you know to be having and and and you know we are fundamentally are thinking can be and easily becomes fundamentally bias and there are statistics and we shouldn't blind our so we shouldn't discard statistical inference but we should understand the nature of such this conference as us as a society as you know we decided to reject statistical inference to favor individual understanding and and deciding on the individual yes we we consciously make that choice so even if the statistics said even if the Cystic said males are more likely to have you know to be violent criminals we still take each person as an individual and we treat them based on the logic and the knowledge of that situation we purposefully and intentionally reject the statistical once we do that at a respect for the individual for the individual yeah and then that requires reasoning and cracking looking forward what Grand Challenges would you like to see in the future because the the Jeopardy challenge you know captivated the world alpha go alpha zero cap day of the world deep blue certainly beating Kasparov Gary's bitterness aside and captivated the world what do you think do you have ideas for next grand challenges for future challenges of that oh you know I look I mean I think there are lots of really great ideas for Grand Challenges I'm particularly focused on one right now which is Kent you know can you demonstrate that they understand that they could read and understand that they can they can acquire these frameworks and communicate you know reason and communicate with humans so it is kind of like the Turing test but it's a little bit more demanding than the Turing test it's not enough it's not enough to convince me that you might be human because you could you know you can parrot a conversation I think you know the the this standard is a little bit higher is for example can you you know the santa is higher and I think one of the challenges of devising this grand challenge is that we're not sure what intelligence is we're not sure how to determine whether or not two people actually understand each other and then what depth they understand it they you know and what to what depth they understand each other so the challenge becomes something along the lines of can you satisfy me that we have a shared understanding so if I were to probe and probe and you probe me can can can can machines really act like thought partners where they could satisfy me that they that we have a share our understanding is shared enough that we can collaborate and produce the answers together and that you know they they can help me explain and justify those answers so maybe here's an idea so we'll have a Isis run for president and convinced that's too easy from sorry oh no you have to convince the voters that they should vote for it so they s what I would again again I that's why I think this is such a challenge because we go back to the emotional persuasion we go back to you know now we're checking off an aspect of human cognition that is in many ways weak or flawed right we're so easily manipulated our minds are drawn for often the wrong reasons right not the reasons that ultimately matter to us but the reasons that can easily persuade us I think we can be persuaded to believe one thing or another for reasons that ultimately don't serve us well in the long term and a good benchmark should not play with those elements of emotional manipulation I don't think so I think that's where we have to set the set the higher standard for ourselves of what you know what does it mean this goes back to rationality and it goes back to objective thinking and can you produce can you acquire information and produce reasoned arguments and to those reasons arguments pass a certain amount of muster and is it and can you acquire new knowledge you know can you can you under can you reason oh I have acquired new knowledge can you identify where it's consistent or contradictory with other things you've learned and can you explain that to me and get me to understand that so I think another way to think about it perhaps is kind of machine teach you can the hell really nice less than that's where to put it can you understand something that you didn't really understand before where's where is you know it's taking it so you're not you know again it's almost like can it can it teach you can it help you learn and and in an arbitrary space so it can open those domain space so can you tell the Machine and again this borrows from some science fiction's abut can you go off and learn about this topic that I'd like to understand better and then work with me to help me understand it that's quite brilliant what the machine that passes that kind of test do you think it would need to have self-awareness or even consciousness what do you think about consciousness and the importance of it maybe in relation to having a body having a presence an entity do you think that's important you know people used to ask if Watson was conscious and I used to think and he said he's the conscious of what exactly I mean I think you know main cell it depends what it is that you're conscious I mean like so you know did it if you you know it's certainly easy for it to answer questions about it would be trivial to program it so the answer questions about whether or not it was playing jeopardy I mean it could certainly answer questions that will imply that it was aware of things exactly what does it mean to be aware and what does it mean to conscious and it's sort of interesting I mean I think that we differ from one another based on what we're conscious of but wait wait for sure there's degrees of consciousness in there so it well in those areas like it's not just agrees what do you what do you what are you aware of like what are you not aware but nevertheless there's a very subjective element to our experience let me even not talk about consciousness let me talk about another to me really interesting topic immortality fear or mortality Watson as far as I could tell did not have a fear of death certainly not most most humans do wasn't conscious of death it wasn't that so there's an element of finiteness to our existence that I think like we like I mentioned survival that adds to the whole thing that I mean consciousness is tied up with that that we are us thing it's a subjective thing that ends and that seems to add a color and flavor to our motivations in a way that seems to be fundamentally important for intelligence or at least the kind of human intelligence well I take for generating goals again I think you could have you could have an intelligence capability and a capability to learn I capability to predict but I think without I mean again you get a fear but essentially without the goal to survive so you think you can just encode that without having to million code I mean can you create a robot now and you could say you know and plug it in and say protect your power source you know and give it some capabilities and we'll sit there and operate to try to protect this power source and survive I mean I so I don't know that that's false awfully a hard thing to demonstrate it sounds like a fairly easy thing to demonstrate that you can give it that goal we'll come up with that goal by itself as you have to program that goal in but there's something because I think as we touched on intelligence is kind of like a social construct the the fact that a robot will be protecting its power source would would add depth and grounding to its intelligence in terms of us being able to respect I mean ultimately it boils down to us acknowledging that it's intelligent and the fact that it can die I think is an important part of that the interesting thing to reflect on is how trivial that would be and and I don't think if you knew how trivial that was you would associate that with being intelligence I mean I literally put in a statement of code that says you know you have the following actions you can take you give it a bunch of actions like you mount a laser gun on her or you may do you the ability to scream a screech or whatever and you know and you you say you know if you see your power source then you could program that in and you know you're gonna print it you're gonna take these actions to protect it you know you teach it checking it on a bunch of things so and and now you're gonna look at that and you say well you know that's intelligence because it's protecting power source maybe but that's again at this human bias that says the thing I had then I identify my intelligence and my conscious so fundamentally with the desire or at least the behaviors associated with the desire to survive that if I see another thing doing that I'm going to assume it's intelligent what timeline year will society have a something that would that you would be comfortable calling an artificial general intelligence system well what's your intuition nobody can predict the future certainly not next few months or twenty years away but what's your intuition how far away are we I the ideas hearts make these predictions and I would be you know I would be guessing and there's so many different variables including just how much we want to invest in it and how important it you know and how important we think it is what kind of investment are willing to make in it what kind of talent we end up bringing to the table all you know the incentive structure all these things so I think it is possible to do this sort of thing I think it's I think trying to sort of ignore many of the variables and things like that is it a ten-year thing as a 23 it's probably closer to a 20-year thing I guess but not as little no I don't think it's several hundred years I don't think it's several hundred years but again so much depends on how committed we are to investing and incentivizing this type of work this type of work and it's sort of interesting like I don't think it's obvious how incentivize we are I think from a task perspective you know if we see business opportunities to take this technique is a technique to solve that problem I think that's the main driver for many from any of these things from a from a general Tosta seems kind of an interesting question are we really motivated to do that and and like we just struggled ourselves right now to even define what it is so it's hard to incentivize when we don't even know what it is we're incentivized to create and if you said mimic a human intelligence I just think there are so many challenges with the the significance and meaning of that there's not a clear directive there's no clear directive to do precisely that thing so assistance in a larger and larger number of tasks so being able to a system that's particularly able to operate my microwave and making a grilled cheese sandwich I don't even know how to make one of those and then the same system would be doing the vacuum cleaning and then the same system would be teaching my kids that I don't have math I think that when when when you get into a general intelligence for learning physical tasks and again yeah I want to go back to your body questions it's on your body question was interesting but you want to go back to you know learning abilities do physical tasks you might have we might get Majan in that timeframe we will get better and better at learning these kinds of tasks whether it's mowing your lawn or driving a car or whatever it is I think we will get better and better at that where it's learning how to make predictions over large bodies of data as if we're going to continue to get better and better at that and machines will out you know outpace humans and and a variety of those things the underlying mechanisms for doing that may be the same meaning that you know maybe these are deep Nats there's infrastructure to train them reusable components to get them to different classes of tasks and we get better and better at building these kinds of machines you could see argue that the general learning infrastructure in there is a form of a general type of intelligence I think what starts getting harder is this notion of you know can we can we effectively communicate and understand and build that shared understanding because of the layers of interpretation that are required to do that and the need for the machine to be engaged with humans at that level at a continuous basis so how do you get in how do you get the machine in the game how do you get the machine in the intellectual game yeah and to solve AGI you probably have to solve that problem you have to get the machine so it's a little bit of a bootstrapping can we get the machine engaged and you know in the intellectual calling a game but in the intellectual dialogue with the humans are the humans sufficiently an intellectual dialogue with each other to generate enough to generate enough data in this context and how do you bootstrap that because every one of those conversations every one of those conversations those intelligent interactions require so much prior knowledge that is a challenge to bootstrap it so that's so as so the question is and how committed so I think that's possible but when I go back to are we incentivized to do that I know we're incentivized to do the former are we incentivize to do the latter significantly enough to people understand what the latter really is well enough part of the elemental cognition mission is to try to articulate that better and better you know through demonstrations and to trying to craft these grand challenges and get people to say look this is a class of intelligence this is a class of AI do we do we want this what what is the potential of this what are the business what's the business potential what's the societal potential to that and so you know and to build up that incentive system around that yeah I think if people don't understand yet I think they will and is a huge business potential here so it's exciting that you're working on it you've kind of skipped over but I'm a huge fan of physical presence of things do you think you know Watson head of body do you think having a body as to the interactive element between the AI system and a human or just in general to intelligence so I think I think going back to that shared understanding bit humans are very connected to their bodies I mean is one of the reasons one of the challenges in getting an AI to kind of be a compatible human intelligence is that our physical bodies are generating a lot of features that make up the input so in other words where our bodies are are the the tool we use to affect output but they're also but they also generate a lot of input for our brains so we generate emotion we generate all these feelings we generate all these signals that machines don't have so missions that have this as the input data and they don't have the the feedback that says okay I've gotten this I've gotten this emotion or I've gotten this idea I now want to process that and then I can it then affects me as a physical being and then I and I and I can play that out in other words I could realize the implications of tax implications again on my bond mind body complex I then process that and the implications again are internal features are generated I learned from them they have an effect on my mind body complex so it's interesting when we think do we want a human intelligence well if we want a human compatible intelligence probably the best thing to do is to embed it embedded in a human body just to clarify and both concepts beautiful is humanoid robots so robots that look like humans is one or did you mean actually sort of what Hamas was working with neural link really embedding intelligence systems that the ride-alongs human bodies know I was riding along is different I meant like if you want to create an intelligence that is human compatible meaning that it can learn and develop a shared understanding of the world around it you have to give it a lot of the same substrate part of that substrate you know is the idea that it generates these kinds of internal features like sort of emotional stuff it has similar senses it has to do a lot of the same things with those same sentences um right so I think if you want that again I don't know that you want that like man like that's not my specific goal I think that's a fascinating scientific goal I think it has all kinds of other implications that's sort of not to go like I want it I want to create I think of it as I create intellectual thought martyrs for humans so that kind of that kind of intelligence I know other companies that are creating physical thought partners the fiscal partners to figure out for you but that's kind of not where we're you know I'm at but but but the the important point is that a big part of how of what we process is that physical experience of the world around us on the point of thought partners what role does an emotional connection or forgive me love have to play in that thought partnership is that something you're interested in put another way sort of having a deep connection beyond intellectual with the AI yeah with the a between human and ass is that something that gets in the way of the the rational discourse is there something that's useful I worry about biases you know obviously so in other words if you develop an emotional relationship with the machines do all of a sudden you start are more likely to believe what it's saying even if it doesn't make any sense so I you know I worry about that but at the same time I think the opportunity to use machines to provide human companionship is actually not crazy and it's again the intellectual and social companionship is not crazy the idea do you have concerns as a few people do you know Musk Sam Harris about long-term existential threats of AI and perhaps short-term threats of AI we talked about bias we talked about different misuses but do you have concerns about thought partners systems that are able to help us make decisions together humans somehow having a significant negative impact on society in the long term I think there aren't things to worry about I think the giving machines too much leverage is a problem and what I mean by leverage is is too much control for things that can hurt us whether it's socially psychological intellectually or physically and if you give them machines too much control I think that's a concern you forget about the AI just when you give them too much control human bad actors can hack them and produce havoc so um you know that's a problem and you imagine hackers taking over the driverless car Network and you know creating all kinds of havoc but you could also imagine given given the ease at which humans could be persuaded one way or the other and now we have algorithms that can easily take control over over that over that and amplify noise and move people one direction or another I mean humans do that to other humans all the time and we have marketing campaigns we have political campaigns that take it to image of our our emotions or our fears and this is done all the time when but with machines machines are like giant mecha phones right we can amplify this and orders of magnitude and can fine-tune its control so we can tailor the message we can now very rapidly and efficiently tailor the message to the audience taking taking advantage of you know of their biases and amplifying them and using them to pursue a them in one direction or another in ways that are not fair not logical not objective not meaningful and humans the machines and power that so so that's what I mean by leverage like it's not new but wow it's powerful because machines can do it more effectively more more you know more quickly and we see that already going on and and and social media not the plays and other places that's scary and and that's why like I'm I'm that's why I go back to saying one of the most important public dialogues we could be having is about the nature of intelligence and the nature of inference and logic and reason and rationality and us understanding our own biases us understanding our own cognitive biases and how they work and then how machines work and how do we use them to complement and sit basically so that in the end we have a stronger overall system that's just incredibly important I don't most people understand that so so like telling telling your kids or telling your students this goes back to the cognition here's how your brain works here's how easy it is to trick your brain right there are fundamental cognitive but you should appreciate the different the different types of thinking and how they work and what you're prone to and you know and what and what do you prefer and under what conditions does this make sense versus that makes sense and then say here's what AI can do here's how it can make this worse and here's how it can make this better and then that's where the as a role is to reveal that then the that trade-off so if you imagine a system that is able to beyond any definition of the Turing test of the benchmark really an AGI system as a thought partner that you one day will create what question what topic of discussion if you get to pick one would you have with that system what would you ask and you get to find out the truth together so you threw me a little bit with finding the truth at the end but this is a whole nother topic but the I think the beauty of it I think what excites me is the beauty of it is if I really have that system I don't have to pick so in other words I can you know I can go to and say this is where I care about today and and and that's what we mean by like this general capability go out read this stuff in the next three milliseconds and I want to talk to you about it I want to draw analogies I want to understand how this affects this decision or that decision what if this were true what if that were true what what knowledge should I be aware of that could impact my decision here's what I'm thinking is the main implication can you find can you prove that out can you give me the evidence that supports that can you give me evidence supports this oh there's a boy that would that be incredible you would that be just incredible just a long discourse just to be part of whether it's a medical diagnosis or whether it's you know the various treatment options or whether it's a legal case or whether it's a social problem that people are discussing like be part of the dialogue one that holds itself and us accountable to reasons an objective dialogue you know I just I get goosebumps talking about it right so when when you create it please come back on the podcast well the discussion together and make it even longer this is a record for the longest conversation now there's an honor it was a pleasure David thank you so much for thanks so much a lot of fun you