Sertac Karaman: Robots That Fly and Robots That Drive | Lex Fridman Podcast #97
the following is a conversation with Suresh Carmen a professor at MIT co-founder of the autonomous vehicle company optimist ride and is one of the top roboticists in the world including robots that drive and robots that fly to me personally he has been a mentor a colleague and a friend he's one of the smartest most generous people I know so it was a pleasure and honor to finally sit down with him for this recorded conversation this is the artificial intelligence podcast if you enjoy it subscribe I knew to review five stars in Apple podcast supported on patreon or simply connect with me on Twitter and Lex Friedman spelled Fri D ma n as usual I'll do a few minutes of ads now and never any ads in the middle that can break the flow of the conversation I hope that works for you it doesn't hurt the listening experience this show is presented by cash app the number one finance app in the App Store when you get it use the code lex podcast cash app lets you send money to friends buy Bitcoin and invest in the stock market with as little as $1 this cash app allows you to send and receive money digitally let me mention a surprising fact about physical money it costs 2.4 cents to produce a single penny in fact I think it costs 85 million dollars annually to produce them that's a crazy little fact about physical money so again if you get cash out from the App Store Google Play and use the collects podcast you get $10 and cash Apple also donate $10 the first an organization that is helping to advanced robotics and STEM education for young people around the world and now here's my conversation with Sir - Carmen since you have worked extensively on both what is the more difficult task autonomous flying or autonomous driving that's a good question I think that autonomous flying just kind of doing it for consumer drones and so on the kinds of applications that we're looking at right now is probably easier and so I think that that's maybe one of the reasons why it took off like literally a little earlier than the autonomous cars but I think if we look ahead I would think that you know the real benefits of autonomous flying unleashing them in like transportation logistics and so on I think it's a lot harder than autonomous driving so I think my guess is that you know we've seen a few kind of machines fly here and there but we really haven't yet seen any kind of you know machine like like at massive scale large-scale being deployed and flown and so on and I think that's gonna be after we kind of resolve some of the large scale deployments of autonomous driving it was the hard part what's your intuition behind why at scale when consumer-facing drones are tough so I think in general its scale is tough like for example I mean you think about it we have actually deployed a lot of robots in the let's say the past 50 years we academics or we business I think we as humanity deployed a lot of robots and I think that when you think about it you know robots they're autonomous they work and they work on their own but they are either like in isolated environments or they are in sort of you know they may be at scale but they're really confined to a certain environment that they don't interact so much with humans and so you know they work in I don't know factory floors their houses they work on Mars you know they are fully autonomous over there but I think that the real challenge of our time is to take these vehicles and put them into places where humans are present so now I know that there's a lot of like human robot interaction type of things that need to be done and so on that's that's one thing but even just from the fundamental algorithms and systems and and the business cases or maybe the business models even like architecture planning societal issues legally there's a whole bunch of pack of things that are related to us putting robotic vehicles into human present environments and these humans you know they will not potentially be even trained to interact with them they may not even be using the services that are provided by these vehicles they may not even know that they're autonomous they're just doing their thing living in environments that are designed for humans not for robots and that I think is one of the biggest challenges I think over our time to put vehicles there and you know to go back to your question I think doing that at scale meaning you know you go out in a city and you have you know like thousands or tens of thousands of autonomous vehicles that are going around it is so dance to the point where if you see one of them you look around you see another one it is that dance and that density we've never done anything like that before and I would I would bet that that kind of density will first happen with autonomous cars because I think you know we can ban the environment a little bit we can especially kind of making them safe is a lot easier when they're like on the ground when they're in the air it's a little bit more complicated but I don't see that there's gonna be a big separation I think that you know there will come a time that we're gonna quickly see these things unfold do you think there will be a time where there's tens of thousands of delivery drones they fill this guy you know I think I think it's possible to be honest delivery drones is one thing but you know you can imagine for transportation like a like an important use cases but you know we're in Boston you want to go from Boston to New York and you want to do it from the top of this building to the top of another building in Manhattan and you're gonna do it in one and a half hours and that's that's a big opportunity I think personal transport so like you and your friend like oh yeah or almost like I like like an uber so like four people six people a people in our work in autonomous vehicles I see that so there's kind of like a bit of a need for you know one person transport but also like like a few people so you and I could take the trip together we could have lunch that you know I think kind of sounds crazy maybe even sounds a bit cheesy but I think that those kinds of things are some of the real opportunities and I think you know it's not like the typical airplane and the airport would disappear very quickly but I would think that you know many people would feel like they would spend an extra hundred dollars on doing that and cutting that for our travel down to one and a half hours so how feasible are flying cars it's been the dream that's like when people imagine the future for 50 plus years they think fine cars it's a it's like all technology is just cheesy to think about now because it seems so far away but overnight it can change but just technically speaking in your view how feasible is it to make that happen I'll get to that question but just one thing is that I think you know sometimes we think about what's gonna happen in the next 50 years it's just really hard to guess right next 50 years I don't know I mean we could yet what's gonna happen in transportation in the next 50 we could get flying saucers I I could bet on that I think there's a 50/50 chance that you know like you can build machines that can ionize the air around them and push it down with magnets and they would fly like a flying saucer that is possible and it might happen in the next 50 years so it's a bit hard to guess like when you think about 50 years before but I would think that you know there's this this kind of notion where there's a certain type of air space that we call the agile airspace and there is there's good amount of opportunities in that airspace so that would be the space that is kind of a little bit higher than the place where you can throw a stone because that's a tough thing when you think about it you know it takes a kid on a stone to take an aircraft down and then what happens but you know imagine the airspace that's high enough so that you cannot throw a stone but it is low enough that you're not interacting with the with the very large aircraft that are you know flying several thousand feet above and that airspace is underutilized or it's actually kind of not utilized at all yeah that's right so there's you know there's like recreational people kind of fly every now and then but it's very few if you look up in the sky you may not see any of them at any given time every night now you'll see one airplane utilizing that space and you'll be surprised and the moment you're outside of an airport a little bit like it's just kind of flies off and it goes out and I think utilizing that airspace the technical challenge is there is you know building an autonomy and ensuring that that kind of autonomy is safe ultimately I think it is going to be building in complex software are complicated so that it's maybe a few orders of magnitude more complicated than what we have on aircraft today and at the same time ensuring just like we ensure on aircraft ensuring that it's safe and so that becomes like building that kind of complicated hardware and a software becomes a challenge especially when you know you build that hardware I mean you build that software with data and so you know it's it's of course there's some rule by software in there that kind of do a certain set of things but but then you know there's a lot of training merit machine learning will be key to these guys to delivering safe vehicles in the future especially not maybe the safe part but I think the intelligent part um I mean there are certain things that we do it with machine learning and it's just there's like right now all the way and and I don't I don't know how else they could be done and you know there's there's always this conundrum I mean we could I could be like we could maybe gather billions of programmers humans who program perception algorithms that detect things in the sky and whatever or you know we I don't know we maybe even have robots like learning a simulation environment and transfer and they might be learning a lot better in a simulation environment than a billion humans put their brains together and try to program humans pretty limited what's uh what's the role of simulations withdrawals if you've done quite a bit of work there how promising just the very thing you said just now how promising is the possibility of training and developing a safe flying robot in simulation and deploying it and having that work pretty well in the real world I think that you know a lot of people when they hear simulation they will focus on training immediately but I think one thing that you said which was interesting it's developing I think simulation environments are actually could be key and great for development and that's not new like for example you know there's people in the automotive industry have been using dynamic simulation for like decades now and and it's pretty standard that you know you would build and you would simulate if you want to build an embedded controller you plug that kind of embedded computer into another computer that other computer would simulate tiny and so on and I think you know fast forward these things you can create pretty crazy simulation environments like for instance one of the things that has happened recently and that you know we can do now is that we can simulate cameras a lot better than we used to simulate them we were able to simulate them before and that's I think we just hit the elbow on that kind of improvement I would imagine that really improvements in hardware especially and with improvements and machine learning I think that we would get to a point where we can simulate cameras very very much similar cameras means simulate how a real camera would see the real world therefore you can explore the limitations of that you can train perception algorithms on the in simulation all that kind of stuff exactly so you know it's it has been easier to simulate what we will called interoceptive sensors like internal sensors so for example inertial sensing has been easy to simulate it has also been easily simulate dynamics like like physics that are governed by ordinary differential equations I mean like how a car goes around maybe have it rolls on the road how they interact with it interacts with the road or even an aircraft flying around like the dynamic the physics of that what has been really hard has been to simulate extra Sept of sensors sensors that kind of like look out from the vehicle and that's a new thing that's coming like laser rangefinders they're a little bit easier cameras radars are a little bit tougher I think once we nail that down the the next challenge I think in simulation will be to simulate human behavior that's also extremely hard even when you imagine like how a human driven car would act around even that is hard but imagine trying to simulate you know a a model of a human just doing a bunch of gestures and so on and and you know it's it's actually simulated it's not captured like with a motion capture but it is similarly that's that's very in fact today I get involved a lot with like sort of this kind of very high-end rendering projects and I have like this test that I've pass it to my friends or my mom you know ice and like two photos two kind of pictures and I say rendered which one is rendered which one is real and it's pretty hard to distinguish except I realized except when we put humans in there it's possible that our brains are trained in a way that we recognize humans extremely well but we don't so much recognize the built environments because built an alarm sort of came after per se we evolved into sort of being humans but but humans were always there same thing happens for example you look at like monkeys and you can't distinguish one from another but they sort of do and it's very possible that they look at humans it's kind of pretty hard to distinguish one from another but we do and so our eyes are pretty well trained to look at humans and understand if something is off we will get it we may not be able to pinpoint it so in my typical friend test or mom test what would happen is that we put like a human walking in you know anything and they they say you know this is not right something is off in this video I don't know what but I can tell you it's too human I can take the human and I can show you like inside of a building or like an apartment and it will look like if we had time to render it it will look great and this should be no surprise a lot of movies that people are watching it's all computer-generated you know even nowadays when you watch a drama movie and like there's nothing going on action wise but it turns out it's kinda like cheaper I guess to render the background and so they would but how do we get there how do we get a human would pass the mom / friend test a simulation of a human walking do you think that's something we can creep up do by just doing kind of a comparison learning or you have humans annotate what's more realistic and not just by watching they go what what's the path is it seems totally mysterious how we thing right simulate human behavior it's it's hard because a lot of the other things that I mentioned to you including simulating cameras right it is the the thing there is that you know we know the physics we know how it works like in the real world and we can write some rules and we can do that like for example simulating cameras there's this thing called ray tracing I mean you literally just kind of imagine it's very similar to it's not exactly the same but it's very similar to tracing photon by photon they're going around bouncing on things and coming your eye a human behavior developing a dynamic like like like a model of that that is mathematical so that you can put it into a processor that would go through that that's gonna be hard and so so what else do you got you can collect data right and you can try to match the data or another thing that you can do is that you know you can show the Frant test you know you can say this or that and this or that and that will be labeling anything that requires human labeling ultimately we're limited by the number of humans that you know we have available at a heart disposal and the things that they can do you know they have to do a lot of other things than also labeling this data so so that modeling human behavior part is is I think going we're gonna realize it's very tough and I think that also effects you know our development of autonomous vehicles I see them self-driving in smile like you want to use so you're building self-driving you know it the first time like right after urban challenge I think everybody focused on localization mapping and localization you know as slam algorithms came in Google was just doing that and so building these HD maps basically that's about knowing where you are and then five years later in 2012-2013 came the kind of coding code AI revolution and that started telling us about everybody else's but we're still missing what everybody else is gonna do next and so you want to know where you are you want to know what everybody else is hopefully you know about what you're gonna do next and then you want to predict what other people are going to do and that last bit has been a real real challenge what do you think is the role your own of you of your the ego vehicle the robot you the the you the robotic you in controlling and having some control of how the future on roles of what's going to happen in the future that seems to be a little bit ignored in trying to predict the future is how you yourself can affect that future by being either aggressive or less aggressive or signaling in some kind of way so this kind of game theoretic dance seems to be ignored for the moment it's yeah it's it's totally ignored I mean it's it's quite interesting actually like how we how we interact with things versus we interact with humans like so if if you see a vehicle that's completely empty and it's trying to do something all of a sudden it becomes a thing so interacted with like you interact with this table and so you can throw your backpack or you can kick your kick it put your feet on it and things like that but when it's a human there's all kinds of ways of interacting with a human so if you know like you and I are face to face we're very civil you know we talk understand each other for the most part you'll see you just but but the thing is that like for example you and I might interact through YouTube comments and you know the conversation may go a totally different angle and so I think people kind of abusing these autonomous vehicles is a real issue in some sense and so when you're an ego vehicle you're trying to you know coordinate your way make your way it's actually kind of harder than being a human you know it's like it's you you you not only need to be as smart as kind of humans are but you also you're a thing so they're gonna abuse you a little bit so you need to make sure that you can get around and do something so yeah III in general believe in that sort of game theoretic aspects I've actually personally have done you know quite a few papers both on that kind of game theory and also like this this kind of understanding people's social value orientation for example you know some people are aggressive some people not so much and and you know like you robot could understand that by just looking at how people drive and as they kind of come and approach you can actually understand like if someone is gonna be aggressive or or not as a robot and you can make certain decisions well in terms of predicting what they're going to do the hard question is you as a robot should you be aggressive or not when faced with it was an aggressive role but right now it seems like aggressive is a very dangerous thing to do because it's costly from a societal perspective how you're perceived people are not very accepting of aggressive robots emotive Society I think that's accurate so that is really is and so I'm not entirely sure like how to have to go about but it I know I know for a fact that how these robots interact with other people in there is going to be and then interaction is always gonna be there I mean you could be interacting with other vehicles or other just people kind of like walking around and like I said the moment there's like nobody in the seat it's like an empty thing just rolling off the street it becomes like no different than like any other thing that's not human and so so people and maybe abuse is the wrong word but you know people may be rightfully even they feel like you know this is a human present environments designed for humans to be and and they they kind of they want to own it and then you know the robots they would they would need to understand and they would need to respond in a certain way and I think that you know this actually opens up like quite a few interesting societal questions for us as we deploy like we talk robots at large scale so what would happen when we try to deploy robots at large scale I think is that we can design systems in a way that they're very efficient or we can design them that they're very sustainable but ultimately the sustainability efficiency trade-offs like they're gonna be right in there we're gonna have to make some choices like we're not going to be able to just kind of put it aside so for example we can be very aggressive and we can reduce transportation delays increase capacity of transportation or you know we can we can be a lot nicer and other people to kind of coding code on the environment and live in a nice place and then efficiency will drop so when you think about it I think sustainability gets attached to energy consumption or I wanna have the impact immediately and those are those are there but like livability is another sustainability impact so you create an environment that people want to live in and if robots are going around being aggressive and you don't want to live in that environment maybe however you should note that if you're not being aggressive then you know you're probably taking up some some delays in transportation and listen that so you're always balancing that and I think this this choice has always been there in transportation but I think the more autonomy comes in the more explicit the choice becomes yeah when it becomes explicit that we can start to optimize it and I will get to ask the very difficult societal questions of what do we value more efficiency or sustainability it's kind of interesting there will happen like I think we're gonna have to like I think that the the interesting thing about like the whole autonomous vehicles question I think is also kind of I think a lot of times you know we have we have focused on technology development like hundreds of years and you know the products somehow followed and and then you know we got to make these choices and things like that but this is this is a good time that you know we even think about you know autonomous taxi type of deployments and the systems that would evolve from there and you realize the business models are different the impact on architecture is different urban planning you get into like regulations and then you get into like these issues that you didn't think about before but like sustainability and ethics is like right in the middle of it I mean even testing autonomous vehicles like think about it you're testing autonomous vehicles in human present environments I mean the risk may be very small but still you know it's it's a it's a it's it's a you know strictly greater than zero risk that you're putting peep and so then you have that innovation you know risk trade-off that you're in that somewhere and we understand that pretty now they pretty well now is that if we don't test the beast day the development will be slower I mean it it doesn't mean that we're not gonna be able to develop I think it's gonna be pretty hard actually maybe we can we don't real I don't know but well the thing is that those kinds of trade-offs we already are making and as these systems become more ubiquitous I think those trade-offs will just really hit so you are one of the founders of optimist ride and town vehicle company and we'll talk about it well let me and that point ask maybe good examples keeping optimist right out of this question sort of exemplars of different strategies on the spectrum of innovation and safety or caution so you dick way Moe Google self-driving car way Moe represents maybe a more cautious approach and then you have Tesla on the other side added by Elon Musk that represents some more however which adjectives you want to use aggressive innovative I don't know but what what do you think about the difference between its two strategies in your view what's more likely what's needed and is more likely to succeed in the short term in the long term definitely some sort of balance is is kind of the right way to go but I do think that the thing that is the most important is actually like an informed public so I don't I don't mind you know I personally like if I were in some place I wouldn't mind so much like taking a certain amount of risk some other people might and so I think the key is for people to be informed and so that they can ideally they can make a choice in some cases that kind of choice making that anonymously is of course very hard but I don't think it's actually that hard to inform people so I think in in in one case like for example even the Tesla approach I don't know it's hard to judge how he informed it is but it is somewhat informed I mean you know things kind of come out I think people know what they're taking and things like that and so on but I think the underlying I do think that these two companies are a little bit kind of representing like babe of course that you know one of them seems a bit safer the other one or you know whatever the objective for that is and the other one seems more aggressive or whatever the ejector for that is but but I think you know when you turn the tables they're actually they're two other orthogonal dimensions that these two are focusing on on the one hand for remo I can see that you know they're I mean they I think they're a little bit see it as research as well so they kind of they don't I'm not sure if they're like really interested in like an immediate product you know they talk about it sometimes there's some pressure to talk about it so they kind of go for it but I think I think that they're thinking maybe in the back of their minds maybe they don't put it this way but I think they they realize that we're building like a new engine it's kind of like call it the AI engine or whatever that is and you know an autonomous vehicles is a very interesting embodiment of that engine that allows you to understand where the ego vehicle is the ego thing is where everything else is what everything else is gonna do and how do you react how do you actually you know interact with humans the right way how do you build these systems and I think they want to know that they want to understand that and so they keep going and doing that and so on the other dimension Tesla is doing something interesting I mean I think that they have a good product people use it think that you know like it's not for me but I can totally see people people like it and and people I think they have a good product outside of automation but I was just referring to the the automation itself I mean you know like it kind of drives itself you still have to be kind of you still have to pay attention to it right you know people seem to use it so it works for something and so people I think people are willing to pay for it people are willing to buy it I think it it's it's one of the other reasons why people buy a Tesla car maybe one of those reasons is Elon Musk is the CEO and you know he seems like a visionary person that's what people think you know it seems like a visionary person and so it adds like 5k to the value of the car and then maybe another 5k is the autopilot and and you know it's it's useful I mean it's useful in the sense that like people are using it and so III can see Tesla and sure of course they want to be visionary they want to kind of put out a certain approach and they may actually get there but I think that there's also a primary benefit of doing all these updates and rolling it out because you know people pay for it and it's it's your home it's basic you know demand supply market and people like it they're happy to pay another 5k 10k for that novelty or whatever that is they and they use it it's not like they get it and they try it a couple times it's a novelty but they use it a lot of the time and so I think that's what Tesla is doing it's actually pretty different like they are on pretty orthogonal dimensions of what kind of things that they're building they are using the same AI engine so it's very possible that you know they're both gonna be sort of one day kind of using a similar almost like an internal internal combustion engine it's a very bad metaphor but similar internal combustion engine and maybe one of them is building like a car the other one is building a truck or something so ultimately the use case is very different so you like I said or one of the founders of Optimus rad let's take a step back it's one of the success stories in the autonomous vehicle space it's a great attack vehicle company let's go from the very beginning what does it take to start autonomous vehicle company how do you go from idea to deploying vehicles like you are and a few a bunch of places including New York I would say that I think that you know what happened to us is it was was the following I think we've realized a lot of kind of talk in the autonomous vehicle industry back in like 2014 even when we wanted to kind of get started and I don't know like I kind of I would hear things like fully autonomous vehicles two years from now three years from now I kind of never bought it you know I was a part of MIT zorbing Channel Gentry it kind of like it has an interesting history so I did in college and in high school sort of a lot of mathematically oriented work and I think I kind of you know at some point it kind of hit me I wanted to build something and so I came to MIT mechanical engineering program and I now realize I think my advisor hired me because I could do like really good math but I told him that no no no I want to work on that urban challenge car I want to build the autonomous car and I think that was that was kind of like a process why we really learned I mean what the challenges are and and what kind of limitations are we up against you know like having the limitations of computers or understanding human behavior there's so many of these things and I think it's just kind of didn't and so so we said hey you know like why don't we take a more like a market-based approach so we focus on a certain kind of market and we build a system for that what we're building is not so much of like an autonomous vehicle only I would say so we build full autonomy into the vehicles but you know the way we kind of see it is that we think that the approach should actually involve humans operating that not just just not sitting in the vehicle and I think today what we have is today we have one person operate one vehicle no matter what that vehicle it could be a forklift it could be a truck it could be a car whatever that is and we want to go from that to ten people operate 50 vehicles how do we do that you're referring to a world of maybe perhaps teleoperation so can you just say what it means for 10 might be confusing for people listening what does it mean for ten people to control 50 vehicles that's a good point so I think it's am I very deliberately didn't call it a law operation because people what people think then is that people think away from the vehicle sits a person sees like maybe put some goggles or something ER and drives the car so that's not at all what we need but we mean the kind of intelligence bye-bye humans are in control except in certain places the vehicles can execute on their own and so imagine like like a room where people can see what the other vehicles are doing and everything and you know there will be some people who are more like more like air traffic controllers call them like AV controllers yeah and so these AV controllers would actually see kind of like like a whole map and they would understand where vehicles are really confident and where they kind of you know need a little bit more help and the help shouldn't be for safety how it should be for efficiency vehicles should be safe no matter what if you had zero people they could be very safe but they be going five miles an hour and so if you want them to go around 25 miles an hour then you need people to come in and and for example you know the vehicle come to an intersection and the vehicle can say you know I can wait I can inch forward a little bit show my intent or I can turn left and right now it's clear I can turn I know that but before you give me the go I won't and so that's one example this doesn't mean necessarily we're doing that actually I think I think if you go down all them all that much detail that every intersection you're kind of expecting a person to press a button then I don't think you'll get the efficiency benefits you want you need to be able to kind of go around and be able to do these things but but I think you need people to be able to set high level behavior to vehicles that's the other thing with autonomous vehicles you know I think a lot of people kind of think about it as follows I mean this happens with technology a lot you know you think alright so I know about cars and I heard robots so I think how this is gonna work out is that I'm gonna buy a car press a button and it's gonna drive itself and when is that gonna happen you know and people kind of tend to think about it that way but when you think about what really happens is that something comes in in a way that you didn't even expect if asked you might have said I don't think I need that or I don't think it should be that and so on and then and then that that becomes the next big thing coding code and so I think that this kind of different ways of humans operating vehicles could be really powerful I think that sooner than then later we might open our eyes up to a world in which you go around walk in a mall and there's a bunch of security they're exactly operated in this way you go into a factory or a warehouse there's a whole bunch of robots they're pretty exactly in this way you go to a you go to the Brooklyn Navy Yard you see a whole bunch of autonomous vehicles Optimus right and they're operated maybe in this way yes but I think people kind of don't see that III sincerely think that it's it's there's a possibility that we may almost see like like a whole mushrooming of this technology in all kinds of places that we didn't expect before and then maybe the real surprise and then one day when your car actually drives itself it may not be all that much of a surprise at all because you see it all the time you interact with them you take the Optimus ride hopefully that's your choice and then you know you you hear a bunch of things you go around you in your act with them I don't know like you have a little delivery vehicle that goes around the sidewalks and delivers you things and then you take it it says thank you and then you get used to that and one day your car actually drives itself and the regulation goes by and you know you can hit the button asleep and it wouldn't be a surprise at all I think that maybe the real reality so there's gonna be a bunch of applications that pop up around autonomous vehicles some some of which maybe many of which we don't expect at all so if we look at Optimus ride what do you think you know the viral application that the one that like really works for people in mobility what do you think optimus ride will connect with in in in near future first um I think that the first place is that that I like the target honestly is like these places where transportation is required within an environment like people typically call a geofence so you can imagine like a roughly two mile by two mile could be bigger could be smaller type of an environment and there's a lot of these kinds of environments they're typically transportation deprived the Brooklyn Navy Yard that you know we're in today we're in a few different places but that's that was the one that was less publicized that's a good example so there's not a lot of transportation there and you wouldn't expect like I don't know I think maybe operating an uber there ends up being sort of a little too expensive or when you compare it with operating uber that becomes the elsewhere becomes the priority and these people whose place has become totally transportation deprived and then what happens is that you know people drive into these places and to go from point A to point B inside this place within that day they use their cars and so we end up building more parking for them to for example take their cars and go to the lunch place and I think that one of the things that can be done is that you know you can put in efficient safe sustainable transportation systems into these types of places first and I think that you know you could deliver mobility in an affordable way affordable accessible you know sustainable way but I think what also enables is that this kind of effort money area land that we spend on parking we could reclaim some of that and that is on the order of like even for a small environment like two mile by two mile it doesn't have to be smack in the middle of New York I mean anywhere else you're talking tens of millions of dollars if you're smack in the middle of New York you're looking at billions of dollars of savings just by doing that and that's the economic part of it and there's a societal part right I mean just look around I mean the places that we live are like built for cars it didn't look like this just like a hundred years ago like today no one walks in the middle of the street it's four cars we no one tells you that growing up but you grow into that reality and so sometimes they close the road it happens here you know like the celebration they close the road still people don't walk in the middle of the road like just walk in and people don't but I think it has so much impact the the car in in the space that we have and and I think we talked about sustainability livability I mean ultimately these kinds of places that parking spots at the very least could change into something more useful or maybe just like park areas recreational and so I think that's the first thing that that we're targeting and I think that we're getting like a really good response both from an economic societal point of view especially places that are a little bit forward-looking and like for example Brooklyn Navy Yard they have tenants there distinct I recall like new lab it's kind of like an Innovation Center there's a bunch of startups there and so you know you get those kinds of people and you know that they're really interested in sort of making that environment more livable and these kinds of solutions that Optimus tried provides almost kind of comes in and and becomes that and many of these places that are transportation deprived you know they have they actually ran shuttles and so you know you can ask anybody the shuttle experience is like terrible people hate shuttles and I can tell you why it's because you know like the driver is very expensive in a shuttle business so what makes sense is to attach 2030 seats to a driver and a lot of people have this misconception they think that shuttle should be big sometimes we get that our optimist right we tell them we're gonna give you like four seater six Cedars and we get asked like how about like twenty Cedars like you know you don't need twenty Cedars you want to split up those seeds so that they can travel faster and the transportation delays would go down that's what you want if you make it big not only you will get delays in transportation but you won't have an agile vehicle it will take a long time to speed up slow down and so on it'll you need to climb up to the thing so it's kind of like really hard to interact with and scheduling too perhaps when you have more smaller vehicles because closer to BER where you can actually get a personal I mean just the logistics of getting the vehicle to you it becomes easier when you have a giant shadow there's fewer of them and it probably goes on a route a specific route that's supposed to hit and when you go on a specific route and all seats travel together versus you know you have a whole bunch of them you can imagine the route you can still have but you can imagine you split up the seats and instead of you know damn traveling like I don't know a mile apart they could be like you know half a mile apart if you split them into two that basically would mean that your delays when you go out you want wait for them for a long time and that's one of the main reasons or you don't have to climb up the other thing is that I think if you split them up in a nice way and if you can actually know where people are going to be somehow you don't even need the app a lot of people ask us the app we say why don't you just walk into the vehicle how about you just walk into the vehicle it recognizes who you are and it gives you a bunch of options of places that you go and you just kind of go there I mean people kind of also internalize the apps everybody needs a nap it's like you don't need an app you just walk into the place walk up but I think I think one of the things that you know we really try to do is to take that shuttle experience that no one likes and tilt it into something that everybody loves and so I think that's another important thing I would like to say that carefully just like today operationally we don't do shuttles you know we're really kind of thinking of this as a system or a network that we're designing but but ultimately we go to places that would normally rent the shuttle service that people wouldn't like as much and we want to tilt it into something that people love so you virtually the second earlier but how many optimist ride vehicles do you think would be needed for any person in Boston or New York if they step outside there will be this this is like a mathematical question there'll be two optimist ride vehicles within line of sight is that the right number - well these for example um that's that's the density so meaning that if you see one vehicle you look around you see another one - imagine like you know Tesla will tell you they collect a lot of data do you see that with Tesla like you just walk around and you look on you see Tesla probably not very specific areas of California maybe maybe you're right like there's a couple zip codes that you know just but I think but I think that's kind of important because you know like maybe the couple zip codes um the one thing that we kind of depend on I'll get to your question in a second but now like we're taking a lot of tangents today oh yeah so so so I think that this is actually important people call this data density or data velocity so it's very good to collect data in a way that you know you see the same place so many times like you can drive 10,000 miles around the country or you drive 10,000 miles in a confined environment you'll see the same intersection hundreds of times and when it comes to dick ting what people are gonna do in that specific intersection we become really good at it versus if you draw in like ten thousand miles around the country you sing that only once and so trying to predict what people do become sorry and I think that you know you said what is needed it's tens of thousands of vehicles you know you really need to be like a specific fraction or vehicle like for example in good times in Singapore you can go and you can just grab a cab and they are like you know 10% 20% of traffic those taxis ultimately that's why you need to get to so that you know you you get to a certain place where you really the benefits really kick off and like orders of magnitude type of a point but once you get there you actually get the benefits and you can certainly carry people I think that's one of the things people really don't like to wait for themselves but for example they can wait a lot more for the goods if they order something like there you were sitting at home and you want to wait half an hour that sounds great people say it's great you want to you're gonna take a cab you're waiting half an hour like that's crazy you don't want to wait that much but I think you know you you can I think really get to a point where the system at peak times really focuses on kind of transporting humans around and then it's it's really it's a good fraction of traffic to the point where you know you go you look around there's something there and you just kind of basically get in there and it's already waiting for you or something like that and then you take it if you do it at that scale like today for instance uber if you talk to a driver right I mean uber takes a certain cut it's a small cut or drivers would argue that it's a large cut but you know it's it's it's when you look at the grand scheme of things most of that money that you pay Hueber kind of goes to the driver and if you talk to the driver the driver will claim that most of it is their time you know they it's not spent on gas they think it's not spent on the the car per se as much it's like their time and if you didn't have a have a person driving or if you're in a scenario where you know like point one person is driving the car a fraction of a person is kind of operating the car because you know your one operates several if you're in that situation you realize that the internal combustion engine type of cars are very inefficient you know we built them to go on highways they pass crash tests they're like really heavy they really don't need to be like 25 times the weight of its passengers or or you know like area wise and so on and but if you get through those inefficiencies and if you really build like urban cars and things like that I think the economics really starts to check out like to the point where I mean I don't know you may be able to get into a car and it may be less than a dollar to go from A to B as long as you don't change your destination you just pay 99 cents and go that if you share it if you take another stop somewhere it becomes a lot better you know these kinds of things at least four models at least for mathematics and theory they start to really check out so I think it's really exciting what Optimus Art is doing in terms of it feels the most reachable like they'll actually be here and have an impact yeah that is the idea and if we contrast that again we'll go back to our old friends way Moe and Tesla so way Moe seems to have sort of technically similar approaches as Optimus ride but a different they're not as interested it has having an impact today these in nature they have a longer term sort of investments almost more of a research project still meaning they're trying to solve as far as I understand maybe you can you can differentiate but they seem to want to do more unrestricted movement meaning move from A to B where A to B is all over the place versus Optimus right is really nicely geofence and really sort of established mobility in a particular environment before you expand it and then Tesla is like the complete opposite which is you know the entirety of the world actually is going to be automated highway driving urban driving every kind of driving you know you kind of creep up to it by incrementally improving the capabilities of the autopilot system so when you contrast all of these and on top of that let me throw a question that nobody likes but his timeline when do you think each of these approaches loosely speaking nobody can predict the future will see mass deployment so yah mosque predicts the the craziest approach is at the I've heard figures like at the end of this year right so that's probably wildly inaccurate but how wildly inaccurate is it I mean first thing to lay out like everybody else it's really it's really hard to guess I mean I don't know I don't know where where Tesla can look at or Elon Musk can look at and say hey you know it's the end of this year I mean I don't know what you can look at you know even the data that you know you I mean if you look at the data even kind of trying to extrapolate the end state without knowing what exactly is gonna go especially for like a machine learning approach I mean it's just kind of very hard to predict but I do think the following does happen I think a lot of people you know what they do is that there's something that I called a couple times time dilation in technology prediction happens let me try to describe a little bit there's a lot of things that are so far ahead people think they're close and there's a lot of things that are actually close people think it's far ahead people tries to kind of look at a whole landscape of technology development admit needs chaos anything can happen in any order at any time and there's a whole bunch of things in that people take it clamp it and put it into the next three years and so then what happens is that there's some things that maybe can happen by the end of the year or next year and so on and they push that into like few years ahead because it's just hard to explain and there are things that are like we were looking at 20 years more maybe you know hopefully in my lifetime type of things and cuz you know we don't know I mean we don't know how hard it is even like that's a problem we don't know like if some of these problems are actually AI complete like we have no idea what's going on and and you know we we take all of that and then we clump it and then we say three years from now and then some of us are more optimistic so they're shooting at the at the end of the year and some of us are more realistic they say like five years but you know we all I think it's just hard to know and and I think trying to predict like products ahead to three years it's it's hard to know in the following sense you know like we typically say okay this is a technology company but sometimes sometimes really you're trying to build something where technology does like there's a technology gap you know like and Tesla had that with electric vehicles you know like when they first started they would look at a chart much like a moose law type of chart and they would just kind of extrapolate that out and they'd say we want to be here what's the technology to get that we don't know it goes like this so it's probably just gonna you know keep going yeah um with bit AI that goes into the cars we don't even have that like we can't I mean what can you quantify yeah like what kind of chart are you looking at you know but so but so I think when there's the technology gap it's just kind of really hard to predict so now I realize I talk like five minutes and avoid your question I didn't tell you anything about and I don't think you I think you've actually argued that it's not used even NES you provide now is not that used to be very hard there's one thing that I really believe in and and you know this is not my idea and it's been you know discussed several times but but this this this kind of like something like a startup or a kind of an innovative company including definitely may want may vary more Tesla maybe even some of the other big companies that are kind of trying things this kind of like iterated learning is very important the fact that we're over there and we're trying things and so on I think that's that's important we try to understand and and I think that you know the coding code Silicon Valley has done that with business models pretty well and now I think we're trying to get to do it well there's a little technology gap I mean before like you know you're trying to build I'm not trying to you know I think these companies are building great technology to for example enable internet search to do it so quickly and that kind of didn't what wasn't there so much but at least like it was a kind of a technology that you could predict to some degree and so on and now we're just kind of trying to build you know things that it's kind of hard to quantify what kind of a metric are we looking at so psychologically is a sort of as a leader of graduate students and an optimist ride a bunch of brilliant engineers just curiosity psychologically do you think it's good to think that you know whatever technology gap we're talking about can be closed by the end of the year or do you you know because we don't know so the way do you want to say that everything is going to improve exponentially to yourself and to others around you as a leader or do you want to be more sort of maybe not cynical but I don't want to use realistic because it's hard to predict but yeah maybe more cynical pessimistic about the ability to close again yeah I I think that you know going back I think that iterated learning is like key that you know you're out there you're running experiments to learn and that doesn't mean sort of like you know you like like your optimist right you're kind of doing something but I like in an environment but like what Tesla is doing I think is also kind of like this this kind of notion and and you know people can go around and say like you know this year next year the other year and so on but but I think that the nice thing about it is that they're out there they're pushing this technology in I think what they should do more of I think that kind of informed people about what kind of technology that they're providing you know the good and the bad and then you know not just sort of you know if it works very well but I think you know I'm not saying they're not doing bad and informing I think they're kind of trying they you know they put up certain things or at the very least YouTube videos comes out on on how the summon function works every now and then and and you know people get informed and so that that kind of cycle continues but you know I I admired I think they're kind of go out there and they do great things they do their own kind of experiment I think we do our own and I think we're closing some similar technology gaps but some also some are orthogonal as well you know I think like like we talked about you know people being remote like it's something or in the kind of environments that we're in or think about a test the car maybe maybe you can enable it one day like there's you know low traffic like you're kind of the stuff on go emotion you just hit the button and the you can really say or maybe there's another you know Lane that you can pass into you going that I think they can enable these kinds of pride believe it and so I think that that part that is really important and that is really key and and beyond that I think you know when is it exactly gonna happen and and and so on I mean it's like I said it's very hard to predict and I would I would imagine that it would be good to do some sort of like a like a one or two year plan when it's a little bit more predictable that you know you the technology gaps you close and and there and the kind of sort of product that would answer so I know that from optimist ride or you know other companies that I get involved in I mean at some point you find yourself in a situation where you're trying to build a product and and people are investing in that in that you know building effort and those investors that they do want to know as they compare the investments they want to make they do want to know what happens in the next one or two years and I think that's good to communicate that but I think beyond that it becomes it becomes a vision that we want to get to someday and saying five years ten years I don't think it means anything but iterative learning is key though you do and learn I think that is key you know I got a sort of throwback right at you criticism in terms of you know like Tesla or somebody communicating you know how someone works and so on I got a chance to visit Optimus ride and you guys are doing some awesome stuff and yet the internet doesn't know about it so you should also communicate more showing off in showing off some of the awesome stuff the stuff that works and stuff that doesn't work I mean it's just the stuff I saw with the tracking different objects and pedestrians so I'm incredible stuff going on there just cool maybe it's just the nerd of me but I think the world would love to see that kind of stuff yeah that's that's well taken I think you know I should say that it's not like you know we we weren't able to I think we made a decision at some point that decision did involve me quite a bit on kind of sort of doing this in kind of coding called stealth mode for a bit but I think that you know we will open it up quite a lot more and I think that we are also that optimist right kind of hitting when you new era you know we're big now we're doing a lot of interesting things and and I think you know some of the deployments that we kind of announced were some of the first bits bits of information that we kind of put out into the world we'll also put out our technology a lot of the things that we've been developing is really amazing and you know we're gonna we're gonna start putting it out now we're especially interested in sort of like being able to work with the best people and I think and I think it's it's good to not just kind of show them and they come to our office for an interview but just put it out there in terms of like you know get people excited about what we're doing so on Thomas vehicle space let me ask one last question so yah mosque famously said that lighter is a crutch so uh I've talked to a bunch of people bought it got asked you you use that crutch quite a bit in the DARPA days so you know and is that his idea in general sort of you know more provocative and fun I think than a technical discussion but the idea is that camera based can't primarily camera based systems is going to be what defines the future of autonomous vehicles so what do you think of this idea ladders a crutch versus primarily uh camera based systems first things first I think you know I'm a big believer in just camera based autonomous vehicle systems like I think that you know you can put in a lot of autonomy and and you can do great things and and it's it's it's very possible that at the time scales like we said we can't predict twenty years from now like you may be able to do do things that we're doing today only what lidar and you may be will do them just with cameras and I think that you know you can just I I think that I will put my name on it to like you know there will be a time when you can only use cameras and you'll be fine at that time though it's very possible that you know you find the lidar system as another robusta fire or or it's so affordable that it's stupid not to you know just kind of put it there and I think and I think we may be looking at a future like that do you think we're over relying on lidar right now because we understand it better it's more reliable anyways internment from a safety easier to build with that's the other that's the other thing I think to be very frank with you I mean you know we've seen a lot of sort of autonomous vehicles companies come and go and the approach has been you know you slap a lidar on a car and it's kind of easy to build with when you have a lighter are you know you just kind of coat it up and and you hit the button and you do a demo so I think there's admittedly there's a lot of people they focus on the lidar because it's easier to build with that doesn't mean that you know without the cameras just cameras you can you cannot do what they're doing but it's just kind of a lot harder and so you need to have certain kind of expertise to exploit that what we believe in and you know you may be seeing some of it is that we believe in computer vision we certainly work on computer vision and optimist ride by a lot like um and and we've been doing that from day one and we also believe in sensor fusion so you know we do we have a relatively minimal use of light ours but but we do use them and I think you know in the future I really believe that the following sequence of events may happen first things first number one there may be a future in which you know there's like cars with light hours and everything and the cameras but you know this in this 50 year ahead future they can drive with cameras as well especially in some isolated environments and cameras they go and they do the thing in the same future it's very possible that you know the white ARS are so cheap and frankly make the software may be a little less compute-intensive at the very least or maybe less complicated so that they can be certified or or insured there of their safety and things like that that it's kind of stupid not to put the lidar like imagine this you either put pay money for the lidar or you pay money for the compute and if you don't put the lidar it's a more expensive system because you have to put in a lot of compute like this is another possibility I do think that a lot of the sort of initial deployments of self-driving vehicles I think they will involve light ARS and especially either low range or short either short range or low resolution light ARS are actually not that hard to build in solid state they're still scanning but like MEMS type of scanning light ours and things like that they're like they're actually not that hard I think they will may be kind of playing with the spectrum and the phaser eyes they're a little bit harder but but I think like you know putting your mom's mirror in there that kind of scans the environment it's not hard the only thing is that you know you just like with a lot of the things that we do nowadays in developing technology you hit fundamental limits of the universe the speed of light becomes a problem in when you're trying to scan the environment so you don't get either good resolution or you don't get range but but you know it's still it's something that you can put in that affordably so let me jump back to drones you've uh you have a role in the Lockheed Martin alpha pilot Innovation Challenge where teams compete in drone racing a super cool super intense interesting application of AI so can you tell me about the very basics of the challenge and where you fit in well your thoughts are on this problem and it's sort of echoes of the early DARPA challenge in the through the desert that we're seeing now now with drone racing yeah I mean one interesting thing about it is that you know people drone racing a this is an eSport and so it's much like you're playing a game but there's a real drone going in an environment the human being is controlling it with goggles on so there's no it is a robot but there's no AI there's no way I am human being is controlling it and so that's already there and and I've been interested in this problem for quite a while actually from a robot assist point of view and that's what's happening in alpha pilot which which probably of aggressive flight of aggressive flight fully autonomous aggressive flight the problem that I'm interested in you asked about alpha pod and I'll get there in a second but the problem that I'm interested in I'd love to build autonomous vehicles like like drones that can go far faster than any human possibly can I think we should recognize that we as humans have you know limitations in how fast we can process information and those are some biological limitations like we think about this AI this way too I mean this has been discussed a lot and this is not sort of my idea per se but a lot of people kind of think about human level III and they think that you know AI is not human level one day it'll be human level and humans in the eyes they kind of interact versus I think that the situation really is that humans are at a certain place and AI keeps improving and at some point just crosses off and then you know it gets smarter and smarter and smarter and so drone releasing the same issue humans play this game and you know you have to like react in milliseconds and there's really you know you see something with your eyes and then that information just flows through your brain into your hands so that you can command it and there's some also delays and you know getting information back and forth but suppose a laser don't exist you just just a delay between your eye and your fingers please delay that a robot doesn't have to have so we end up building in my research group like systems that you know see things at a kilohertz like a human eye would barely hit a hundred Hertz so imagine things that see stuff in slow motion like 10x slow motion it will be very useful like we talked a lot about autonomous car so you know we don't get to see it but the hundred lives are lost every day just in the United States on traffic accidents and many of them are like known cases you know like the you're coming through like like a ramp going into a highway you hit somebody and you're off or you know like you kind of get confused you try to like swerve into the next lane you go off the road and you crash whatever and I'm I think if you had enough computer in a car and a very fast camera right at the time of an accident you could use all compute you have like you could shut down the infotainment system and use that kind of computing resources instead of rendering you use it for the kind of artificial intelligence that goes in there the autonomy and you can you can either take control of the car and bring it to a full stop but even even if you can't do that you can deliver what the human is trying to do human is trying to change the lane but goes off the road not being able to do that with motor skills and the eyes and you know you can get in that and I was there's so many other things that you can enable with what I would call high throughput computing you know data is coming in extremely fast and in real time you have to process it and the current CPUs have ever fast you clock it are typically not enough you need to build those computers from the ground up so that they can ingest all that data that I'm really interested in just on that point really quick is the currently what's the bottom like you mentioned the delays in humans is it the hardware so you work a lot with NVIDIA hardware is it the hardware is it the software I think it's both I think it's both in fact they need to be co-developed I think in the future I mean that's a little bit what Nvidia does sort of like they almost like build the hardware and then they build the neural networks and then they build the hardware back and the neural networks back and it goes back and forth but it's that Co design and I think that you know like we try to way back we try to build a faster own that could use a camera image to like track what's moving in order to find where it is in the world this typical sort of you know visual inertial state estimation problems that we would solve and you know we just kind of realize that we're at the limit sometimes of you know doing simple tasks we're at the limit of the camera frame rate because you know you really want to track things you want the camera image to be 90% kind of like or or some somewhat the same from one frame to the next and why are we at the limit of the camera frame rate it's because camera captures data it puts it into some serial connection it could be USB or like there's something called camera serial interface that we use a lot it puts into some serial connection and copper wires can only transmit so much data and you hit the Shannon limit on copper wires and you know you you hit yet another kind of Universal limit that you can transfer the data so you have to be much more intelligent on how you capture those pixels you can take compute and put it right next to the pixels people are governed all that you do how hard is to get past the bottleneck of the copper wire yeah you need to you need to do a lot of parallel processing as you can imagine the same thing happens in the GPUs you know like the data is transferred in parallel somehow it gets into some parallel processing I think that you know like now we're really kind of diverted off into so many different dimensions but great so its aggressive light how do we make drones see many more frame just a second in order to enable aggressive fight that's a super interesting problem that's an interesting problem so but like think about it you have you have CPUs you clock them at you know several gigahertz we don't clock them faster largely because you know we run into some heating issues and things like that but another thing is that 3 gigahertz clock light travels kind of like on the order of a few inches or an inch that's the size of a chip and so you pass a clock cycle and as the clock signal is going around in the chip you pass another one and so trying to coordinate that the design of the complexity of the chip becomes so hard I mean we have hit the fundamental limits of the universe in so many things that we're designing I don't know I realize that it's great but like we can't make transistors smaller because like quantum effects that electrons start to tunnel around we can't clock it faster one of the reasons why is because like the information doesn't travel faster in the universe yeah and we're limited by that same thing with the laser scanner but so then it becomes clear that you know the way you organize the chip into a CPU or even a GPU you now need to look at how to redesign that if you're gonna stick with silicon yes you could go do other things too I mean there's that too but you really almost need to take those transistors put them in a different way so that the information travels on those transistors in a different way in a much more way that is specific to the high-speed cameras coming in and so that's one of the things that that we talk about quite a bit so drone racing kind of really makes that embodies that he embodies that and that's what is exciting it's exciting for people you know students like it it embodies all those problems but going back we're building coding code and other engine and that engine I hope one day we'll be just like how impactful seatbelts were in in driving I hope so Wow or it could enable your next generation autonomous air taxis and things like that I mean it sounds crazy but one day we may need to purge that these things if you really want to go from Boston to New York in more than a half hours you may want to fixed-wing aircraft most of these companies that are kind of doing Concorde flying cars they're focusing on that but then how do you land it on top of a building you may need to pull off like kind of fast maneuvers for a robot like perch land it's gonna go perch into into a building if you want to do that like you need these kinds of systems and so drone racing you know it's being able to go very faster than anything we can't comprehend take an aircraft forget the quadcopter we take your fixed-wing while you're at it you might as well put some like rocket engines in the back you just light it you go through the gate and everyone looks at it and just said what just happened yeah and they would say it's impossible for me to do that and that's closing the same technology gap that would you know one day steer cars out of accidents so but let's get back to the practical which is sort of just getting the thing to work in a race environment which is kind of what the is another kind of exciting thing which the DARPA challenge to the desert did you know theoretically we had autonomous vehicles but making them successfully finish a race first of all which nobody finished the first year and then the second year just to get you know to finish and really go at a reasonable time is really difficult engineering practically speaking challenge so that let me ask about the the Alpha pilot challenge is a I guess a big prize potentially associated with it but let me ask reminiscent of the DARPA Days predictions you think anybody will finish well not not soon I think that depends on how you set up the race course and so if the race course is a slalom course I think people will kind of do it but can you set up some course like literally sunk or you get to design it there is the algorithm developer can you set up some course so that you can beat the best human when is that gonna happen like that's not very easy even just setting up some course if you let the human that you're competing with set up the course it becomes a worries a lot harder hmm so how many in the space of all possible courses are would humans win and quad machines were a great question let's get to that I want to answer your other question which is like the DARPA challenge days right what was really hard I think I think we understand we understood what we wanted to build but still building things that experimentation that iterated learning that takes up a lot of time actually and and so in my group for example in order for us to be able to develop fast we build like VR environments will take an aircraft will put it in a motion capture room big huge motion capture room and we'll fly it in real time will render other images and beam it back to the drone that sounds kind of notionally simple but it's actually hard because now you're trying to fit all that data through the air into the drone and so you need to do a few crazy things to make it happen but once you do that then at least you can try things if you crash into something you didn't actually crash so it's like the whole drone is in VR we can do augmented reality and so on and so I think at some point testing becomes very important one of the nice things about alpha pilot is that they built the drone and they build a lot of drones and and it's okay to crash in fact I think maybe you know the viewers may kind of like to see things that suppose that potentially could be the most exciting part it could be the exciting part and I think you know as an engineer it's a very different situation to be in like in academia a lot of my colleagues who are actually in this race and they're really great researchers but I've seen them trying to do similar things whereby they built this one drone and you know some somebody with like a face mask and a glows are going you know right behind the drone is trying to hold it if it if it falls down imagine you don't have to do that I think that's one of the nice things about auto pilot challenge where you know we have this drones and we're going to design the courses in a way that will keep pushing people up until the crashes start to happen and you know we'll hopefully sort of I don't think you want to tell people crashing is okay if we want to be careful here but because you know we don't people to crash a lot but certainly you know we want we want them to push it so that you know everybody crashes once or twice and and and you know they're really pushing it to their limits and that's where iterated learning comes in as ever every crash is a lesson is the lesson exactly so in terms of the space of possible courses how do you think about it in in the in the war of the video versus machines or do machines when we look at that quite a bit I mean I think that you know you will see quickly that like you can design a course and you know in in in certain courses like in the middle somewhere if if you kind of run through the course once you know the Machine gets beaten pretty much consistently by slightly but if you go through the course like 10 times humans get beaten very slightly but consistently so humans at some point you know you get confused you get tired and things like that versus machine is just executing the same line of code tirelessly just going back to the beginning and doing the same thing exactly I think I think that kind of thing happens and I realize sort of as humans there's the classical things you know that everybody has realized like like if you put in some sort of like strategic thinking that's a little bit harder for machines that I think sort of comprehend precision is easy to do so that's what they excel in and also sort of repeatability is easier to do that's what they excel in they can you can build machines that excel in strategy as well and beat humans that way too but that's a lot harder to build I have a million more questions but in the interest of time last question yeah what is the most beautiful idea you've come across in robotics well their simple equation experiment a demo simulation piece of software what just gives you pause that's an interesting question I have done a lot of work myself in decision making so I've been interested in that area so you know in robotics you have somehow the field has split into like you know there's people who would work on like perception how robots perceive the environment then how do you actually make like decisions and there's people also like how do you interact people interact with drove us is a whole bunch of different fields and and you know I I have admittedly worked a lot on the more control and decision-making than the others and I think that you know the one equation that has always kind of baffled me is Bellman's equation and so it's it's this person who have realized like way back you know more than half a century ago on like how do you actually sit down and if you have several variables that you're kind of jointly trying to determine how do you determine that and there is one beautiful equation that you know like today people do reinforcement where we still use it and and it's it's baffling to me because it both kind of tells you the simplicity because it's a single equation that anyone can write now we can teach it in the first course on decision-making at the same time it tells you how computation we how hard the problem is I feel like my like a lot of the things that I've done at MIT for research has been kind of just this fight against computational efficiency things like how can we get it faster to the point where we now got to like let's just redesign this chip like maybe that's the way but I think it talks about how computationally hard certain problems can be by nowadays what people call curse of dimensionality and so as the number of variables cannot grow the number of decisions you can make grows rapidly like if you have you know 100 variables each one of them take ten values all possible assignments is more than the number of atoms in the universe it's just crazy and and that kind of thinking is just embodied in that one equation that I really like and the beautiful balance between it being theoretically optimal and somehow practically speaking given the curse of dimensionality nevertheless in practice works among you know despite all those challenges which is quite incredible it's just quite incredible so you know I would say that it's kind of like quite baffling actually you know in a lot of fields that we think about how little we know you know like and so I think here too you know we know that in the worst case things are pretty hard but you know in practice generally things work so it's just kind of its kind of baffling decision-making how little we know just like how little we know about the beginning of time how little we know about you know our own future like if you actually go into like from balanced equation all the way down I mean there's also how little we know about like mathematics I mean we don't even know the axioms are like consistent it's just crazy yeah yeah I think a good lesson the lesson there just as you said we tend to focus on the worst case or the the boundaries of everything we're studying and then the average case seems to somehow work out if you think about life in general we mess it up a bunch you know we freaked out about a bunch of the traumatic stuff but in the end it seems to work out ok yeah that seems like a good metaphor sir touch it thank you so much for being a friend a colleague a mentor that really appreciates it on and talk to you like mice thank you thanks thanks for listening to this conversation with sir - karma and thank you to a presenting sponsor cash app please consider supporting the podcast by downloading cash app and using code lex podcast enjoy this podcast subscribe my youtube review it with 5 stars an apple podcast supported on patreon or simply connect with me on Twitter at Lex Friedman and now let me leave you with some words from Hal 9000 from the movie 2001 a Space Odyssey I'm putting myself to the fullest possible use which is all I think that any conscious entity can ever hope to do thank you for listening and hope to see you next time you