Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI | Lex Fridman Podcast #333
think it's possible that physics has exploits and we should be trying to find them arranging some kind of a crazy quantum mechanical system that somehow gives you buffer overflow somehow gives you a rounding error in the floating Point synthetic intelligences are kind of like the next stage of development and I don't know where it leads to like at some point I suspect the universe is some kind of a puzzle these synthetic AIS will uncover that puzzle and solve it the following is a conversation with Andre capothy previously the director of AI at Tesla and before that at open Ai and Stanford he is one of the greatest scientists engineers and Educators in the history of artificial intelligence this is the Lex Friedman podcast to support it please check out our sponsors and now dear friends here's Andre capathi what is a neural network and why does it seem to uh do such a surprisingly good job of learning what is a neural network it's a mathematical abstraction of the brain I would say that's how it was originally developed at the end of the day it's a mathematical expression and it's a fairly simple mathematical expression when you get down to it it's basically a sequence of Matrix multiplies which are really dot products mathematically and some nonlinearities thrown in and so it's a very simple mathematical expression and it's got knobs in it many knobs many knobs and these knobs are Loosely related to basically the synapses in your brain they're trainable they're modifiable and so the idea is like we need to find the setting of The Knobs that makes the neural nut do whatever you want it to do like classify images and so on and so there's not too much mystery I would say in it like um you might think that basically don't want to endow it with too much meaning with respect to the brain and how it works it's really just a complicated mathematical expression with knobs and those knobs need a proper setting for it to do something uh desirable yeah but poetry is just the collection of letters with spaces but it can make us feel a certain way and in that same way when you get a large number of knobs together whether it's in a inside the brain or inside a computer they seem to they seem to surprise us with the with their power yeah I think that's fair so basically I'm underselling it by a lot because you definitely do get very surprising emergent behaviors out of these neurons when they're large enough and trained on complicated enough problems like say for example the next uh word prediction in a massive data set from the internet and then these neurons take on a pretty surprising magical properties yeah I think it's kind of interesting how much you can get out of even very simple mathematical formalism when your brain right now I was talking is it doing next word prediction or is it doing something more interesting well definitely some kind of a generative model that's a gpt-like and prompted by you um yeah so you're giving me a prompt and I'm kind of like responding to it in a generative way and by yourself perhaps a little bit like are you adding extra prompts from your own memory inside your head automatically feels like you're referencing some kind of a declarative structure of like memory and so on and then uh you're putting that together with your prompt and giving away some messages like how much of what you just said has been said by you before uh nothing basically right no but if you actually look at all the words you've ever said in your life and you do a search you'll probably said a lot of the same words in the same order before yeah it could be I mean I'm using phrases that are common Etc but I'm remixing it into a pretty uh sort of unique sentence at the end of the day but you're right definitely there's like a ton of remixing what you didn't you it's like Magnus Carlsen said uh I'm I'm rated 2900 whatever which is pretty decent I think you're talking very uh you're not giving enough credit to neural Nets here why do they seem to what's your best intuition about this emergent Behavior I mean it's kind of interesting because I'm simultaneously underselling them but I also feel like there's an element to which I'm over like it's actually kind of incredible that you can get so much emergent magical Behavior out of them despite them being so simple mathematically so I think those are kind of like two surprising statements that are kind of just juxtapose together and I think basically what it is is we are actually fairly good at optimizing these neural Nets and when you give them a hard enough problem they are forced to learn very interesting Solutions in the optimization and those solution basically have these immersion properties that are very interesting there's wisdom and knowledge in the knobs and so what's this representation that's in the knobs does it make sense to you intuitively the large number of knobs can hold the representation that captures some deep wisdom about the data it has looked at it's a lot of knobs it's a lot of knobs and somehow you know so speaking concretely um one of the neural Nets that people are very excited about right now are are gpts which are basically just next word prediction networks so you consume a sequence of words from the internet and you try to predict the next word and uh once you train these on a large enough data set um they you can basically uh prompt these neural amounts in arbitrary ways and you can ask them to solve problems and they will so you can just tell them you can you can make it look like you're trying to um solve some kind of a mathematical problem and they will continue what they think is the solution based on what they've seen on the internet and very often those Solutions look very remarkably consistent look correct potentially do you still think about the brain side of it so as neural Nets is an abstraction or mathematical abstraction of the brain you still draw wisdom from from the biological neural networks or even the bigger question so you're a big fan of biology and biological computation what impressive thing is biology do doing to you that computers are not yet that Gap I would say I'm definitely on I'm much more hesitant with the analogies to the brain than I think you would see potentially in the field um and I kind of feel like certainly the way neural network started is everything stemmed from inspiration by the brain but at the end of the day the artifacts that you get after training they are arrived at by a very different optimization process than the optimization process that gave rise to the brain and so I think uh I kind of think of it as a very complicated alien artifact um it's something different I'm not sorry the uh the neuralness that we're training okay they are complicated uh Alien artifact uh I do not make analogies to the brain because I think the optimization process that gave rise to it is very different from the brain so there was no multi-agent self-play kind of uh setup uh and evolution it was an optimization that is basically a what amounts to a compression objective on a massive amount of data okay so artificial neural networks are doing compression and biological neural networks are not to survive and they're not really doing any they're they're an agent in a multi-agent self-place system that's been running for a very very long time that said Evolution has found that it is very useful to to predict and have a predictive model in the brain and so I think our brain utilizes something that looks like that as a part of it but it has a lot more you know gadgets and gizmos and uh value functions and ancient nuclei that are all trying to like make a survive and reproduce and everything else and the whole thing through embryogenesis is built from a single cell I mean it's just the code is inside the DNA and it just builds it up like the entire organism yes and like it does it pretty well it should not be possible so there's some learning going on there's some there's some there's some kind of computation going through that building process I mean I I don't know where if you were just to look at the entirety of history of life on Earth where do you think is the most interesting invention is it the origin of life itself is it just jumping to eukaryotes is it mammals is it humans themselves Homo sapiens the the origin of intelligence or highly complex intelligence or or is it all just in continuation the same kind of process certainly I would say it's an extremely remarkable story that I'm only like briefly learning about recently all the way from um actually like you almost have to start at the formation of Earth and all of its conditions and the entire solar system and how everything is arranged with Jupiter and Moon and the habitable zone and everything and then you have an active Earth that's turning over material and um and then you start with a biogenesis and everything and so it's all like a pretty remarkable story I'm not sure that I can pick like a single Unique Piece of it that I find most interesting um I guess for me as an artificial intelligence researcher it's probably the last piece we have lots of animals that uh you know are are not building technological Society but we do and um it seems to have happened very quickly it seems to have happened very recently and uh something very interesting happened there that I don't fully understand I almost understand everything else kind of I think intuitively uh but I don't understand exactly that part and how quick it was both explanations would be interesting one is that this is just a continuation of the same kind of process there's nothing special about humans that would be deeply understanding that would be very interesting that we think of ourselves as special but it was obvious all it was already written in the in the code that you would have greater and greater intelligence emerging and then the other explanation which is something truly special happened something like a rare event whether it's like crazy rare event like uh Space Odyssey what would it be see if you say like the invention of Fire or the uh as Richard rangham says the beta males deciding a clever way to kill the alpha males by collaborating so just optimizing the collaborations really the multi-agent aspect of the multi-agent and that really being constrained on resources and trying to survive the collaboration aspect is what created the complex intelligence but it seems like it's a natural outgrowth of the evolution process like what could possibly be a magical thing that happened like a rare thing that would say that humans are actually human level intelligence is actually a really rare thing in the universe yeah I'm hesitant to say that it is rare by the way but it definitely seems like it's kind of like a punctuated equilibrium where you have lots of exploration and then you have certain leaps sparse leaps in between so of course like origin of life would be one um you know DNA sex eukaryotic system eukaryotic life um the endosymbiosis event or the archaeon 8 little bacteria you know just the whole thing and then of course emergence of Consciousness and so on so it seems like definitely there are sparse events where mass amount of progress was made but yeah it's kind of hard to pick one so you don't think humans are unique gotta ask you how many intelligent aliens civilizations do you think are out there and uh is there intelligence different or similar to ours yeah I've been preoccupied with this question quite a bit recently uh basically the for me Paradox and just thinking through and and the reason actually that I am very interested in uh the origin of life is fundamentally trying to understand how common it is that there are technological societies out there uh um in space and the more I study it the more I think that um uh there should be quite a few quite a lot why haven't we heard from them because I I agree with you it feels like I just don't see why what we did here on Earth is so difficult to do yeah and especially when you get into the details of it I used to think origin of life was very um it was this magical rare event but then you read books like for example McLean um uh the vital question a life ascending Etc and he really gets in and he really makes you believe that this is not that rare basic chemistry you have an active Earth and you have your alkaline Vents and you have lots of alkaline Waters mixing whether it's a devotion and you have your proton gradients and you have the little porous pockets of these alkaline vents that concentrate chemistry and um basically as he steps through all of these little pieces you start to understand that actually this is not that crazy you could see this happen on other systems um and he really takes you from just a geology to primitive life and he makes it feel like it's actually pretty plausible and also like the origin of life um didn't uh was actually fairly fast after formation of Earth um if I remember correctly just a few hundred million years or something like that after basically when it was possible life actually arose and so that makes me feel like that is not the constraint that is not the limiting variable and that life should actually be fairly common um and then it you know where the drop-offs are is very um is very interesting to think about I currently think that there's no major drop-offs basically and so there should be quite a lot of life and basically what it where that brings me to then is the only way to reconcile the fact that we haven't found anyone and so on is that um we just can't we can't see them we can't observe them just a quick brief comment Nick Lane and a lot of biologists I talked to they really seem to think that the jump from bacteria to more complex organisms is the hardest jump the eukaryotic glyphosis yeah which I don't I get it they're much more knowledgeable uh than me about like the intricacies of biology but that seems like crazy because how much how many single cell organisms are there like and how much time you have surely it's not that difficult like in a billion years it's not even that long of a time really just all these bacteria under constrained resources battling it out I'm sure they can invent more complex again I don't understand it's like how to move from a hello world program to like like invent a function or something like that I don't yeah I I so I don't yeah so I'm with you I just feel like I don't see any if the origin of life that would be my intuition that's the hardest thing but if that's not the hardest thing because it happens so quickly then it's got to be everywhere and yeah maybe we're just too dumb to see it well it's just we don't have really good mechanisms for seeing this life I mean uh by what how um so I'm not an expert just to preface this but just said it was I want to meet an expert on alien intelligence and how to communicate I'm very suspicious of our ability to to find these intelligences out there and to find these Earths like radio waves for example are are terrible uh their power drops off as basically one over R square uh so I remember reading that our current radio waves would not be uh the ones that we we are broadcasting would not be uh measurable by our devices today only like was it like one tenth of a light year away like not even basically tiny distance because you really need like a targeted transmission of massive power directed somewhere for this to be picked up on long distances and so I just think that our ability to measure is um is not amazing I think there's probably other civilizations out there and then the big question is why don't they build binomial probes and why don't they Interstellar travel across the entire galaxy and my current answer is it's probably Interstellar travel is like really hard uh you have the interstellar medium if you want to move at closer speed of light you're going to be encountering bullets along the way because even like tiny hydrogen atoms and little particles of dust are basically have like massive kinetic energy at those speeds and so basically you need some kind of shielding you need you have all the cosmic radiation uh it's just like brutal out there it's really hard and so my thinking is maybe Interstellar travel is just extremely hard to build hard it feels like uh it feels like we're not a billion years away from doing that it just might be that it's very you have to go very slowly potentially as an example through space um right as opposed to close the speed of light so I'm suspicious basically of our ability to measure life and I'm suspicious of the ability to um just permeate all of space in the Galaxy or across galaxies and that's the only way that I can certainly I can currently see a way around it yeah it's kind of mind-blowing to think that there's trillions of intelligent alien civilizations out there kind of slowly traveling through space to meet each other and some of them meet some of them go to war some of them collaborate or they're all just uh independent they are all just like little pockets I don't know well statistically if there's like if it's there's trillions of them surely some of them some of the pockets are close enough to get some of them happen to be close yeah in the close enough to see each other and then once you see once you see something that is definitely complex life like if we see something yeah we're probably going to be severe like intensely aggressively motivated to figure out what the hell that is and try to meet them what would be your first instinct to try to like at a generational level meet them or defend against them or what would be your uh Instinct as a president of the United States and the scientists I don't know which hat you prefer in this question yeah I think the the question it's really hard um I will say like for example for us um we have lots of primitive life forms on Earth um next to us we have all kinds of ants and everything else and we share space with them and we are hesitant to impact on them and to we are and we're trying to protect them by default because they are amazing interesting dynamical systems that took a long time to evolve and they are interesting and special and I don't know that you want to um destroy that by default and so I like complex dynamical systems that took a lot of time to evolve I think I'd like to I like to preserve it if I can afford to and I'd like to think that the same would be true about uh the galactic resources and that uh they would think that we're kind of incredible interesting story that took time it took a few billion years to unravel and you don't want to just destroy it I could see two aliens talking about Earth right now and saying uh I'm I'm a big fan of complex dynamical systems so I think it was a value to preserve these and who basically are a video game they watch or show a TV show that they watch yeah I think uh you would need like a very good reason I think to to destroy it uh like why don't we destroy these ant farms and so on it's because we're not actually like really in direct competition with them right now uh we do it accidentally and so on but um there's plenty of resources and so why would you destroy something that is so interesting and precious well from a scientific perspective you might probe it yeah you might interact with it later you might want to learn something from it right so I wonder there's could be certain physical phenomena that we think is a physical phenomena but it's actually interacting with us to like poke the finger and see what happens I think it should be very interesting to scientists other alien scientists what happened here um and you know it's a what we're seeing today is a snapshot basically it's a result of a huge amount of computation uh of over like billion years or something like that so it could have been initiated by aliens this could be a computer running a program like when okay if you had the power to do this when you okay for sure at least I would I would pick uh a Earth-like planet that has the conditions based my understanding of the chemistry prerequisites for life and I would see it with life and run it right like yeah wouldn't you 100 do that and observe it and then protect I mean that that's not just a hell of a good TV show it's it's a good scientific experiment yeah and that in his it's physical simulation right maybe maybe the evolution is the most like actually running it uh is the most efficient way to uh understand computation or to compute stuff or to understand life or you know what life looks like and uh what branches it can take it does make me kind of feel weird that we're part of a science experiment but maybe it's everything's a science experiments how to does that change anything for us for a science experiment um I don't know two descendants of Apes talking about being inside of a science experience I'm suspicious of this idea of like a deliberate Pence Premiere as you described it service and I don't see a divine intervention in some way in the in the historical record right now I do feel like um the story in these in these books like Nick Lane's books and so on sort of makes sense uh and it makes sense how life arose on Earth uniquely and uh yeah I don't need a I need I don't need to reach for more exotic explanations right now sure but NPCs inside a video game don't don't don't observe any divine intervention either and we might just be all NPCs running a kind of code maybe eventually they will currently NPCs are really dumb but once they're running gpts um maybe they will be like hey this is really suspicious what the hell so you uh famously tweeted it looks like if you bombard Earth with photons for a while you can emit A roadster so if like an Hitchhiker's Guide to the Galaxy we would summarize the story of Earth so in in that book it's mostly harmless uh what do you think is all the possible stories like a paragraph long or a sentence long that Earth could be summarized as once it's done it's computation so like all the Possible full if Earth is a book right yeah uh probably there has to be an ending I mean there's going to be an end to Earth and it could end in all kinds of ways it could end soon it can end later what do you think are the possible stories well definitely there seems to be yeah you're sort of it's pretty incredible that these self-replicating systems will basically arise from the Dynamics and then they perpetuate themselves and become more complex and eventually become conscious and build a society and I kind of feel like in some sense it's kind of like a deterministic wave uh that you know that kind of just like happens on any you know any sufficiently well arranged system like Earth and so I kind of feel like there's a certain sense of inevitability in it um and it's really beautiful and it ends somehow right so it's a it's a chemically a diverse environment where complex dynamical systems can evolve and become more more further and further complex but then there's a certain um what is it there's certain terminating conditions yeah I don't know what the terminating conditions are but definitely there's a trend line of something and we're part of that story and like where does that where does it go so you know we're famously described often as a biological Bootloader for AIS and that's because humans I mean you know we're an incredible uh biological system and we're capable of computation and uh you know and love and so on um but we're extremely inefficient as well like we're talking to each other through audio it's just kind of embarrassing honestly they were manipulating like seven symbols uh serially we're using vocal chords it's all happening over like multiple seconds yeah it's just like kind of embarrassing when you step down to the uh frequencies at which computers operate or are able to cooperate on and so basically it does seem like um synthetic intelligences are kind of like the next stage of development and um I don't know where it leads to like at some point I suspect uh the universe is some kind of a puzzle and these synthetic AIS will uncover that puzzle and um solve it and then what happens after right like what because if you just like Fast Forward Earth many billions of years it's like uh it's quiet and then it's like to tourmal you see like city lights and stuff like that and then what happens like at the end like is it like a is it or is it like a calming is it explosion is it like Earth like open like a giant because you said emit Roasters like well let's start emitting like like a giant number of Like Satellites yes it's some kind of a crazy explosion and we're living we're like we're stepping through a explosion and we're like living day to day and it doesn't look like it but it's actually if you I saw a very cool animation of Earth uh and life on Earth and basically nothing happens for a long time and then the last like two seconds like basically cities and everything and just in the low earth orbit just gets cluttered and just the whole thing happens in the last two seconds and you're like this is exploding this is a statement explosion so if you play yeah yeah if you play it at normal speed yeah it'll just look like an explosion it's a firecracker we're living in a firecracker where it's going to start emitting all kinds of interesting things yeah and then so explosion doesn't it might actually look like a little explosion with with lights and fire and energy emitted all that kind of stuff but when you look inside the details of the explosion there's actual complexity happening where there's like uh yeah human life or some kind of life we hope it's not destructive firecracker it's kind of like a constructive uh firecracker all right so given that I think uh hilarious disgusting it is a really interesting to think about like what the puzzle of the universe is that the creator of the universe uh give us a message like for example in the book contact UM Carl Sagan uh there's a message for Humanity for any civilization in uh digits in the expansion of Pi and base 11 eventually which is kind of interesting thought uh maybe maybe we're supposed to be giving a message to our creator maybe we're supposed to somehow create some kind of a quantum mechanical system that alerts them to our intelligent presence here because if you think about it from their perspective it's just say like Quantum field Theory massive like cellular automaton like thing and like how do you even notice that we exist you might not even be able to pick us up in that simulation and so how do you uh how do you prove that you exist that you're intelligent and that you're a part of the universe so this is like a touring test for intelligence from Earth yeah the Creator is uh I mean maybe this is uh like trying to complete the next word in a sentence this is a complicated way of that like Earth is just is basically sending a message back yeah the puzzle is basically like alerting the Creator that we exist or maybe the puzzle is just to just break out of the system and just uh you know stick it to the Creator in some way uh basically like if you're playing a video game you can um you can somehow find an exploit and find a way to execute on the host machine in the arbitrary code there's some for example I believe someone got Mario a game of Mario to play Pong just by uh exploiting it and then um creating a basically writing writing code and being able to execute arbitrary code in the game and so maybe we should be maybe that's the puzzle is that we should be um uh find a way to exploit it so so I think like some of these synthetic ads will eventually find the universe to be some kind of a puzzle and then solve it in some way and that's kind of like the end game somehow do you often think about it as a as a simulation so as the universe being a kind of computation that has might have bugs and exploits yes yeah I think so is that what physics is essentially I think it's possible that physics has exploits and we should be trying to find them arranging some kind of a crazy quantum mechanical system that somehow gives you buffer overflow somehow gives you a rounding error and a floating Point uh yeah that's right and like more and more sophisticated exploits like those are jokes but that could be actually very close yeah we'll find some way to extract infinite energy for example when you train a reinforcement learning agents um and physical simulations and you ask them to say run quickly on the flat ground they'll end up doing all kinds of like weird things um in part of that optimization right they'll get on their back leg and they will slide across the floor and it's because of the optimization the enforcement learning optimization on that agent has figured out a way to extract infinite energy from the friction forces and basically their poor implementation and they found a way to generate infinite energy and just slide across the surface and it's not what you expected it's just a it's sort of like a perverse solution and so maybe we can find something like that maybe we can be that little dog in this physical simulation the the cracks or escapes the intended consequences of the physics that the Universe came up with we'll figure out some kind of shortcut to some weirdness yeah and then oh man but see the problem with that weirdness is the first person to discover the weirdness like sliding in the back legs that's all we're going to do yeah it's very quickly because everybody does that thing so like the the paper clip maximizer is a ridiculous idea but that very well you know could be what then we'll just uh we'll just all switch that because it's so fun well no person will Discover it I think by the way I think it's going to have to be uh some kind of a super intelligent AGI of a third generation like we're building the first generation AGI you know third generation yeah so the the Bootloader for an AI the that AI yeah will be a Bootloader for another AI yeah and then there's no way for us to introspect like what that might even uh I think it's very likely that these things for example like say you have these agis it's very like for example they will be completely inert I like these kinds of sci-fi books sometimes where these things are just completely inert they don't interact with anything and I find that kind of beautiful because uh they probably they've probably figured out the meta game of the universe in some way potentially they're they're doing something completely beyond our imagination um and uh they don't interact with simple chemical life forms like why would you do that so I find those kinds of ideas compelling what's their source of fun what are they what are they doing what's the source of solving in the universe but inert so can you define what it means inert so they escape as in um they will behave in some very like strange way to us because they're uh they're beyond they're playing The Meta game uh and The Meta game is probably say like arranging quantum mechanical systems in some very weird ways to extract Infinite Energy uh solve the digital expansion of Pi to whatever amount uh they will build their own like little Fusion reactors or something crazy like they're doing something Beyond Comprehension and uh not understandable to us and actually brilliant under the hood what if quantum mechanics itself is the system and we're just thinking it's physics but we're really parasites on on or not parasite we're not really hurting physics we're just living on this organisms this organism and we're like trying to understand it but really it is an organism and with a deep deep intelligence maybe physics itself is uh the the organism that's doing a super interesting thing and we're just like one little thing yeah ant sitting on top of it trying to get energy from it we're just kind of like these particles in a wave that I feel like is mostly deterministic and takes uh Universe from some kind of a big bang to some kind of a super intelligent replicator some kind of a stable point in the universe given these laws of physics you don't think uh as Einstein said God doesn't play dice so you think it's mostly deterministic there's no Randomness in the thing I think it's deterministic oh there's tons of uh well I'm I want to be careful with Randomness pseudo random yeah I don't like random uh I think maybe the laws of physics are deterministic um yeah I think they're determinants just got really uncomfortable with this question do you have anxiety about whether the universe is random or not what's there's no Randomness uh you say you like Goodwill Hunting it's not your fault Andre it's not it's not your fault man um so you don't like Randomness uh yeah I think it's uh unsettling I think it's a deterministic system I think that things that look random like say the uh collapse of the wave function Etc I think they're actually deterministic just entanglement uh and so on and uh some kind of a Multiverse Theory something something okay so why does it feel like we have a free will like if I raise the hand I chose to do this now um what that doesn't feel like a deterministic thing it feels like I'm making a choice it feels like it okay so it's all feelings it's just feelings yeah so when an RL agent is making a choice is that um it's not really making a choice the choices are all already there yeah you're interpreting the choice and you're creating a narrative for or having made it yeah and now we're talking about the narrative it's very meta looking back what is the most beautiful or surprising idea in deep learning or AI in general that you've come across you've seen this field explode and grow in interesting ways just what what cool ideas like like we made you sit back and go hmm small big or small well the one that I've been thinking about recently the most probably is the the Transformer architecture um so basically uh neural networks have a lot of architectures that were trendy have come and gone for different sensory modalities like for Vision Audio text you would process them with different looking neural nuts and recently we've seen these convergence towards one architecture the Transformer and you can feed it video or you can feed it you know images or speech or text and it just gobbles it up and it's kind of like a bit of a general purpose uh computer that is also trainable and very efficient to run on our Hardware and so uh this paper came out in 2016 I want to say um attention is all you need attention is all you need you criticize the paper title in retrospect that it wasn't um it didn't foresee the bigness of the impact yeah that it was going to have yeah I'm not sure if the authors were aware of the impact that that paper would go on to have probably they weren't but I think they were aware of some of the motivations and design decisions beyond the Transformer and they chose not to I think expand on it in that way in the paper and so I think they had an idea that there was more um than just the surface of just like oh we're just doing translation and here's a better architecture you're not just doing translation this is like a really cool differentiable optimizable efficient computer that you've proposed and maybe they didn't have all of that foresight but I think is really interesting isn't it funny sorry to interrupt that title is memeable that they went for such a profound idea they went with the I don't think anyone used that kind of title before right protection is all you need yeah it's like a meme or something exactly it's not funny that one like uh maybe if it was a more serious title it wouldn't have the impact honestly I yeah there is an element of me that honestly agrees with you and prefers it this way yes if it was two grand it would over promise and then under deliver potentially so you want to just uh meme your way to greatness that should be a t-shirt so you you tweeted the Transformers the Magnificent neural network architecture because it is a general purpose differentiable computer it is simultaneously expressive in the forward pass optimizable via back propagation gradient descent and efficient High parallelism compute graph can you discuss some of those details expressive optimizable efficient yeah for memory or or in general whatever comes to your heart you want to have a general purpose computer that you can train on arbitrary problems like say the task of next word prediction or detecting if there's a cat in the image or something like that and you want to train this computer so you want to set its weights and I think there's a number of design criteria that sort of overlap in the Transformer simultaneously that made it very successful and I think the authors were kind of uh deliberately trying to make this really powerful architecture and um so basically it's very powerful in the forward pass because it's able to express um very general computation as a sort of something that looks like message passing you have nodes and they all store vectors and these nodes get to basically look at each other and it's each other's vectors and they get to communicate and basically notes get to broadcast hey I'm looking for certain things and then other nodes get to broadcast hey these are the things I have those are the keys and the values so it's not just the tension yeah exactly Transformer is much more than just the attention component it's got many pieces architectural that went into it the residual connection of the way it's arranged there's a multi-layer perceptron in there the way it's stacked and so on um but basically there's a message passing scheme where nodes get to look at each other decide what's interesting and then update each other and uh so I think the um when you get to the details of it I think it's a very expressive function uh so it can express lots of different types of algorithms and forward paths not only that but the way it's designed with the residual connections layer normalizations the softmax attention and everything it's also optimizable this is a really big deal because there's lots of computers that are powerful that you can't optimize or they're not easy to optimize using the techniques that we have which is back propagation and gradient and send these are first order methods very simple optimizers really and so um you also need it to be optimizable um and then lastly you want it to run efficiently in the hardware our Hardware is a massive throughput machine like gpus they prefer lots of parallelism so you don't want to do lots of sequential operations you want to do a lot of operations serially and the Transformer is designed with that in mind as well and so it's designed for our hardware and it's designed to both be very expressive in a forward pass but also very optimizable in the backward pass and you said that uh the residual connections support a kind of ability to learn short algorithms fast them first and then gradually extend them longer during training yeah what's what's the idea of learning short algorithms right think of it as a so basically a Transformer is a series of uh blocks right and these blocks have attention and a little multi-layer perceptron and so you you go off into a block and you come back to this residual pathway and then you go off and you come back and then you have a number of layers arranged sequentially and so the way to look at it I think is because of the residual pathway in the backward path the gradients uh sort of flow along it uninterrupted because addition distributes the gradient equally to all of its branches so the gradient from the supervision at the top uh just floats directly to the first layer and the all the residual connections are arranged so that in the beginning during initialization they contribute nothing to the residual pathway um so what it kind of looks like is imagine the Transformer is kind of like a uh python uh function like a death and um you get to do various kinds of like lines of code say you have a hundred layers deep Transformer typically they would be much shorter say 20. so if 20 lines of code then you can do something in them and so think of during the optimization basically what it looks like is first you optimize the first line of code and then the second line of code can kick in and the third line of code can and I kind of feel like because of the residual pathway and the Dynamics of the optimization you can sort of learn a very short algorithm that gets the approximate tensor but then the other layers can sort of kick in and start to create a contribution and at the end of it you're you're optimizing over an algorithm that is 20 lines of code except these lines of code are very complex because it's an entire block of a transformer you can do a lot in there what's really interesting is that this Transformer architecture actually has been a remarkably resilient basically the Transformer that came out in 2016 is the Transformer you would use today except you reshuffle some of the layer norms the layer normalizations have been reshuffled to a pre-norm formulation and so it's been remarkably stable but there's a lot of bells and whistles that people have attached on and try to uh improve it I do think that basically it's a it's a big step in simultaneously optimizing for lots of properties of a desirable neural network architecture and I think people have been trying to change it but it's proven remarkably resilient but I do think that there should be even better architectures potentially but it's uh your you admire the resilience here yeah there's something profound about this architecture that that at least so maybe we can everything can be turned into a uh into a problem that Transformers can solve currently definitely looks like the Transformers taking over Ai and you can feed basically arbitrary problems into it and it's a general differentiable computer and it's extremely powerful and uh this convergence in AI has been really interesting to watch uh for me personally what else do you think could be discovered here about Transformers like what's surprising thing or or is it a stable um I went to stable place is there something interesting we might discover about Transformers like aha moments maybe has to do with memory uh maybe knowledge representation that kind of stuff definitely the Zeitgeist today is just pushing like basically right now this ad guys is do not touch the Transformer touch everything else yes so people are scaling up the data sets making them much much bigger they're working on the evaluation making the evaluation much much bigger and uh um they're basically keeping the architecture unchanged and that's how we've um that's the last five years of progress in AI kind of what do you think about one flavor of it which is language models have you been surprised uh has your sort of imagination been captivated by you mentioned GPT and all the bigger and bigger and bigger language models and uh what are the limits of those models do you think so just let the task of natural language basically the way GPT is trained right is you just download a mass amount of text Data from the internet and you try to predict the next word in a sequence roughly speaking you're predicting will work chunks but uh roughly speaking that's it and what's been really interesting to watch is uh basically it's a language model language models have actually existed for a very long time um there's papers on language modeling from 2003 even earlier can you explain that case what a language model is uh yeah so language model just basically the rough idea is um just predicting the next word in a sequence roughly speaking uh so there's a paper from for example bengio and the team from 2003 where for the first time they were using a neural network to take say like three or five words and predict the um next word and they're doing this on much smaller data sets and the neural net is not a Transformer it's a multiple error perceptron but but it's the first time that a neural network has been applied in that setting but even before neural networks there were language models except they were using engram models so engram models are just a count based models so um if you try to if you start to take two words and predict the third one you just count up how many times you've seen any two word combinations and what came next and what you predict that's coming next is just what you've seen the most of in the training set and so language modeling has been around for a long time neural networks have done language modeling for a long time so really what's new or interesting or exciting is just realizing that when you scale it up with a powerful enough neural net Transformer you have all these emergent properties where basically what happens is if you have a large enough data set of text you are in the task of predicting the next word you are multitasking a huge amount of different kinds of problems you are multitasking understanding of you know chemistry physics human nature lots of things are sort of clustered in that objective it's a very simple objective but actually you have to understand a lot about the world to make that prediction you just said the U word understanding uh are you in terms of chemistry and physics and so on what do you feel like it's doing is it searching for the right context uh in in like what what is it what is the actual process Happening Here Yeah so basically it gets a thousand words and it's trying to predict a thousand at first and uh in order to do that very very well over the entire data set available on the internet you actually have to basically kind of understand the context of of what's going on in there yeah um and uh it's a sufficiently hard problem that you uh if you have a powerful enough computer like a Transformer you end up with uh interesting Solutions and uh you can ask it uh to all do all kinds of things and um it it shows a lot of emerging properties like in context learning that was the big deal with GPT and the original paper when they published it is that you can just sort of uh prompt it in various ways and ask it to do various things and it will just kind of complete the sentence but in the process of just completing the sentence it's actually solving all kinds of really uh interesting problems that we care about do you think it's doing something like understanding like and when we use the word understanding for us humans I think it's doing some understanding it in its weights it understands I think a lot about the world and it has to in order to predict the next word in a sequence so let's train on the data from the internet uh what do you think about this this approach in terms of data sets of using data from the internet do you think the internet has enough structured data to teach AI about human civilization yeah so I think the internet has a huge amount of data I'm not sure if it's a complete enough set I don't know that uh text is enough for having a sufficiently powerful AGI as an outcome um of course there is audio and video and images and all that kind of stuff yeah so text by itself I'm a little bit suspicious about there's a ton of things we don't put in text in writing uh just because they're obvious to us about how the world works and the physics of it and the Things fall we don't put that stuff in text because why would you we share that understanding and so Texas communication medium between humans and it's not a all-encompassing medium of knowledge about the world but as you pointed out we do have video and we have images and we have audio and so I think that that definitely helps a lot but we haven't trained models uh sufficiently uh across both across all those modalities yet so I think that's what a lot of people are interested in but I wonder what that shared understanding of like well we might call Common Sense has to be learned inferred in order to complete the sentence correctly so maybe the fact that it's implied on the internet the model is going to have to learn that not by reading about it by inferring it in the representation so like common sense just like we I don't think we learn common sense like nobody says tells us explicitly we just figure it all out by interacting with the world right so here's a model of reading about the way people interact with the world it might have to infer that I wonder yeah uh you you briefly worked on a project called the world of bits training in our RL system to take actions on the internet versus just consuming the internet like we talked about do you think there's a future for that kind of system interacting with the internet to help the learning yes I think that's probably the uh the final frontier for a lot of these models because um so as you mentioned I was at open AI I was working on this project world of bits and basically it was the idea of giving neural networks access to a keyboard and a mouse and the idea could possibly go wrong so basically you um you perceive the input of the screen pixels and basically the state of the computer is sort of visualized for human consumption in images of the web browser and stuff like that and then you give the neural network the ability to press keyboards and use the mouse and we're trying to get it to for example complete bookings and you know interact with user interfaces and um what did you learn from that experience like what was some fun stuff this is super cool idea yeah I mean it's like uh yeah I mean the the step between Observer to actor yeah is a super fascinating step yeah well the universal interface in the digital realm I would say and there's a universal interface in like the Physical Realm which in my mind is a humanoid form factor kind of thing we can later talk about Optimus and so on but I feel like there's a they're kind of like a similar philosophy in some way where the human the world the physical world is designed for the human form and the digital world is designed for the human form of seeing the screen and using keyword keyboard and mouse and so as the universal interface that can basically uh command the digital infrastructure we've built up for ourselves and so it feels like a very powerful interface to to command and to build on top of now to your question as to like what I learned from that it's interesting because the world of bits was basically uh too early I think at open AI at the time this is around 2015 or so and the Zeitgeist at that time was very different in AI from the Zeitgeist today at the time everyone was super excited about reinforcement learning from scratch this is the time of the Atari paper where uh neural networks were playing Atari games and beating humans in some cases uh alphago and so on so everyone's very excited about train training neural networks from scratch using reinforcement learning um directly it turns out that reinforcement learning is extremely inefficient way of training neural networks because you're taking all these actions and all these observations and you get some sparse rewards once in a while so you do all this stuff based on all these inputs and once in a while you're like told you did a good thing you did a bad thing and it's just an extremely hard problem you can't learn from that you can burn forest and you can sort of Brute Force through it and we saw that I think with uh you know with uh go and DOTA and so on and it does work but it's extremely inefficient uh and not how you want to approach problems uh practically speaking and so that's the approach that at the time we also took to World of bits we would uh have an agent initialize randomly so with keyboard mash and mouse mash and try to make a booking and it's just like revealed the insanity of that approach very quickly where you have to stumble by the correct booking in order to get a reward of you did it correctly and you're never going to stumble by it by chance at random so even with a simple web interface there's too many options there's just too many options uh and uh it's two sparse of reward signal and you're starting from scratch at the time and so you don't know how to read you don't understand pictures images buttons you don't understand what it means to like make a booking but now what's happened is uh it is time to revisit that and open your eyes interested in this uh companies like Adept are interested in this and so on and uh the idea is coming back because the interface is very powerful but now you're not training an agent from scratch you are taking the GPT as an initialization so GPT is pre-trained on all of text and it understands what's a booking it understands what's a submit it understands um quite a bit more and so it already has those representations they are very powerful and that makes all the training significantly more efficient and makes the problem tractable should the interaction be with like the way humans see it with the buttons and the language or it should be with the HTML JavaScript and this and the CSS what's what do you think is the better so today all this interest is mostly on the level of HTML CSS and so on that's done because of computational constraints but I think ultimately everything is designed for human visual consumption and so at the end of the day there's all the additional information is in the layout of the web page and what's next to you and what's a red background and all this kind of stuff and what it looks like visually so I think that's the final frontier as we are taking in pixels and we're giving out keyboard mouse commands but I think it's impractical still today do you worry about bots on the internet given given these ideas given how exciting they are do you worry about bots on Twitter being not the the stupid boss that we see now with the cryptobots but the Bots that might be out there actually that we don't see that they're interacting in interesting ways so this kind of system feels like it should be able to pass the I'm not a robot click button whatever um which you actually understand how that test works I don't quite like there's there's a there's a check box or whatever that you click it's presumably tracking oh I see like Mouse movement and the timing and so on yeah so exactly this kind of system we're talking about should be able to pass that so yeah what do you feel about um Bots that are language models Plus have some interact ability and are able to tweet and reply and so on do you worry about that world uh yeah I think it's always been a bit of an arms race uh between sort of the attack and the defense uh so the attack will get stronger but the defense will get stronger as well our ability to detect that how do you defend how do you detect how do you know that your karpate account on Twitter is is human how do you approach that like if people were claim you know uh how would you defend yourself in the court of law that I'm a human um this account is yeah at some point I think uh it might be I think the society Society will evolve a little bit like we might start signing digitally signing uh some of our correspondents or you know things that we create uh right now it's not necessary but maybe in the future it might be I do think that we are going towards the world where we share we share the digital space with uh AIS synthetic beings yeah and uh they will get much better and they will share our digital realm and they'll eventually share our Physical Realm as well it's much harder uh but that's kind of like the world we're going towards and most of them will be benign and awful and some of them will be malicious and it's going to be an arms race trying to detect them so I mean the worst isn't the AI is the worst is the AIS pretending to be human so mine I don't know if it's always malicious there's obviously a lot of malicious applications but yeah it could also be you know if I was an AI I would try very hard to pretend to be human because we're in a human world yeah I wouldn't get any respect as an AI yeah I want to get some love and respect I don't think the problem is intractable people are people are thinking about the proof of personhood yes and uh we might start digitally signing our stuff and we might all end up having like uh yeah basically some some solution for proof of personhood it doesn't seem to be intractable it's just something that we haven't had to do until now but I think once the need like really starts to emerge which is soon I think when people think about it much more so but that too will be a race because um obviously you can probably uh spoof or fake the the the proof of personhood so you have to try to figure out how to probably I mean it's weird that we have like Social Security numbers and like passports and stuff it seems like it's harder to fake stuff in the physical space than the residual space it just feels like it's going to be very tricky very tricky to out um because it seems to be pretty low cost fake stuff what are you gonna put an AI in jail for like trying to use a fake fake personhood proof you can I mean okay fine you'll put a lot of AIS in jail but there'll be more ai's arbitrary like exponentially more the cost of creating a bot is very low uh unless there's some kind of way to track accurately like you're not allowed to create any program without showing uh tying yourself to that program like you any program that runs on the internet you'll be able to uh Trace every single human program that was involved with that program yeah maybe you have to start declaring when uh you know we have to start drawing those boundaries and keeping track of okay uh what our digital entities versus human entities and uh what is the ownership of human entities and digital entities and uh something like that um I don't know but I think I'm optimistic that this is uh this is uh possible and at some in some sense we're currently in like the worst time of it because um all these Bots suddenly have become very capable but we don't have defenses yet built up as a society and but I think uh that doesn't seem to be intractable it's just something that we have to deal with it seems weird that the Twitter but like really crappy Twitter Bots are so numerous like is it so I presume that the engineers at Twitter are very good so it seems like what I would infer from that uh is it seems like a hard problem it they're probably catching all right if I were to sort of steal them on the case it's a hard problem and there's a huge cost to uh false positive to to removing a post by somebody that's not a bot because creates a very bad user experience so they're very cautious about removing so maybe it's uh and maybe the boss are really good at learning what gets removed and not such that they can stay ahead of the removal process very quickly my impression of it honestly is there's a lot of blowing for it I mean yeah just that's what I it's not subtle it's my impression of it it's not so but you have to yeah that's my impression as well but it feels like maybe you're seeing the the tip of the iceberg maybe the number of bots is in like the trillions and you have to like just it's a constant assault of bots and yeah you yeah I don't know um you have to still man the case because the boss I'm seeing are pretty like obvious I could write a few lines of code that catch these Bots I mean definitely there's a lot of longing fruit but I will say I agree that if you are a sophisticated actor you could probably create a pretty good bot right now um you know using tools like gpts because it's a language model you can generate faces that look quite good now uh and you can do this at scale and so I think um yeah it's quite plausible and it's going to be hard to defend there was a Google engineer that claimed that the Lambda was sentient do you think there's any inkling of Truth to what he felt and more importantly to me at least do you think language models will achieve sentence or the illusion of sentience soonish fish yeah to me it's a little bit of a canary Nicole mine kind of moment honestly a little bit because uh so this engineer spoke to like a chatbot at Google and uh became convinced that uh this bot is sentient yeah as there's some existential philosophical questions and it gave like reasonable answers and looked real and uh and so on so to me it's a uh he was he was uh he wasn't sufficiently trying to stress the system I think and uh exposing the truth of it as it is today um but uh I think this will be increasingly harder over time uh so uh yeah I think more and more people will basically uh become um yeah I think more and more there will be more people like that over time as this gets better like form an emotional connection to to an AI yeah perfectly plausible in my mind I think these AIS are actually quite good at human human connection human emotion a ton of text on the Internet is about humans and connection and love and so on so I think they have a very good understanding in some in some sense of of how people speak to each other about this and um they're very capable of creating a lot of that kind of text the um there's a lot of like sci-fi from 50s and 60s that imagined AIS in a very different way they are calculating cold vulcan-like machines that's not what we're getting today we're getting pretty emotional AIS that actually uh are very competent and capable of generating you know possible sounding text with respect to all of these topics see I'm really hopeful about AI systems that are like companions that help you grow develop as a human being help you maximize long-term happiness but I'm also very worried about AI systems that figure out from the internet the humans get attracted to drama and so these would just be like shit talking AIS that's just constantly did you hear like they'll do gossip they'll do uh they'll try to plant seeds of Suspicion to like other humans that you love and trust and just kind of mess with people uh in the you know because because that's going to get a lot of attention so drama maximize drama on the path to maximizing uh engagement and US humans will feed into that machine yeah and get it'll be a giant drama shitstorm so I'm worried about that so it's the objective function really defines the way that human civilization progresses with AIS in it yeah I think right now at least today they are not sort of it's not correct to really think of them as goal seeking agents that want to do something they have no long-term memory or anything they it's literally a good approximation of it is you get a thousand words and you're trying to predict a thousand at first and then you continue feeding it in and you are free to prompt it in whatever way you want so in text so you say okay you are a psychologist and you are very good and you love humans and here's a conversation between you and another human human colon Something you something and then it just continues the pattern and suddenly you're having a conversation with a fake psychologist who's not trying to help you and so it's still kind of like in a realm of a tool it is a um people can prompt their arbitrary ways and it can create really incredible text but it doesn't have long-term goals over long periods of time it doesn't try to uh so it doesn't look that way right now yeah but you can do short-term goals that have long-term effects so if my prompting short-term goal is to get Andre capacity to respond to me on Twitter when I like I think AI might that's the goal but he might figure out that talking shit to you it would be the best in a highly sophisticating interesting way and then you build up a relationship when you respond once and then it like over time it gets to not be sophisticated and just like just talk shit and okay maybe you won't get to Andre but it might get to another celebrity it might get into other big accounts and then it'll just so with just that simple goal get them to respond yeah maximize the probability of actual response yeah I mean you could prompt a uh powerful model like this with their its opinion about how to do any possible thing you're interested in so they will discuss they're kind of on track to become these oracles I could I sort of think of it that way they are oracles uh currently is just text but they will have calculators they will have access to Google search they will have all kinds of couches and gizmos they will be able to operate the internet and find different information and um yeah in some sense that's kind of like currently what it looks like in terms of the development do you think it'll be an improvement eventually over what Google is for access to human knowledge like it'll be a more effective search engine to access human knowledge I think there's definite scope in building a better search engine today and I think Google they have all the tools all the people they have everything they need they have all the puzzle pieces they have people training Transformers at scale they have all the data uh it's just not obvious if they are capable as an organization to innovate on their search engine right now and if they don't someone else will there's absolute scope for building a significantly better search engine built on these tools it's so interesting a large company where the search there's already an infrastructure it works as it brings out a lot of money so where structurally inside a company is their motivation to Pivot yeah to say we're going to build a new search engine yep that's really hard so it's usually going to come from a startup right that's um that would be yeah or some other more competent organization um so uh I don't know so currently for example maybe Bing has another shot at it you know so Microsoft Edge because we're talking offline um I mean I definitely it's really interesting because search engines used to be about okay here's some query here's here's here's web pages that look like the stuff that you have but you could just directly go to answer and then have supporting evidence um and these uh these models basically they've read all the texts and they've read all the web pages and so sometimes when you see yourself going over to search results and sort of getting like a sense of like the average answer to whatever you're interested in uh like that just directly comes out you don't have to do that work um so they're kind of like uh yeah I think they have a way to this of distilling all that knowledge into like some level of insight basically do you think of prompting as a kind of teaching and learning like this whole process like another layer you know because maybe that's what humans are we already have that background model and then your the world is prompting you yeah exactly I think the way we are programming these computers now like gpts is is converging to how you program humans I mean how do I program humans via prompt I go to people and I I prompt them to do things I prompt them from information and so uh natural language prompt is how we program humans and we're starting to program computers directly in that interface it's like pretty remarkable honestly so you've spoken a lot about the idea of software 2.0 um all good ideas become like cliches so quickly like the terms it's kind of hilarious um it's like I think Eminem once said that like if he gets annoyed by a song He's written very quickly that means it's going to be a big hit because it's it's too catchy but uh can you describe this idea and how you're thinking about it has evolved over the months and years since since you coined it yeah yeah so I had a blog post on software 2.0 I think several years ago now um and the reason I wrote that post is because I kept I kind of saw something remarkable happening in like software development and how a lot of code was being transitioned to be written not in sort of like C plus and so on but it's written in the weights of a neural net basically just saying that neural Nets are taking over software the realm of software and uh taking more and more tasks and at the time I think not many people understood uh this uh deeply enough that this is a big deal it's a big transition uh neural networks were seen as one of multiple classification algorithms you might use for your data set problem on kaggle like this is not that this is a change in how we program computers and I saw neural Nets as uh this is going to take over the way we program computers is going to change is not going to be people writing a software in C plus or something like that and directly programming the software it's going to be accumulating training sets and data sets and crafting these objectives by which we train these neural Nets and at some point there's going to be a compilation process from the data sets and the objective and the architecture specification into the binary which is really just uh the neural nut you know weights and the forward pass of the neural net and then you can deploy that binary and so I was talking about that sort of transition and uh that's what the post is about and I saw this sort of play out in a lot of fields uh you know autopilot being one of them but also just a simple image classification people thought originally you know in the 80s and so on that they would write the algorithm for detecting a dog in an image and they had all these ideas about how the brain does it and first we detected corners and then we detect lines and then we stitched them up and they were like really going at it they were like thinking about how they're going to write the algorithm and this is not the way you build it and there was a smooth transition where okay first we thought we were going to build everything then we were building the features uh so like Hawk features and things like that that detect these little statistical patterns from image patches and then there was a little bit of learning on top of it like a support Vector machine or binary classifier for cat versus dog and images on top of the features so we wrote the features but we trained the last layer sort of the the classifier and then people are like actually let's not even design the features because we can't honestly we're not very good at it so let's also learn the features and then you end up with basically a convolutional neural net where you're learning most of it you're just specifying the architecture and the architecture has tons of fill in the blanks which is all the knobs and you let the optimization write most of it and so this transition is happening across the industry everywhere and uh suddenly we end up with a ton of code that is written in neural net weights and I was just pointing out that the analogy is actually pretty strong and we have a lot of developer environments for software 1.0 like we have Ides um how you work with code how you debug code how do you how do you run code how do you maintain code we have GitHub so I was trying to make those analogies in the new realm like what is the GitHub or software 2.0 it turns out that something that looks like hugging face right now uh you know and so I think some people took it seriously and built cool companies and uh many people originally attacked the post it actually was not well received when I wrote it and I think maybe it has something to do with the title but the post was not well received and I think more people sort of have been coming around to it over time yeah so you were the director of AI at Tesla where I think this idea was really implemented at scale which is how you have engineering teams doing software 2.0 so can you sort of Linger on that idea of I think we're in the really early stages of everything you just said which is like GitHub Ides like how do we build engineering teams that that work in software 2.0 systems and and the the data collection and the data annotation which is all part of that software 2.0 like what do you think is the task of programming a software 2.0 is it debugging in the space of hyper parameters or is it also debugging the space of data yeah the way by which you program the computer and influence its algorithm is not by writing the commands yourself you're changing mostly the data set uh you're changing the um loss functions of like what the neural net is trying to do how it's trying to predict things but yeah basically the data sets and the architectures of the neural net and um so in the case of the autopilot a lot of the data sets have to do with for example detection of objects and Lane line markings and traffic lights and so on So You accumulate massive data sets of here's an example here's the desired label and then uh here's roughly how the architect here's roughly what the algorithm should look like and that's a conclusional neural net so the specification of the architecture is like a hint as to what the algorithm should roughly look like and then to fill in the blanks process of optimization is the training process and then you take your neural nut that was trained it gives all the right answers on your data set and you deploy it so there's in that case perhaps it all machine learning cases there's a lot of tasks so is coming up formulating a task like uh for a multi-headed neural network is formulating a task part of the programming yeah very much so how you break down a problem into a set of tasks yeah I'm on a high level I would say if you look at the software running in in the autopilot I gave a number of talks on this topic I would say originally a lot of it was written in software 1.0 there's imagine lots of C plus plus all right and then gradually there was a tiny neural net that was for example predicting given a single image is there like a traffic light or not or is there a landline marking or not and this neural net didn't have too much to do in this in the scope of the software it was making tiny predictions on individual little image and then the rest of the system stitched it up so okay we're actually we don't have just a single camera with eight cameras we actually have eight cameras over time and so what do you do with these predictions how do you put them together how do you do the fusion of all that information and how do you act on it all of that was written by humans um in C plus and then we decided okay we don't actually want uh to do all of that Fusion in C plus code because we're actually not good enough to write that algorithm we want the neural Nets to write the algorithm and we want to Port uh all of that software into the 2.0 stack and so then we actually had neural Nets that now take all the eight camera images simultaneously and make predictions for all of that so um and and actually they don't make predictions in a in the space of images they now make predictions directly in 3D and actually they don't in three dimensions around the car and now actually we don't um manually fuse the predictions over in 3D over time we don't trust ourselves to write that tracker so actually we give the neural net uh the information over time so it takes these videos now and makes those predictions and so your sort of just like putting more and more power into the neural network processing and at the end of it the eventual sort of goal is to have most of the software potentially be in the 2.0 land um because it works significantly better humans are just not very good at writing software basically so the prediction is space happening in this like 4D land yeah was three-dimensional world over time yeah how do you do annotation in that world what what have you as it's just a data annotation whether it's self-supervised or manual by humans is um is a big part of this software 2.0 world right I would say by far in the industry if you're like talking about the industry and how what is the technology of what we have available everything is supervised learning so you need data sets of input desired output and you need lots of it and um there are three properties of it that you need you need it to be very large you need it to be accurate No mistakes and you need it to be diverse you don't want to uh just have a lot of correct examples of one thing you need to really cover the space of possibility as much as you can and the more you can cover the space of possible inputs the better the algorithm will work at the end now once you have really good data sets that you're collecting curating um and cleaning you can train uh your neural net um on top of that so a lot of the work goes into cleaning those data sets now as you pointed out it's probably it could be the question is how do you achieve a ton of uh if you want to basically predict in 3D you need data in 3D to back that up so in this video we have eight videos coming from all the cameras of the system and this is what they saw and this is the truth of what actually was around there was this car there was this car this car these are the lane line markings this is geometry of the road there's a traffic light in this three-dimensional position you need the ground truth um and so the big question that the team was solving of course is how do you how do you arrive at that ground truth because once you have a million of it and it's large clean and diverse then training a neural network on it works extremely well and you can ship that into the car and uh so there's many mechanisms by which we collected that training data you can always go for human annotation you can go for simulation as a source of ground truth you can also go for what we call the offline tracker um that we've spoken about at the AI day and so on which is basically an automatic reconstruction process for taking those videos and recovering the three-dimensional sort of reality of what was around that car so basically think of doing like a three-dimensional reconstruction as an offline thing and then understanding that okay there's 10 seconds of video this is what we saw and therefore here's all the lane last cars and so on and then once you have that annotation you can train your neural Nets to imitate it and how difficult is the reconstruct the 3D reconstruction it's difficult but it can be done so there's so the there's overlap between the cameras and you do the Reconstruction and there's uh perhaps if there's any inaccuracy so that's caught in The annotation step uh yes the nice thing about The annotation is that it is fully offline you have infinite time you have a chunk of one minute and you're trying to just offline in a super computer somewhere figure out where were the positions of all the cars all the people and you have your full one minute of video from all the Angles and you can run all the neural Nets you want and they can be very efficient massive neural Nets there can be neural Nets that can't even run in the car later at this time so they can be even more powerful neurons than what you can eventually deploy so you can do anything you want three-dimensional reconstruction neural Nets uh anything you want just to recover that truth and then you supervise that truth what have you learned you said no mistakes about humans doing annotation because I assume humans are uh there's like a range of things they're good at in terms of clicking stuff on screen it's not how interesting is that to you of a problem of designing an annotator where humans are accurate enjoy it like what are they even the metrics are efficient or productive all that kind of stuff yeah so uh I grew The annotation team at Tesla from basically zero to a thousand uh while I was there that was really interesting you know my background is a PhD student researcher so growing that common organization was pretty crazy uh but uh yeah I think it's extremely interesting and part of the design process very much behind the autopilot as to where you use humans humans are very good at certain kinds of annotations they're very good for example at two-dimensional annotations of images they're not good at annotating uh cars over time in three-dimensional space very very hard and so that's why we were very careful to design the tasks that are easy to do for humans versus things that should be left to the offline tracker like maybe the maybe the computer will do all the triangulation and 3D reconstruction but the human will say exactly these pixels of the image are car exactly these pixels are human and so co-designing the the data annotation pipeline was very much bread and butter was what I was doing daily do you think there's still a lot of open problems in that space um just in general annotation where the stuff the machines are good at machines do and the humans do what they're good at and there's maybe some iterative process right I think to a very large extent we went through a number of iterations and we learned a ton about how to create these data sets I'm not seeing big open problems like originally when I joined I was like I was really not sure how this would turn out yeah but by the time I left I was much more secure in actually we sort of understand the philosophy of how to create these data sets and I was pretty comfortable with where that was at the time so what are strengths and limitations of cameras for the driving test in your understanding when you formulate the driving task as a vision task with eight cameras you've seen that the entire you know most of the history of the computer vision field when it has to do with neural networks what just if you step back what are the strengths and limitations of pixels of using pixels to drive yeah pixels I think are a beautiful sensory beautiful sensor I would say the thing is like cameras are very very cheap and they provide a ton of information ton of bits uh so it's uh extremely cheap sensor for a ton of bits and each one of these bits as a constraint on the state of the world and so you get lots of megapixel images uh very cheap and it just gives you all these constraints for understanding what's actually out there in the world so vision is probably the highest bandwidth sensor it's a very high bandwidth sensor and um I love that pixels it is a is a constraint on the world This is highly complex uh high bandwidth constraint in the world on the stage of the world that's fascinating it's not just that but again this real real importance of it's the sensor that humans use therefore everything is designed for that sensor yeah the text the writing the flashing signs everything is designed for vision and so and you just find it everywhere and so that's why that is the interface you want to be in um talking again about these Universal interfaces and uh that's where we actually want to measure the world as well and then develop software uh for that sensor but there's other constraints on the state of the world that humans use to understand the world I mean Vision ultimately is the main one but we're like we're like referencing our understanding of human behavior and some common sense physics that could be inferred from vision from from a perception perspective but it feels like we're using some kind of reasoning to predict the world yeah not just the pixels I mean you have a powerful prior uh sorry right for how the world evolves over time Etc so it's not just about the likelihood term coming up from the data itself telling you about what you are observing but also the prior term of like where where are the likely things to see and how do they likely move and so on and the question is how complex is the uh the the range of possibilities that might happen in the driving task right that's still is is that to you still an open problem of how difficult is driving like philosophically speaking like do you all the time you've worked on driving do you understand how hard driving is yeah driving is really hard because it has to do with the predictions of all these other agents and the theory of mind and you know what they're gonna do and are they looking at you are they where are they looking what are they thinking yeah there's a lot that goes there at the at the full tail of you know the the expansion of the nines that we have to be comfortable with eventually the final problems are of that form I don't think those are the problems that are very common uh I think eventually they're important but it's like really in the tail end in the tail and the rare edge cases from the vision perspective what are the toughest parts of the vision problem of driving um well basically the sensor is extremely powerful but you still need to process that information um and so going from brightnesses of these pixel values to hey here the three-dimensional world is extremely hard and that's what the neural networks are fundamentally doing and so um the difficulty really is in just doing an extremely good job of engineering the entire pipeline uh the entire data engine having the capacity to train these neural nuts having the ability to evaluate the system and iterate on it uh so I would say just doing this in production at scale is like the hard part it's an execution problem so the data engine but also the um the sort of deployment of the system such that has low latency performance so it has to do all these steps yeah for the neural net specifically just making sure everything fits into the chip on the car yeah and uh you have a finite budget of flops that you can perform and uh and memory bandwidth and other constraints and you have to make sure it flies and you can squeeze in as much compute as you can into the tiny what have you learned from that process because it maybe that's one of the bigger like new things coming from a research background where there's there's a system that has to run under heavily constrained resources right has to run really fast what what kind of insights have you uh learned from that yeah I'm not sure if it's if there's too many insights you're trying to create a neural net that will fit in what you have available and you're always trying to optimize it and we talked a lot about it on the AI day and uh basically the the triple backflips that the team is doing to make sure it all fits and utilizes the engine uh so I think it's extremely good engineering um and then there's also all kinds of little insights peppered in on how to do it properly let's actually zoom out because I don't think we talked about the data engine the entirety of the layout of this idea that I think is just beautiful with humans in the loop can you describe the data engine yeah the data engine is what I call the almost biological feeling like process by which you uh perfect the training sets for these neural networks um so because most of the programming now is in the level of these data sets and make sure they're large diverse and clean oh basically you have a data set that you think is good you train your neural net you deploy it and then you observe how well it's performing and you're trying to uh always increase the quality of your data set so you're trying to catch scenarios basically there are basically rare and uh it is in these scenarios that the neural Nets will typically struggle in because they weren't told what to do in those rare cases in the data set but now you can close the loop because if you can now collect all those at scale you can then feed them back into the Reconstruction process I described and uh reconstruct the truth in those cases and add it to the data set and so the whole thing ends up being like a staircase of improvement of perfecting your training set and you have to go through deployments so that you can mine uh the parts that are not yet represented well in the data set so your data set is basically imperfect it needs to be diverse it has pockets there are missing and you need to pad out the pockets you can sort of think of it that way in the data what role do humans play in this so what's the uh this biological system like a human body is made up of cells what what role like how do you optimize the human uh system the the multiple Engineers collaborating figuring out what to focus on what to contribute which which task to optimize in this neural network uh who's in charge of figuring out which task needs more data can you speak to the hyper parameters the human uh system right it really just comes down to extremely good execution from an engineering team and does what they're doing they understand intuitively the philosophical insights underlying the data engine and the process by which the system improves and uh how to again like delegate the strategy of the data collection and how that works and then just making sure it's all extremely well executed and that's where most of the work is is not even the philosophizing or the research or the ideas of it it's just extremely good execution it's so hard when you're dealing with data at that scale so your role in the data engine executing well on it it is difficult and extremely important is there a priority of like uh like a vision board of saying like we really need to get better at stop lights yeah like the the prioritization of tasks is that essentially and that comes from the data that comes to um a very large extent to what we are trying to achieve in the product for a map where we're trying to the release we're trying to get out um in the feedback from the QA team worth it where the system is struggling or not the things we're trying to improve and the QA team gives some signal some information in aggregate about the performance of the system in various conditions and then of course all of us drive it and we can also see it it's really nice to work with the system that you can also experience yourself you know it drives you home it's is there some insight you can draw from your individual experience that you just can't quite get from an aggregate statistical analysis of data yeah it's so weird right yes it's it's not scientific in a sense because you're just one anecdotal sample yeah I think there's a ton of uh it's a source of truth it's your interaction with the system yeah and you can see it you can play with it you can perturb it you can get a sense of it you have an intuition for it I think numbers just like have a way of numbers and plots and graphs are you know much harder yeah it hides a lot of it's like if you train a language model it's a really powerful way is by you interacting with it yeah 100 try to build up an intuition yeah I think like Elon also like he always wanted to drive the system himself he drives a lot and uh I'm gonna say almost daily so uh he also sees this as a source of Truth you driving the system uh and it performing and yeah so what do you think tough questions here uh so Tesla last year removed radar from um from the sensor suite and now just announced that it's going to remove all ultrasonic sensors relying solely on Vision so camera only does that make the perception problem harder or easier I would almost reframe the question in some way so the thing is basically you would think that additional sensors by the way can I just interrupt good I wonder if a language model will ever do that if you prompt it let me reframe your question that would be epic this is the wrong problem sorry it's like a little bit of a wrong question because basically you would think that these sensors are an asset to you yeah but if you fully consider the entire product in its entirety these sensors are actually potentially reliability because these sensors aren't free they don't just appear on your car you need something you need to have an entire supply chain you have people procuring it there can be problems with them they may need replacement they are part of the manufacturing process they can hold back the line in production you need to Source them you need to maintain them you have to have teams that write the firmware all of it and then you also have to incorporate and fuse them into the system in some way and so it actually like bloats the organ the a lot of it and I think Elon is really good at simplify simplified best part is no part and he always tries to throw away things that are not essential because he understands the entropy in organizations and approach and I think uh in this case the cost is high and you're not potentially seeing it if you're just a computer vision engineer and I'm just trying to improve my network and you know is it more useful or less useful how useful is it and the thing is if once you consider the full cost of a sensor it actually is potentially a liability and you need to be really sure that it's giving you extremely useful information in this case we looked at using it or not using it and the Delta was not massive and so it's not useful is it also blow in the data engine like having more sensors is a distraction and these sensors you know they can change over time for example you can have one type of say radar you can have other type of radar they change over time I suddenly need to worry about it now suddenly you have a column in your sqlite telling you oh which sensor type was it and they all have different distributions and then uh they can they just they contribute noise and entropy into everything and they bloat stuff and also organizationally has been really fascinating to me that it can be very distracting um if you if all if you only want to get to work is Vision all the resources are on it and you're building out a data engine and you're actually making forward progress because that is the the sensor with the most bandwidth the most constraints on the world and you're investing fully into that and you can make that extremely good if you're uh you're only a finite amount of sort of spend of focus across different facets of the system and uh this kind of reminds me of Rich Sutton's a bitter lesson it just seems like simplifying the system yeah in the long run now of course you don't know what the long run it seems to be always the right solution yeah yes in that case it was 4rl but it seems to apply generally across all systems that do computation yeah so where uh what do you think about the lidar as a crutch debate uh the battle between point clouds and pixels yeah I think this debate is always like slightly confusing to me because it seems like the actual debate should be about like do you have the fleet or not that's like the really important thing about whether you can achieve a really good functioning of an AI system at this scale so data collection systems yeah do you have a fleet or not it's significantly more important whether you have lidar or not it's just another sensor um and uh yeah I think similar to the radar discussion basically I um but yeah I don't think it it um basically doesn't offer extra extra information is extremely costly it has all kinds of problems you have to worry about it you have to calibrate it Etc it creates bloat and entropy you have to be really sure that you need this uh this um sensor in this case I basically don't think you need it and I think honestly I will make a stronger statement I think the others some of the other uh companies are using it are probably going to drop it yeah so you have to consider the sensor in the full in considering can you build a big Fleet that collects a lot of data and can you integrate that sensor with that that data and that sensor into a data engine that's able to quickly find different parts of the data that then continuously improves whatever the model that you're using yeah another way to look at it is like vision is necessary in a sense that uh the drive the world is designed for human visual consumption so you need vision is necessary and then also it is sufficient because it has all the information that you that you need for driving and humans obviously is a vision to drive so it's both necessary and sufficient so you want to focus resources and you have to be really sure if you're going to bring in other sensors you could you could you could add sensors to Infinity at some point you need to draw the line and I think in this case you have to really consider the full cost of any One sensor that you're adopting and do you really need it and I think the answer in this case is no so what do you think about the idea of the that the other companies are forming high resolution maps and constraining heavily the geographic regions in which they operate is that approach not in your in your view um not going to scale over time to the entirety of the United States I think I'll take two as you mentioned like they pre-map all the environments and they need to refresh the map and they have a perfect centimeter level accuracy map of everywhere they're going to drive it's crazy how are you going to when we're talking about autonomy actually changing the world we're talking about the deployment on a on a global scale of autonomous systems for transportation and if you need to maintain a centimeter accurate map for Earth or like for many cities and keep them updated it's a huge dependency that you're taking on huge dependency it's a massive massive dependency and now you need to ask yourself do you really need it and humans don't need it um right so it's it's very useful to have a low-level map of like okay the connectivity of your road you know that there's a fork coming up when you drive an environment you sort of have that high level understanding it's like a small Google Map and Tesla uses Google Map like similar kind of resolution information in the system but it will not pre-map environments to send me a level accuracy it's a crutch it's a distraction it costs entropy and it diffuses the team it dilutes the team and you're not focusing on what's actually necessary which is the computer vision problem what did you learn about machine learning about engineering about life about yourself as one human being from working with Elon Musk I think the most I've learned is about how to sort of run organizations efficiently and how to create efficient organizations and how to fight entropy in an organization so human Engineering in the fight against entropy yeah there's a there's a I think Elon is a very efficient warrior in the fight against entropy in organizations what is the entropy in an organization look like exactly it's process it's it's process and inefficiencies and that kind of stuff yeah meetings he hates meetings he keeps telling people to skip meetings if they're not useful um he basically runs the world's biggest uh startups I would say uh Tesla SpaceX are the world's biggest startups Tesla actually has multiple startups I think it's better to look at it that way and so I think he's he's extremely good at uh at that and uh yeah he's a very good intuition for streamline processes making everything efficient uh best part is no part uh simplifying focusing um and just kind of removing barriers uh moving very quickly making big moves all this is a very startupy sort of seeming things but at scale so strong drive to simplify for me from your perspective I mean that um that also probably applies to just designing systems and machine learning and otherwise yeah like simplify simplify yes what do you think is the secret to maintaining the startup culture in a company that grows is there can you introspect that I do think you need someone in a powerful position with a big hammer like Elon who's like the cheerleader for that idea and ruthless ruthlessly pursues it if no one has a big enough Hammer everything turns into committees democracy within the company uh process talking to stakeholders decision making just everything just crumbles yeah if you have a big person who's also really smart and has a big hammer things move quickly so you said your favorite scene in interstellar is the intense docking scene with the AI and Cooper talking saying uh Cooper what are you doing docking it's not possible no it's necessary such a good line by the way just so many questions there why in AI in that scene presumably is supposed to be able to compute a lot more than the human is saying it's not optimal why the human I mean that's a movie but shouldn't they AI know much better than the human anyway uh what do you think is the value of setting seemingly impossible goals so like uh our initial intuition which seems like something that you have taken on that Elon espouses that where the initial intuition of the community might say this is very difficult and then you take it on anyway with a crazy deadline you're just from a human engineering perspective um uh have you seen the value of that I wouldn't say that setting impossible goals exactly is is a good idea but I think setting very ambitious goals is a good idea I think there's a what I call sublinear scaling of difficulty uh which means that 10x problems are not 10x hard usually 10x 10x harder problem is like 2 or 3x harder to execute on because if you want to actually like if you want to improve the system by 10 it costs some amount of work and if you want to 10x improve the system it doesn't cost you know 100x amount of the work and it's because you fundamentally change the approach and it if you start with that constraint then some approaches are obviously dumb and not going to work and it forces you to reevaluate um and I think it's a very interesting way of approaching problem solving but it requires a weird kind of thinking it's just going back to your like PhD days it's like how do you think which ideas in in the machine Learning Community are solvable yes it's uh it requires what is that I mean there's the cliche of first prince people's thinking but like it requires to basically ignore what the community is saying because doesn't the community doesn't a community in science usually draw lines of what isn't isn't possible right and like it's very hard to break out of that without going crazy yep I mean I think a good example here is you know the Deep learning revolution in some sense because you could be in computer vision at that time when during the Deep learning sort of revolution of 2012 and so on uh you could be improving your computer vision stack by 10 or we can just be saying actually all this is useless and how do I do 10x better computer vision well it's not probably by tuning a hog feature detector I need a different approach um I need something that is scalable going back to uh Richard Sutton's um and understanding sort of like the philosophy of the uh bitter lesson and then being like actually I need a much more scalable system like a neural network that in principle works and then having some deep Believers that can actually execute on that mission and make it work so that's the 10x solution what do you think is the timeline to solve the problem of autonomous driving this still in part an open question yeah I think the tough thing with timelines of self-driving obviously is that no one has created self-driving yeah so it's not like what do you think is a timeline to build this bridge well we've built million Bridges before here's how long that takes it's it you know it's uh no one has built autonomy it's not obvious uh some parts turn out to be much easier than others so it's really hard to forecast you do your best based on trend lines and so on and based on intuition but that's why fundamentally it's just really hard to forecast this no one has even still like being inside of it is hard to uh to do yes some things turn out to be much harder and some things turn out to be much easier do you try to avoid making forecasts because like Elon doesn't avoid them right and heads of car companies in the past have not avoided it either uh Ford and other places have made predictions that we're going to solve at level four driving by 2020 2021 whatever and now they're all kind of Backtrack on that prediction IU as a as an AI person do you free yourself privately make predictions or do they get in the way of like your actual ability to think about a thing yeah I would say like what's easy to say is that this problem is tractable and that's an easy prediction to make extractable it's going to work yes it's just really hard some things turn out to be harder than some things turn out to be easier uh so uh but it definitely feels tractable and it feels like at least the team at Tesla which is what I saw internally is definitely on track to that how do you form a uh strong representation that allows you to make a prediction about tractability so like you're the leader of a lot a lot of humans you have to kind of say this is actually possible like how do you build up that intuition it doesn't have to be even driving it could be other tasks it could be um and I wonder what difficult tasks did you work on in your life I mean classification achieving certain just an image that certain level of superhuman level performance yeah expert intuition it's just intuition it's belief so just like thinking about it long enough like studying looking at sample data like you said driving uh my intuition has really flawed on this like I don't have a good intuition about tractability it could be either it could be anything it could be solvable like uh you know the driving task could could be simplified into something quite trivial like uh the solution to the problem would be quite trivial and at scale more and more cars driving perfectly might make the problem much easier Yeah the more cars you have driving like people learn how to drive correctly not correctly but in a way that's more optimal for a heterogeneous system of autonomous and semi-autonomous and manually driven cars that could change stuff then again also I've spent a ridiculous number of hours just staring at pedestrians crossing streets thinking about humans and it feels like the way we use our eye contact it sends really strong signals and there's certain quirks and edge cases of behavior and of course a lot of the fatalities that happen have to do with drunk driving and um both on The Pedestrian side and the driver's side so there's that problem of driving at night and all that kind of yeah so I wonder you know it's like the space of possible solution to autonomous driving includes so many human factor issues that it's almost impossible to predict there could be super clean nice Solutions yeah I would say definitely like to use a game analogy there's some fog of War but you definitely also see the frontier of improvement and you can measure historically how much you've made progress and I think for example at least what I've seen in uh roughly five years at Tesla when I joined it barely kept laying on the highway I think going up from Palo Alto to SF was like three or four interventions anytime the road would do anything geometrically or turn too much it would just like not work and so going from that to like a pretty competent system in five years and seeing what happens also under the hood and what the scale which the team is operating now with respect to data and compute and everything else uh is just a massive progress so there's a you're climbing a mountain and it's fog but you're making a lot of progress fog you're making progress and you see what the next directions are and you're looking at some of the remaining challenges and they're not like uh they're not perturbing you and they're not changing your philosophy and you're not contorting yourself you're like actually these are the things that we still need to do yeah the fundamental components of solving the problems seem to be there for the data engine to the compute to the the computer on the car to the compute for the training all that kind of stuff so you've done uh over the years you've been a test you've done a lot of amazing uh breakthrough ideas and Engineering all of it um from the data engine to The Human Side all of it can you speak to why you chose to leave Tesla basically as I described that ran I think over time during those five years I've kind of uh gotten myself into a little bit of a managerial position most of my days were you know meetings and growing the organization and making decisions about sort of high level strategic decisions about the team and what it should be working on and so on and uh it's kind of like a corporate executive role and I can do it I think I'm okay at it but it's not like fundamentally what I what I enjoy and so I think uh when I joined uh there was no computer vision team because Tesla was just going from the transition of using mobileye a third-party vendor for all of its computer vision to having to build its computer vision system so when I showed up there were two people training deep neural networks and they were training them at a computer at their at their legs like uh kind of basic classification task yeah and so I kind of like grew that into what I think is a fairly respectable deep learning team a massive compute cluster a very good um data annotation organization and uh I was very happy with where that was it became quite autonomous and so I kind of stepped away and I uh you know I'm very excited to do much more technical things again yeah and kind of like we focus on AGI what was this soul searching like because you took a little time off and think like what um how many mushrooms did you take no I'm just uh I mean what what was going through your mind the human lifetime is finite yeah he did a few incredible things you're you're one of the best teachers of AI in the world you're one of the best and I don't mean that I mean that in the best possible way you're one of the best tinkerers in the AI world meaning like understanding the fundamental fundamentals of how something works by building it from scratch and playing with it with the basic intuitions it's like Einstein feinmen were all really good at this kind of stuff like a small example of a thing to to play with it to try to understand it uh so that and obviously now with us that you help build a team of machine learning um uh like engineers and a system that actually accomplishes something in the real world so given all that like what was the soul searching like well it was hard because obviously I love the company a lot and I love I love Elon I love Tesla I want um it was hard to leave I love the team basically um but yeah I think actually I would potentially like interested in revisiting it maybe coming back at some point uh working in Optimus working in AGI at Tesla uh I think Tesla is going to do incredible things it's basically like uh it's a massive large-scale robotics kind of company with a ton of In-House talent for doing really incredible things and I think uh human robots are going to be amazing I think autonomous transportation is going to be amazing all this is happening at Tesla so I think it's just a really amazing organization so being part of it and helping it along I think was very basically I enjoyed that a lot yeah it was basically difficult for those reasons because I love the company uh but you know I'm happy to potentially at some point come back for act two but I felt like at this stage I built the team it felt autonomous and uh I became a manager and I wanted to do a lot more technical stuff I wanted to learn stuff I wanted to teach stuff and uh I just kind of felt like it was a good time for for a change of pace a little bit what do you think is uh the best movie sequel of all time speaking of part two because like because most of them suck in movie sequels yeah and you tweet about movies so just in a tiny tangent is there what's your what was like a favorite movie sequel Godfather Part Two um are you a fan of Godfather because you didn't even tweet or mention the Godfather yeah I don't love that movie I know it hasn't edit that out we're gonna edit out the hate towards the Godfather how dare you just I think I will make a strong statement I don't know why I don't know why but I basically don't like any movie before 1995 something like that didn't you mention Terminator two okay okay that's like uh Terminator 2 was a little bit later 1990 no I think Terminator 2 was a name I like Terminator one as well so okay so like a few exceptions but by and large for some reason I don't like movies before 1995 or something they feel very slow the camera is like zoomed out it's boring it's kind of naive it's kind of weird and also Terminator was very much ahead of its time yes and The Godfather there's like no AGI [Laughter] I mean but you have Good Will Hunting was one of the movies you mentioned and that doesn't have any AGI either I guess that's mathematics yeah I guess occasionally I do enjoy movies that don't feature or like Anchorman that has no that's the increment it's so good I don't understand um speaking of AGI because I don't understand why Will Ferrell is so funny it doesn't make sense it doesn't compute there's just something about him and he's a singular human because you don't get that many comedies these days and I wonder if it has to do about the culture uh or the like the machine of Hollywood or does it have to do with just we got lucky with certain people and comedy it came together because he is a singular human that was a ridiculous tangent I apologize but you mentioned humanoid robot so what do you think about Optimus about Tesla bot do you think we'll have robots in the factory in in the home in 10 20 30 40 50 years yeah I think it's a very hard project I think it's going to take a while but who else is going to build humano robots at scale yeah and I think it is a very good form factor to go after because like I mentioned the the world is designed for humanoid form factor these things would be able to operate our machines they would be able to sit down in chairs uh potentially even drive cars uh basically the world is designed for humans that's the form factor you want to invest into and make work over time uh I think you know there's another school of thought which is okay pick a problem and design a robot to it but actually designing a robot and getting a whole data engine and everything behind it to work is actually an incredibly hard problem so it makes sense to go after General interfaces that uh okay they are not perfect for any one given task but they actually have the generality of just with a prompt with English able to do something across and so I think it makes a lot of sense to go after a general uh interface um in the physical world and I think it's a very difficult project I think it's going to take time but I see no other no other company that can execute on that Vision I think it's going to be amazing like uh basically physical labor like if you think transportation is a large Market try physical labor insane well but it's not just physical labor to me the thing that's also exciting is the social robotics so the the relationship we'll have on different levels with those robots that's why I was really excited to see Optimus like um people have criticized me for the excitement but I've I've worked with uh uh a lot of research Labs that do humanoid legged robots Boston Dynamics unitary a lot there's a lot of companies that do legged robots but that's the the Elegance of the movement is a tiny tiny part of the big picture so integrating the two big exciting things to me about Tesla doing humanoid or any Lego robots is clearly integrating it into the data engine so the the data engine aspect so the actual intelligence for the perception and the and the control and the planning and all that kind of stuff integrating into this huge the fleet that you mentioned right um and then speaking of Fleet the second thing is the mass manufacturers Just knowing uh culturally uh driving towards a simple robot that's cheap to produce at scale yeah and doing that well having experience to do that well that changes everything that's why that's a very different culture and style than Boston Dynamics who by the way those those robots are just the the way they move it's uh like it'll be a very long time before Tesla could achieve the smoothness of movement but that's not what it's about it's it's about uh it's about the entirety of the system like we talked about the data engine and the fleet that's super exciting even the initial sort of models uh but that too was really surprising that in a few months you can get a prototype yep and the reason that happened very quickly is as you alluded to there's a ton of copy based from what's happening in the autopilot yes a lot the amount of expertise that like came out of the Woodworks at Tesla for building the human robot was incredible to see like basically Elon said at one point we're doing this and then next day basically like all these CAD models started to appear and people talk about like the supply chain and Manufacturing and uh people showed up with like screwdrivers and everything like the other day and started to like put together the body and I was like whoa like all these people exist at Tesla and fundamentally building a car is actually not that different from building a robot the same and that is true uh not just for uh the hardware pieces and also let's not forget Hardware not just for a demo but manufacturing of that Hardware at scale is like a whole different thing but for software as well basically this robot currently thinks it's a car uh it's gonna have a midlife crisis at some point it thinks it's a car um some of the earlier demos actually we were talking about potentially doing them outside in the parking lot because that's where all of the computer vision that was like working out of the box instead of like in inside um but all the operating system everything just copy pastes uh computer vision mostly copy paste I mean you have to retrain the neural Nets but the approach and everything in data engine and offline trackers and the way we go about the occupancy tracker and so on everything copy paste you just need to retrain the neural Lots uh and then the planning control of course has to change quite a bit but there's a ton of copy paste from what's happening at Tesla and so if you were to if you were to go with goal of like okay let's build a million human robots and you're not Tesla that's that's a lot to ask if you're a Tesla it's actually like it's not it's not that crazy and then the the follow-up question is and how difficult just like we're driving how difficult is the manipulation task uh such that it can have an impact at scale I think depending on the context the really nice thing about robotics is the um unless you do a manufacturing that kind of stuff is there's more room for error driving is so safety critical and so that and also time critical robot is allowed to move slower which is nice yes I think it's going to take a long time but the way you want to structure the development is you need to say okay it's going to take a long time how can I set up the uh product development roadmap so that I'm making Revenue along the way I'm not setting myself up for a zero one loss function where it doesn't work until it works you don't want to be in that position you want to make it useful almost immediately and then you want to slowly deploy it uh and uh at scale and you want to set up your data engine your improvement Loops the Telemetry the evaluation the harness and everything and you want to improve the product over time incorrectly and you're making Revenue along the way that's extremely important because otherwise you cannot build these these uh large undertakings just like don't make sense economically and also from the point of view of the team working on it they need the dopamine along the way they're not just going to make a promise about this being useful this is going to change the world in 10 years when it works this is not where you want to be you want to be in a place like I think autopilot is today where it's offering increased safety and um and uh convenience of driving today people pay for it people like it people purchase it and then you also have the greater mission that you're working towards and you see that so the dopamine for the team that that was a source of Happiness yes you're deploying this people like it people drive it people pay for it they care about it there's all these YouTube videos your grandma drives it she gives you feedback people like it people engage with it you engage with it huge do uh people that drive Teslas like recognize you and give you love like uh like hey thanks for the for the this nice feature that it's doing yeah I think the tricky thing is like some people really love you some people unfortunately like you're working on something that you think is extremely valuable useful Etc some people do hate you there's a lot of people who like hate me and the team and whatever the whole project and I think they have Tesla drivers uh many cases they're not actually yeah that's that's actually makes me sad about humans or the current the ways that humans interact I think that's actually fixable I think humans want to be good to each other I think Twitter and social media is part of the mechanism that actually somehow makes the negativity more viral but it doesn't deserve like disproportionately uh add of like a viral viral boost yeah negativity but like I wish people would just get excited about uh so suppress some of the jealousy some of the ego and just get excited for others and then there's a Karma aspect to that you get excited for others they'll get excited for you same thing in Academia if you're not careful there's a like a dynamical system there if you if you think of in silos and get jealous of somebody else being successful that actually perhaps counterintuitively uh leads the less productivity of you as a community and you individually I feel like if you keep celebrating others that actually makes you more successful yeah I think people haven't in depending on the industry haven't quite learned that yet yeah some people are also very negative and very vocal so they're very prominently featured but actually there's a ton of people who are cheerleaders but they're silent cheerlead cheerleaders and uh when you talk to people just in the world they will all tell you it's amazing it's great especially like people who understand how difficult it is to get this stuff working like people who have built products and makers entrepreneur entrepreneurs like make making this work and changing something is is incredibly hard those people are more likely to cheerlead you well one of the things that makes me sad is some folks in the robotics Community uh don't do the cheerleading and they should there's uh because they know how difficult it is well they actually sometimes don't know how difficult it is to create a product at scale right they actually deploy in the real world a lot of the development of robots and AI systems is done on very specific small benchmarks um and as opposed to real world conditions yes yeah I think it's really hard to work on robotics in academic setting or AI systems that apply in the real world you you've criticized you uh flourished and loved for time the imagenet the famed image in that data set and I've recently had some words uh of criticism that the academic research ml Community gives a little too much love still to the imagenet or like those kinds of benchmarks can you speak to the strengths and weaknesses of data sets used in machine learning research actually I don't know that I recall the specific instance where I was uh unhappy or criticizing imagenet I think imagenet has been extremely valuable uh it was basically a benchmark that allowed the Deep Learning Community to demonstrate that deep neural networks actually work it was uh there's a massive value in that um so I think imagenet was useful but um basically it's become a bit of an eminist at this point so eminist is like the 228 by 28 grayscale digits there's kind of a joke data set that everyone like just crushes if there's no Papers written on MNS though right maybe they should have strong papers like papers that focus on like how do we learn with a small amount of data that kind of stuff yeah I could see that being helpful but not in sort of like Mainline computer vision research anymore of course I think the way I've heard you somewhere maybe I'm just imagining things but I think you said like image that was a huge contribution to the community for a long time and now it's time to move past those kinds of well image that has been crushed I mean you know the error rates are uh yeah we're getting like 90 accuracy in in one thousand classification way uh prediction and I've seen those images and it's like really high that's really that's really good if I remember correctly the top five error rate is now like one percent or something given your experience with a gigantic real world data set would you like to see benchmarks move in certain directions that the research Community uses unfortunately I don't think academics currently have the next imagenet uh We've obviously I think we've crushed mnist we've basically kind of crushed imagenet uh and there's no next sort of big Benchmark that the entire Community rallies behind and uses um you know for further development of these networks uh yeah what it takes for data set to Captivate the imagination of everybody like where they all get behind it that that could also need like a viral like a leader right you know somebody with popularity I mean that yeah why did image of that take off is there or is it just the accident of History it was the right amount of difficult uh it was the right amount of difficult and simple and uh interesting enough it just kind of like it was it was the right time for that kind of a data set question from Reddit uh what are your thoughts on the role that synthetic data and game engines will play in the future of neural net model development I think um as neural Nets converge to humans uh the value of simulation to neural Nets will be similar to value of simulation to humans so people use simulation for uh people use simulation because they can learn something in that kind of a system and without having to actually experience it um but are you referring to the simulation we're doing our head no sorry simulation I mean like video games or uh you know other forms of simulation for various professionals well so let me push back on that because maybe their simulation that we do in our heads like simulate if I do this what do I think will happen Okay that's like internal simulation yeah internal isn't that what we're doing let's assuming before we act oh yeah but that's independent from like the use of uh simulation in the sense of like computer games or using simulation for training set creation or you know is it independent or is it just Loosely correlated because like uh isn't that useful to do like um counterfactual or like Edge case simulation to like you know what happens if there's a nuclear war what happens if there's you know like those kinds of things yeah that's a different simulation from like Unreal Engine that's how I interpreted the question uh so like simulation of the average case is that what's Unreal Engine what what what what what do you mean by Unreal Engine so simulating a world yeah physics of that world why is that different like because you also can add Behavior to that world and you can try all kinds of stuff right like you could throw all kinds of weird things into it so Unreal Engine is not just about similar I mean I guess it is about submitting the physics of the world it's also doing something with that yeah the graphics the physics and the Agents that you put into the environment and stuff like that yeah see I think you I feel like you said that it's not that important I guess for the future of AI development is that is that correct to interpret you that way uh I think humans use uh simulators for um humans use simulators and they find them useful and so computers will use simulators and find them useful okay so you're saying it's not I I don't use simulators very often I play a video game every once in a while but I don't think I derive any wisdom about my own existence from from those video games it's a momentary escape from reality versus a source of wisdom about reality so I don't so I think that's a very polite way of saying simulation is not that useful yeah maybe maybe not I don't see it as like a fundamental really important part of like training neural Nets currently uh but I think uh as neural Nets become more and more powerful I think you will need fewer examples to train additional behaviors and uh simulation is of course there's a domain Gap in a simulation that's not the real world there's slightly something different but uh with a powerful enough neural net uh you need um The Domain Gap can be bigger I think because neural network will sort of understand that even though it's not the real world it like has all this high level structure that I'm supposed to be able to learn from so then you'll know we'll actually yeah you'll be able to Leverage the synthetic data better yes by closing the get better understanding in which ways this is not real data exactly uh right to do better questions next time that was that was a question but I'm just kidding all right um so is it possible do you think speaking of feminist to construct neural Nets and training processes that require very little data so we've been talking about huge data sets like the internet for training I mean one way to say that is like you said like the querying itself is another level of training I guess and that requires a little data yeah but do you see any uh value in doing research and kind of going down the direction of can we use very little data to train to construct a knowledge base 100 I just think like at some point you need a massive data set and then when you pre-train your massive neural nut and get something that you know is like a GPT or something then you're able to be very efficient at training any arbitrary new task uh so a lot of these gpts you know you can do tasks like sentiment analysis or translation or so on just by being prompted with very few examples here's the kind of thing I want you to do like here's an input sentence here's the translation into German input sentence translation to German input sentence blank and the neural network will complete the translation to German just by looking at sort of the example you've provided and so that's an example of a very few shot uh learning in the activations of the neural net instead of the weights of the neural land and so I think basically uh just like humans neural Nets will become very data efficient at learning any other new task but at some point you need a massive data set to pre-train your network to get that and probably we humans have something like that do we do we have something like that do we have a passive in the background background model constructing thing that just runs all the time in a self-supervised way we're not conscious of it I think humans definitely I mean obviously we have uh we learn a lot during during our life span but also we have a ton of Hardware that helps us initialize initialization coming from sort of evolution and so I think that's also a really big a big component a lot of people in the field I think they just talk about the amounts of like seconds and the you know that a person has lived pretending that this is a table arasa sort of like a zero initialization of a neural net and it's not like you can look at a lot of animals like for example zebras zebras get born and they see and they can run there's zero train data in their lifespan they can just do that so somehow I have no idea how Evolution has found a way to encode these algorithms and these neural net initializations are extremely good to 80 CGS and I have no idea how this works but apparently it's possible because here's a proof by existence there's something magical about going from a single cell to an organism that is born to the first few years of life I kind of like the idea that the reason we don't remember anything about the first few years of our life is that it's a really painful process like it's a very difficult challenging training process yeah like intellectually like and maybe yeah I mean I don't why don't we remember any of that there might be some crazy training going on and the that maybe that's the background model training that uh is is very painful and so it's best for the system once it's trained not to remember how it's constructed I think it's just like the hardware for long-term memory is just not fully developed sure I kind of feel like the first few years of uh of infants is not actually like learning it's brain maturing yeah um we're born premature um and there's a theory along those lines because of the birth canal and the swelling of the brain and so we're born premature and then the first few years we're just the brains maturing and then there's some learning eventually um it's my current view on it what do you think do you think neural Nets can have long-term memory like that approach is something like humans do you think you know do you think there needs to be another meta architecture on top of it to add something like a knowledge base that learns facts about the world and all that kind of stuff yes but I don't know to what extent it will be explicitly constructed um it might take unintuitive forms where you are telling the GPT like hey you have a you have a declarative memory bank to which you can store and retrieve data from and whenever you encounter some information that you find useful just save it to your memory bank and here's an example of something you have retrieved and Heiser how you say it and here's how you load from it you just say load whatever you teach it in text in English and then it might learn to use a memory bank from from that oh so so the neural net is the architecture for the background model the the base thing and then yeah everything else is just on top of this it's not just a text right it's you're giving it gadgets and gizmos so uh you're teaching in some kind of a special Language by which we can it can save arbitrary information and retrieve it at a later time and you're telling about these special tokens and how to arrange them to use these interfaces it's like hey you can use a calculator here's how you use it just do five three plus four one equals and when equals is there uh a calculator will actually read out the answer and you don't have to calculate it yourself and you just like tell it in English this might actually work do you think in that sense gato is interesting the the Deep Mind system that it's not just new language but actually throws it all uh in the same pile images actions all that kind of stuff that's basically what we're moving towards yeah I think so so gato is uh is very much a kitchen sink approach to like um reinforcement learning lots of different environments with a single fixed Transformer model right um I think it's a very sort of early result in that in that realm but I think uh yeah it's along the lines of what I think things will eventually look like right so this is the early days of a system that eventually will look like this like from a rigid Rich sudden perspective yeah I'm not super huge fan of I think all these interfaces that like look very different um I would want everything to be normalized into the same API so for example it's green pixels versus same API instead of having like different world environments at a very different physics and Joint configurations and appearances and whatever and you're having some kind of special tokens for different games that you can plug I'd rather just normalize everything to a single interface so it looks the same to the neural net if that makes sense so it's all going to be pixel based pong in the end I think so okay uh let me ask you about your own personal life a lot of people want to know you're one of the most productive and brilliant people in the history of AI what is a productive day in the life of Andre capathi look like what time do you wake up because imagine um some kind of dance between the average productive day and a perfect productive day so the perfect productive day is the thing we strive towards in the average is kind of what it kind of converges to getting all the mistakes and human eventualities and so on yeah so what times you wake up are you morning person I'm not a morning person I'm a night owl for sure I think stable or not that's semi-stable like a eight or nine or something like that during my PhD it was even later I used to go to sleep usually at 3am I think uh the am hours are are precious and very interesting time to work because everyone is asleep um at 8 AM or 7 A.M the east coast is awake so there's already activity there's already some text messages whatever there's stuff happening you can go in like some news website and there's stuff happening it's distracting uh at 3am everything is totally quiet and so you're not going to be bothered and you have solid chunks of time to do your work um so I like those periods Night Owl by default and then I think like productive time basically um what I like to do is you need you need to like build some momentum on the problem without too much distraction and um you need to load your Ram uh your working memory with that problem and then you need to be obsessed with it when you're taking shower when you're falling asleep you need to be obsessed with the problem and it's fully in your memory and you're ready to wake up and work on it right there so there's a scale of uh is this in a scale temporal scale of a single day or a couple of days a week a month so I can't talk about one day basically in isolation because it's a whole process when I want to get when I want to get productive in the problem I feel like I need a span of a few days where I can really get in on that problem and I don't want to be interrupted and I'm going to just uh be completely obsessed with that problem and that's where I do most of my good workouts you've done a bunch of cool like little projects in a very short amount of time very quickly so that that requires you just focusing on it yeah basically I need to load my working memory with the problem and I need to be productive because there's always like a huge fixed cost to approaching any problem uh you know like I was struggling with this for example at Tesla because I want to work on like small side projects but okay you first need to figure out okay I need to SSH into my cluster I need to bring up a vs code editor so I can like work on this I need to I run into some stupid error because of some reason like you're not at a point where you can be just productive right away you are facing barriers and so it's about uh really removing all that barrier and you're able to go into the problem and you have the full problem loaded in your memory and somehow avoiding distractions of all different forms like uh news stories emails but also distractions from other interesting projects that you previously worked on are currently working on and so on you just want to really focus your mind and I mean I can take some time off for distractions and in between but I think it can't be too much uh you know most of your day is sort of like spent on that problem and then you know I drink coffee I have my morning routine I look at some news uh Twitter Hacker News Wall Street Journal Etc so basically you wake up you have some coffee are you trying to get to work as quickly as possible do you do taking this diet of of like what the hell's happening in the world first I am I do find it interesting to know about the world I don't know that it's useful or good but it is part of my routine right now so I do read through a bunch of news articles and I want to be informed and um I'm suspicious of it I'm suspicious of the practice but currently that's where I am Oh you mean suspicious about the positive effect yeah of that practice on your productivity and your well-being my well-being psychologically uh and also on your ability to deeply understand the world because how there's a bunch of sources of information you're not really focused on deeply integrating yeah it's a little bit distracting or yeah in terms of a perfectly productive day for how long of a stretch of time in one session do you try to work and focus on a thing it's a couple hours is it one hours or 30 minutes is 10 minutes I can probably go like a small few hours and then I need some breaks in between for like food and stuff and uh yeah but I think like uh it's still really hard to accumulate hours I was using a Tracker that told me exactly how much time I've spent coding any one day and even on a very productive day I still spent only like six or eight hours yeah and it's just because there's so much padding commute talking to people food Etc there's like the cost of life just living and sustaining and homeostasis and just maintaining yourself as a human is very high and and there seems to be a desire within the human mind to to uh to participate in society that creates that padding yeah because I yeah the most productive days I've ever had is just completely from start to finish just tuning out everything yep and just sitting there and then and then you could do more than six and eight hours yeah is there some wisdom about what gives you strength to do like uh tough days of long Focus yeah just like whenever I get obsessed about a problem something just needs to work something just needs to exist it needs to exist and you so you're able to deal with bugs and programming issues and technical issues and uh design decisions that turn out to be the wrong ones you're able to think through all of that given given that you want to think to exist yeah it needs to exist and then I think to me also a big factor is uh you know are other humans are going to appreciate it are they going to like it that's a big part of my motivation if I'm helping humans and they seem happy they say nice things uh they tweet about it or whatever that gives me pleasure because I'm doing something useful so like you do see yourself sharing it with the world like with yes on GitHub with a blog post or through videos yeah I was thinking about it like suppose I did all these things but did not share them I don't think I would have the same amount of motivation that I can build up you enjoy the feeling of other people uh gaining value and happiness from the stuff you've created yeah uh what about diet is there I saw you playing with in intermittent fast do you fast does that help with everything well the things you played what's been most beneficial to the your ability to mentally focus on a thing and just meant the mental productivity and happiness you still fast yeah it's so fast but I do intermittent fasting but really what it means at the end of the day is I skip breakfast yeah so I do uh 18 6 roughly by default when I'm in my steady state if I'm traveling or doing something else I will break the rules but in my steady state I do 18 6 so I eat only from 12 to 6. not a hard Rule and I break it often but that's my default and then um yeah I've done a bunch of random experiments for the most part right now uh where I've been for the last year and a half I want to say is I'm um plant-based or planned forward I heard plant forward it sounds better exactly I didn't actually know the differences but it sounds better in my mind but it just means I prefer plant-based food and raw or cooked or I prefer cooked uh and blunt paste so plant-based oh forgive me I don't actually know how wide the category of plant entails Wellness just means that you're not uh and you can flex and uh you just prefer to eat plants and you know you're not making you're not trying to influence other people and if someone is you come to someone's house party and they serve you a stake that they're really proud of you will eat it yes right it's just not judgment oh that's beautiful I mean that's um on the flip side of that but I'm very sort of flexible have you tried doing one meal a day uh I have uh accidentally not consistently but I've accidentally had that I don't I don't like it I think it makes me feel uh not good it's too it's too much too much of a hit yeah and uh So currently I have about two meals a day 12 and six I do that non-stop I'm doing it now I'm doing one meal a day okay so it's interesting it's a interesting feeling have you ever fasted longer than a day yeah I've done a bunch of water fasts because I was curious what happens uh anything interesting yeah I would say so I mean you know what's interesting is that you're hungry for two days and then starting day three or so you're not hungry it's like such a weird feeling because you haven't eaten in a few days and you're not hungry isn't that weird it's really one of the many weird things about human biology is figure something out it finds finds another source of energy or something like that or uh relaxes the system I don't know how yeah the body is like you're hungry you're hungry and then it just gives up it's like okay I guess we're fasting now there's nothing and then it just kind of like focuses on trying to make you not hungry uh and you know not feel the the damage of that and uh trying to give you some space to figure out the food situation so are you still to this day most productive uh at night I would say I am but it is really hard to maintain my PhD schedule um especially when I was say working at Tesla and so on it's a non-starter so but even now like you know people want to meet for various events they Society lives in a certain period of time and you sort of have to like work so that's it's hard to like do a social thing and then after that return and do work yeah it's just really hard uh that's why I try to do social things I try not to do too uh too much drinking so I can return and continue doing work um but a Tesla is there is there conversions in Tesla but any any company is there a convergence towards the schedule or is there more is that how humans behave when they collaborate I need to learn about this yeah do they try to keep a consistent schedule you're all awake at the same time I mean I do try to create a routine and I try to create a steady state in which I'm uh comfortable in uh so I have a morning routine I have a day routine I try to keep things to do a steady state and um things are predictable and then you can sort of just like your body just sort of like sticks to that and if you try to stress that a little too much it will create uh you know when you're traveling and you're dealing with jet lag you're not able to really Ascend to you know where you need to go yeah yeah that's weird as humans with the habits and stuff uh what are your thoughts on work-life balance throughout a human lifetime so the testing part was known for sort of pushing people to their limits in terms of what they're able to do in terms of what they're uh trying to do in terms of how much they work all that kind of stuff yeah I mean I will say teslaq is still too much uh bad rep for this because what's happening is Tesla is a it's a bursting environment uh so I would say the Baseline uh my only point of reference is Google where I've interned three times and I saw what it's like inside Google and and deepmind um I would say the Baseline is higher than that but then there's a punctuated equilibrium where once in a while there's a fire and uh someone like people work really hard and so it's spiky and bursty and then all the stories get collected about the bursts yeah and then it gives the appearance of like total insanity but actually it's just a bit more intense environment and there are fires and Sprints and so I think uh you know definitely though I I would say um it's a more intense environment than something you would get but you in your person forget all of that just in your own personal life um what do you think about the happiness of a human being a brilliant person like yourself about finding a balance between work and life or is it such a thing not a good thought experiment yeah I think I think balance is good but I also love to have Sprints that are out of distribution and that's what I think I've been pretty uh creative and um as well Sprints out of distribution means that most of the time you have a yeah quote-unquote balance I have balance most of the time yes I like being obsessed with something once in a while once in a while is what once a week once a month once a year yeah probably like say once a month or something yeah and that's when we get a new GitHub repo come on yeah that's when you like really care about a problem it must exist this will be awesome you're obsessed with it and now you can't just do it on that day you need to pay the fixed cost of getting into the groove and then you need to stay there for a while and then Society will come and they will try to mess with you and they will try to distract you yeah yeah the worst thing is like a person who's like I just need five minutes of your time yeah this is the cost of that is not five minutes and Society needs to change how it thinks about just five minutes of your time right it's never it's never just one minute it's just 30 it's just a quick what's the big deal why are you being so yeah no uh what's your computer setup what uh what's like the perfect are you somebody that's flexible to no matter what laptop four screens yeah uh or do you uh prefer a certain setup that you're most productive um I guess the one that I'm familiar with is one large screen uh 27 inch um and my laptop on the side with operating system I do Max that's my primary for all tasks I would say OS X but when you're working on deep learning everything as Linux your SSH into a cluster and you're working remotely but what about the actual development like that using the IDE yeah you would use uh I think a good way is you just run vs code um my favorite editor right now on your Mac but you are actually you have a remote folder through SSH um so the actual files that you're manipulating are on the cluster somewhere else so what's the best IDE uh vs code what else do people so I use emacs still that's cool uh so it may be cool I don't know if it's maximum productivity um so what what do you recommend in terms of editors you worked with a lot of software Engineers editors for python C plus plus machine learning applications I think the current answer is vs code currently I believe that's the best um IDE it's got a huge amount of extensions it has a GitHub co-pilot um uh integration which I think is very valuable what do you think about the the co-pilot integration I was actually uh I got to talk a bunch with Guido and Rossum who's a creative Python and he loves Coppola he like he programs a lot with it yeah uh do you yeah he's copilot I love it and uh it's free for me but I would pay for it yeah I think it's very good and the utility that I found with it was is in is it I would say there is a learning curve and you need to figure out when it's helpful and when to pay attention to its outputs and when it's not going to be helpful where you should not pay attention to it because if you're just reading its suggestions all the time it's not a good way of interacting with it but I think I was able to sort of like mold myself to it I find it's very helpful number one in copy paste and replace some parts so I don't um when the pattern is clear it's really good at completing the pattern and number two sometimes it suggests apis that I'm not aware of so it tells you about something that you didn't know so and that's an opportunity to discover and you it's an opportunity to see I would never take copilot code AS given I almost always uh copy a copy this into a Google Search and you see what this function is doing and then you're like oh it's actually actually exactly what I need thank you copilot so you learned something so it's in part a search engine apart maybe getting the exact syntax correctly that once you see it yep it's that NP hard thing it's like once you see it you know yes exactly correct exactly you yourself you can struggle you can verify efficiently but you you can't generate efficiently and copilot really I mean it's it's autopilot for programming right and currently it's doing the link following which is like the simple copy paste and sometimes suggest uh but over time it's going to become more and more autonomous and so the same thing will play out in not just coding but actually across many many different things probably but coding is an important one right like writing programs yeah what how do you see the future of that developing uh the program synthesis like being able to write programs that are more and more complicated because right now it's human supervised in interesting ways yes like what it feels like the transition will be very painful my mental model for it is the same thing will happen as with the autopilot uh So currently it's doing link following is doing some simple stuff and eventually we'll be doing autonomy and people will have to intervene less and less and there could be like you like testing mechanisms like if it writes a function and that function looks pretty damn correct but how do you know it's correct because you're like getting lazier and lazier as a programmer like your ability to because like little bugs but I guess it won't make a little no it will it copilot will make uh off by one subtle bugs it has done that to me but do you think future systems will or is it really the off by one is actually a fundamental challenge of programming in that case it wasn't fundamental and I think things can improve but uh yeah I think humans have to supervise I am nervous about people not supervising what comes out and what happens to for example the proliferation of bugs in all of our systems I'm nervous about that but I think and there will probably be some other copilots for bug finding and stuff like that at some point because there will be like a lot more automation for uh oh man so it's like a program a co-pilot that generates a compiler for one that does a linter yes one that does like a a type Checker yes it's a committee of like a GPT sort of like and then they'll be like a manager for the committee yeah and then there'll be somebody that says a new version of this is needed we need to regenerate it yeah there were 10 gpts that were forwarded and gave 50 suggestions another one looked at it and picked a few that they like a bug one looked at it and it was like it's probably a bug they got re-ranked by some other thing and then a final Ensemble uh GPT comes in it's like okay given everything you guys have told me this is probably the next token you know the feeling is the number of programmers in the world has been growing and growing very quickly do you think it's possible that it'll actually level out and drop to like a very low number with this kind of world because then you'll be doing software 2.0 programming um and you'll be doing this kind of generation of copilot type systems programming but you won't be doing the old school software 1.0 program I don't currently think that they're just going to replace human programmers um it's I'm so hesitant saying stuff like this right because this is going to be replaced in five years I don't know it's going to show that like this is where we thought because I I agree with you but I think we might be very surprised right like what are the next I I what's your sense of what we're seeing with language models like does it feel like the beginning or the middle or the end the beginning 100 I think the big question in my mind is for sure GPT will be able to program quite well competently and so on how do you steer the system you still have to provide some guidance to what you actually are looking for and so how do you steer it and how do you say how do you talk to it how do you um audit it and verify that what is done is correct and how do you like work with this and it's as much not just an AI problem but a UI ux problem yeah um so beautiful fertile ground for so much interesting work for vs code plus plus where you're not just it's not just human programming anymore it's amazing yeah so you're interacting with the system so not just one prompt but it's iterative prompting yeah you're trying to figure out having a conversation with the system yeah that actually I mean to me that's super exciting to have a conversation with the program I'm writing yeah maybe at some point uh you're just conversing with it it's like okay here's what I want to do actually this variable maybe it's not even that low level as variable but you can also Imagine like can you translate this to c plus and back to python yeah that already kind of existence no but just like doing it as part of the program experience like I think I'd like to write this function in C plus plus or or like you just keep changing for different uh different programs because they're different syntax maybe I want to convert this into a functional language and so like you get to become multilingual as a programmer and dance back and forth efficiently yeah I mean I think the UI ux of it though is like still very hard to think through because it's not just about writing code on a page you have an entire developer environment you have a bunch of hardware on it uh you have some environmental variables you have some scripts that are running in the Chrome job like there's a lot going on to like working with computers and how do these uh systems set up environment flags and work across multiple machines and set up screen sessions and automate different processes like how all that works and it's auditable by humans and so on is like massive question at the moment you've built archive sanity what is archive and what is the future of academic research publishing that you would like to see so archive is This pre-print Server so if you have a paper you can submit it for publication to journals or conferences and then wait six months and then maybe get a decision pass or fail or you can just upload it to Archive and then people can tweet about it three minutes later and then everyone sees it everyone reads it and everyone can profit from it uh in their own ways you can cite it and it has an official look to it it feels like a pub like it feels like a publication process yeah it feels different than you if you just put in a blog post oh yeah yeah I mean it's a paper and usually the the bar is higher for something that you would expect on archive as opposed to and something you would see in a blog post well the culture created the bar because you could probably yes host a pretty crappy face for an archive um so what's that make you feel like what's that make you feel about peer review so rigorous peer review by two three experts versus the peer review of the community right as it's written yeah basically I think the community is very well able to peer review things very quickly on Twitter and I think maybe it just has to do something with AI machine learning field specifically though I feel like things are more easily auditable um and the verification is is easier potentially than the verification somewhere else so it's kind of like um you can think of these uh scientific Publications there's like little blockchains where everyone's building on each other's work and setting each other and you sort of have ai which is kind of like this much faster and loose blockchain but then you have and any one individual entry is like very um very cheap to make and then you have other fields where maybe that model doesn't make as much sense um and so I think in AI at least things are pretty easily verifiable and so that's why when people upload papers they're a really good idea and so on people can try it out like the next day and they can be the final Arbiter whether it works or not on their problem and the whole thing just moves significantly faster so I kind of feel like Academia still has a place sorry this like conference Journal process still has a place but it's sort of like an um it lags behind I think and it's a bit more maybe higher quality process but it's not sort of the place where you will discover Cutting Edge work anymore yeah it used to be the case when I was starting my PhD that you go to conferences and journals and you discuss all the latest research now when you go to a conference or Journal like no one discusses anything that's there because it's already like three generations ago irrelevant yes which makes me sad about like deepmind for example where they they still they still publish in nature and these big prestigious I mean there's still value as opposed to The Prestige that comes with these big venues but the the result is that they they'll announce some breakthrough performance and it'll take like a year to actually publish the details I mean and those details in if they were published immediately would Inspire the community to move in certain directions with that yeah it would speed up the rest of the community but I don't know to what extent that's part of their objective function also that's true so it's not just the prestige a little bit of the delay is uh is part yeah they certainly deepmind specifically has been um working in the regime of having a slightly higher quality basically process and latency and uh publishing those papers that way another question from Reddit do you or have you suffered from imposter syndrome being the director of AI Tesla being this person when you're at Stanford where like the world looks at you as the expert in AI to teach teach the world about machine learning when I was leaving Tesla after five years I spent a ton of time in meeting rooms uh and you know I would read papers in the beginning when I joined Tesla I was writing code and then I was writing lesson last code and I was reading code and then I was reading lesson less code and so this is just a natural progression that happens I think and uh definitely I would say near the tail end that's when it sort of like starts to hit you a bit more that you're supposed to be an expert but actually the source of Truth is the code that people are writing the GitHub and the actual the actual code itself and you're not as familiar with that as you used to be and so I would say maybe there's some like insecurity there yeah that's actually pretty profound that a lot of the insecurity has to do with not writing the code in the computer science space like that because that is the truth that that right there code is the source of Truth the papers and everything else it's a high level summary I don't uh yeah just a high level summary but at the end of the day you have to read code it's impossible to translate all that code into actual uh you know paper form uh so when when things come out especially when they have a source code available that's my favorite place to go so like I said you're one of the greatest teachers of machine learning AI ever uh from cs231n to today what advice would you give to beginners interested in getting into machine learning beginners are often focused on like what to do and I think the focus should be more like how much you do so I I'm kind of like believer on a high level in this 10 000 hours kind of concept where you just kind of have to just pick the things where you can spend time and you you care about and you're interested in you literally have to put in 10 000 hours of work um it doesn't even like matter as much like where you put it and your you'll iterate and you'll improve and you'll waste some time I don't know if there's a better way you need to put in 10 000 hours but I think it's actually really nice because I feel like there's some sense of determinism about uh being an expert at a thing if you spend ten thousand hours you can literally pick an arbitrary thing and I think if you spend ten thousand hours of deliberate effort and work you actually will become an expert at it and so I think it's kind of like a nice thought um and so uh basically I would focus more on like are you spending 10 000 hours that's what I focus on so and then thinking about what kind of mechanisms maximize your likelihood of getting to ten thousand dollars exactly which for us silly humans means probably forming a daily habit of like every single day actually doing the thing whatever helps you so I do think to a large extent is a psychological problem for yourself uh one other thing that I help that I think is helpful for the psychology of it is many times people compare themselves to others in the area I think this is very harmful only compare yourself to you from some time ago like say a year ago are you better than you year ago this is the only way to think um and I think this then you can see your progress and it's very motivating that's so interesting that focus on the quantity of ours because I think a lot of people uh in the beginner stage but actually throughout get paralyzed uh by uh the choice like which one do I pick this path or this path yeah like they'll literally get paralyzed by like which ID to use well they're worried yeah they're worried about all these things but the thing is some of the you you will waste time doing something wrong yes you will eventually figure out it's not right you will accumulate scar tissue and next time you'll grow stronger because next time you'll have the scar tissue and next time you'll learn from it and now next time you come into a similar situation you'll be like all right I messed up I've spent a lot of time working on things that never materialize into anything and I have all that scar tissue and I have some intuitions about what was useful what wasn't useful how things turned out uh so all those mistakes were uh were not dead work you know so I just think you should just focus on working what have you done what have you done last week uh that's a good question actually to ask for for a lot of things not just machine learning um it's a good way to cut the the I forgot what the term will use but the fluff the blubber whatever the uh the inefficiencies in life uh what do you love about teaching you seem to find yourself often in the like drawn to teaching you're very good at it but you're also drawn to it I mean I don't think I love teaching I love happy humans and happy humans like when I teach yes I I wouldn't say I hate teaching I tolerate teaching but it's not like the act of teaching that I like it's it's that um you know I I have some I have something I'm actually okay at it yes I'm okay at teaching and people appreciate it a lot yeah and uh so I'm just happy to try to be helpful and uh teaching itself is not like the most I mean it's really it can be really annoying frustrating I was working on a bunch of lectures just now I was reminded back to my days of 231 and just how much work it is to create some of these materials and make them good the amount of iteration and thought and you go down blind alleys and just how much you change it so creating something good um in terms of like educational value is really hard and uh it's not fun it's difficult so for people should definitely go watch your new stuff you put out there are lectures where you're actually building the thing like from like you said the coldest truth so discussing back propagation by building it by looking through and just the whole thing so how difficult is that to prepare for I think that's a really powerful way to teach how did you have to prepare for that or are you just live thinking through it I will typically do like say three takes and then I take like the the better take uh so I do multiple takes and I take some of the better takes and then I just build out a lecture that way uh sometimes I have to delete 30 minutes of content because it just went down the Nelly that I didn't like too much there's about a bunch of iteration and it probably takes me you know somewhere around 10 hours to create one hour of content to give one hour it's interesting I mean is it difficult to go back to the like the basics do you draw a lot of like wisdom from going back to the basics yeah going back to back propagation loss functions where they come from and one thing I like about teaching a lot honestly is it definitely strengthens your understanding uh so it's not a purely altruistic activity it's a way to learn if you have to explain something to someone uh you realize you have gaps in knowledge uh and so I even surprised myself in those lectures like also the result will obviously look at this and then the result doesn't look like it and I'm like okay I thought I understood this yeah but that's why it's really cool to literally code you run it in a notebook and it gives you a result and you're like oh wow and like actual numbers actual input act you know actual code yeah it's not mathematical symbols Etc the source of Truth is the code it's not slides it's just like let's build it it's beautiful you're a rare human in that sense uh what advice would you give to researchers uh trying to develop and publish idea that have a big impact in the world of AI so maybe um undergrads maybe early graduate students yep I mean I would say like they definitely have to be a little bit more strategic than I had to be as a PhD student because of the way AI is evolving it's going the way of physics where you know in physics you used to be able to do experiments on your benchtop and everything was great and you could make progress and now you have to work in like LHC or like CERN and and so AI is going in that direction as well um so there's certain kinds of things that's just not possible to do on the bench top anymore and uh I think um that didn't used to be the case at the time do you still think that there's like Gan type papers to be written where like uh like very simple idea that requires just one computer to illustrate a simple example I mean one example that's been very influential recently is diffusion models diffusion models are amazing the fusion models are six years old for the longest time people were kind of ignoring them as far as I can tell and they're an amazing generative model especially in uh in images and so stable diffusion and so on it's all diffusion based the fusion is new it was not there and came from well it came from Google but a researcher could have come up with it in fact some of the first actually no those came from Google as well but a researcher could come up with that in an academic Institution yeah what do you find Most Fascinating about diffusion models so from the societal impact to the the technical architecture what I like about the fusion is it works so well is that surprising to you the amount of the variety almost the novelty of the synthetic data is generating yeah so the stable diffusion images are incredible it's the speed of improvement in generating images has been insane uh we went very quickly from generating like tiny digits to the tiny faces and it all looked messed up and now we have stable diffusion and that happened very quickly there's a lot that Academia can still contribute you know for example um flash attention is a very efficient kernel for running the attention operation inside the Transformer that came from academic environment it's a very clever way to structure the kernel uh that that's the calculation so it doesn't materialize the attention Matrix um and so there's I think there's still like lots of things to contribute but you have to be just more strategic do you think neural networks could be made to reason uh yes do you think they're already reason yes what's your definition of reasoning uh information processing so in a way that humans think through a problem and come up with novel ideas it it feels like a reasoning yeah so the the novelty I don't want to say but out of distribution ideas you think it's possible yes and I think we're seeing that already in the current neural Nets you're able to remix the training set information into true generalization in some sense that doesn't appear it doesn't matter like you're doing something interesting algorithmically you're manipulating you know some symbols and you're coming up with some correct a unique answer in a new setting what would uh illustrate to you holy shit this thing is definitely thinking to me thinking or reasoning is just information processing and generalization and I think the neural Nets already do that today so being able to perceive the world or perceive the whatever the inputs are and to make uh predictions based on that or actions based on that that's that's the reason yeah you're giving correct answers in novel settings by manipulating information you've learned the correct algorithm you're not doing just some kind of a lookup table and there's neighbor search let me ask you about AGI what what are some moonshot ideas you think might make significant progress towards AGI or maybe in other ways what are big blockers that we're missing now so basically I am fairly bullish on our ability to build agis uh basically automated systems that we can interact with and are very human-like and we can interact with them in a digital realm or Physical Realm currently it seems most of the models that sort of do these sort of magical tasks are in a text Realm um I think uh as I mentioned I'm suspicious that the text realm is not enough to actually build full understanding of the world I do actually think you need to go into pixels and understand the physical world and how it works so I do think that we need to extend these models to consume images and videos and train on a lot more data that is multimodal in that way if you think you need to touch the world to understand it also well that's the big open question I would say in my mind is if you also require the embodiment and the ability to uh sort of interact with the world run experiments and have a data of that form then you need to go to Optimus or something like that and so I would say Optimus in some way is like a hedge in AGI because it seems to me that it's possible that just having data from the internet is not enough if that is the case then Optimus may lead to AGI because Optimus would I to me there's nothing Beyond optimism you have like this humanoid form factor that can actually like do stuff in the world you can have millions of them interacting with humans and so on and uh if that doesn't give a rise to AGI at some point like not I'm not sure what will um so from a completeness perspective I think that's the uh that's a really good platform but it's a much harder platform because uh you are dealing with atoms and you need to actually like build these things and integrate them into society so I think that path takes longer but it's much more certain and then there's a path of the internet and just like training these compression models effectively uh on uh trying to compress all the internet and that might also give these agents as well compress the internet but also interact with the internet yeah so it's not obvious to me in fact I suspect you can reach AGI without ever entering the physical world and which is a little bit more uh concerning because it might that results in it happening faster so it just feels like we're in again boiling water we won't know as it's happening I would like to I'm not afraid of AGI I'm excited about it there's always concerns but I would like to know when it happens yeah or and have like hints about when it happens like a year from now it will happen that kind of thing yeah I just feel like in the digital realm it just might happen yeah I think all we have available to us because no one has built AGI again so all we have available to us is uh is there enough for cow ground on the periphery I would say yes and we have the progress so far which has been very rapid and uh there are next steps that are available and so I would say uh yeah it's quite likely that we'll be interacting with digital entities how will you know that somebody's birthday it's going to be a slow I think it's going to be a slow incremental transition is going to be product based and focused it's going to be GitHub co-pilot getting better and then uh GPT is helping you right and then these oracles that you can go to with mathematical problems I think we're on a on the verge of being able to ask very complex questions in chemistry physics math of these oracles and have them complete Solutions so AGI to use primarily focus on intelligence so Consciousness doesn't enter into uh into it so in my mind Consciousness is not a special thing you will you will figure out and bolt-on I think it's an emerging phenomenon of a large enough and complex enough um generative model sort of so um if you have a complex and Alpha World model that understands the world then it also understands its predicament in the world as being a language model which to me is a form of Consciousness or self-awareness and so in order to understand the world deeply you probably have to integrate yourself into the world yeah and in order to interact with humans and other living beings Consciousness is a very useful tool I think Consciousness is like a modeling insight modeling Insight yeah it's a you have a powerful enough model of understanding the world that you actually understand that you are an entity in it yeah but there's also this um perhaps just a narrative we tell ourselves there's a it feels like something to experience the world the hard problem of Consciousness yeah but that could be just the narrative that we tell ourselves yeah I don't think what yeah I think it will emerge I think it's going to be something uh very boring like we'll be talking to these uh digital AIS they will claim they're conscious they will appear conscious they will do all the things that you would expect of other humans and uh it's going to just be a stalemate I I think there would be a lot of actual fascinating ethical questions like Supreme Court level questions of whether you're allowed to turn off a conscious AI if you're allowed to build the conscious AI maybe there would have to be the same kind of debates that you have around um sorry to bring up a political topic but you know abortion which is the deeper question with abortion is what is life and the Deep question with AI is also what is life and what is conscious and I think that'll be very fascinating to bring up it might become illegal to build systems that are capable that like of such a level of intelligence that Consciousness would emerge and therefore the capacity to suffer would emerge and some A system that says no please don't kill me well that's what the Lambda compute the Lambda chatbot already told um this Google engineer right like it it was talking about not wanting to die or so on so that might become illegal to do that right I because otherwise you might have a lot of a lot of creatures that don't want to die and they will uh you can just spawn Infinity of them on a cluster and then that might lead to like horrible consequences because then there might be a lot of people that secretly love murder and they'll start practicing murder on those systems I mean there's just I to me all of this stuff just brings a beautiful mirror to The Human Condition and human nature we'll get to explore it and that's what like the best of uh the Supreme Court of all the different debates we have about ideas of what it means to be human we get to ask those deep questions that we've been asking throughout human history there's always been the other in human history uh we're the good guys and that's the bad guys and we're going to uh you know throughout human history let's murder the bad guys and the same will probably happen with robots it'll be the other at first and then we'll get to ask questions of what does it mean to be alive what does it mean to be conscious yeah and I think there's some Canary in the coal mines even with what we have today um and uh you know for example these there's these like waifus that you like work with and some people are trying to like this company is going to shut down but this person really like yeah love their waifu and like is trying to like Port it somewhere else and like it's not possible and like I think like definitely uh people will have feelings towards uh towards these um systems because in some sense they are like a mirror of humanity because they are like sort of like a big average of humanity yeah in a way that it's trained but we can that average we can actually watch it's nice to be able to interact with the big average of humanity yeah and do like a search query on it yeah yeah it's very fascinating and uh we can also of course also like shape it it's not just a pure average we can mess with the training data we can mess with the objective we can fine tune them in various ways so we have some um you know impact on what those systems look like if you want to achieve AGI um and you could have a conversation with her and ask her uh talk about anything maybe ask her a question what kind of stuff would you would you ask I would have some practical questions in my mind like uh do I or my loved ones really have to die uh what can we do about that do you think it will answer clearly or would it answer poetically I would expect it to give Solutions I would expect it to be like well I've read all of these textbooks and I know all these things that you've produced and it seems to me like here are the experiments that I think it would be useful to run next and hear some Gene therapies that I think would be helpful and uh here are the kinds of experiments that you should run okay let's go over the Start experiment okay imagine that mortality is actually uh like a prerequisite for happiness so if we become immortal we'll actually become deeply unhappy and the model is able to know that so what is this supposed to tell you stupid human about it yes you can become a mortal but you will become deeply unhappy if if the model is if the AGI system is trying to empathize with you human what is this supposed to tell you that yes you don't have to die but you're really not going to like it because that is it going to be deeply honest like there's a Interstellar what is it the AI says like humans want 90 honesty so like you have to pick how honest I want to answer these practical questions yeah I love Yeah Interstellar by the way I think it's like such a sidekick to the entire story but at the same time it's like really interesting it's kind of limited in certain ways right yeah it's limited and I think that's totally fine by the way I don't think uh I think it's find impossible to have a limited and imperfect agis is that the feature almost as an example like it has a fixed amount of compute on its physical body and it might just be that even though you can have a super amazing Mega brain super intelligent AI you also can have like you know less intelligent AIS that you can deploy in a power efficient way and then they're not perfect they might make mistakes no I meant more like say you had infinite compute and it's still good to make mistakes sometimes like in order to integrate yourself like um what is it going back to Goodwill Hunting uh Robin Williams character says like the human imperfections that's the good stuff right isn't it isn't that this like we don't want perfect we want flaws in part to form connections with each other because it feels like something you can attach your feelings to the the flaws in that same way you want an AI That's flawed I don't know I feel like perfectionist but then you're saying okay yeah but that's not AGI but see AGI would need to be intelligent enough to give answers to humans that humans don't understand and I think perfect isn't something humans can't understand because even science doesn't give perfect answers there's always gabs and Mysteries and I don't know I I don't know if humans want perfect yeah I could imagine just uh having a conversation with this kind of Oracle entity as you'd imagine them and uh yeah maybe it can tell you about you know based on my analysis of Human Condition uh you might not want this and here are some of the things that might but every every dumb human will say yeah yeah trust me I can give me the truth I can handle it but that's the beauty a lot of people can choose uh so but then the old marshmallow test with the kids and so on I feel like too many people uh like it can't handle the truth probably including myself like the Deep truth of The Human Condition I don't I don't know if I can handle it like what if there's some dark stuff what if we are an alien science experiment and it realizes that what if it had I mean I mean this is the Matrix you know the middle over again I don't know I would what would I talk about I don't even yeah I uh probably I will go with the save for scientific questions at first that have nothing to do with my own personal life yeah immortality just like about physics and so on yeah uh to build up like let's see where it's at or maybe see if it has a sense of humor that's another question would it be able to uh presumably in order to if it understands humans deeply would able to generate uh yep to generate humor yeah I think that's actually a wonderful Benchmark almost like is it able I think that's a really good point basically to make you laugh yeah if it's able to be like a very effective stand-up comedian that is doing something very interesting computationally I think being funny is extremely hard yeah because it's hard in a way like a touring test the original intent of the touring test is hard because you have to convince humans and there's nothing that's why that's why when comedians talk about this like there's this is deeply honest because if people can't help but laugh and if they don't laugh that means you're not funny they laugh that's funny and you're showing you need a lot of knowledge to create to create humor about like the occupational Human Condition and so on and then you need to be clever with it uh you mentioned a few movies you tweeted movies that I've seen five plus times but I'm ready and willing to keep watching Interstellar Gladiator contact Goodwill Hunting The Matrix Lord of the Rings all three Avatar Fifth Element so on goes on Terminator two Mean Girls I'm not gonna ask about that mean girls is great um what are some of the jump onto your memory that you love and why like you mentioned the Matrix as a computer person why do you love The Matrix there's so many properties that make it like beautiful and interesting so there's all these philosophical questions but then there's also agis and there's simulation and it's cool and there's you know the black uh you know uh the look of it the feel of it the look of it the feel of it the action the bullet time it was just like innovating in so many ways and then uh Good Good Will Hunting why do you like that one yeah I just I really like this uh torture genius sort of character who's like grappling with whether or not he has like any responsibility or like what to do with this gift that he was given or like how to think about the whole thing and uh there's also a dance between the genius and the the personal like what it means to love another human being and there's a lot of themes there it's just a beautiful movie and then the fatherly figure The Mentor in the in the psychiatrist and the it like really like uh it messes with you you know there's some movies that's just like really mess with you uh on a deep level do you relate to that movie at all no it's not your fault doctor as I said Lord of the Rings that's self-explanatory Terminator 2 which is interesting you we watch that a lot is that better than Terminator one you like you like Arnold I do like Terminator one as well uh I like Terminator 2 a little bit more but in terms of like its surface properties [Laughter] do you think Skynet is at all a possibility oh yes well like the actual sort of uh autonomous uh weapon system kind of thing do you worry about that uh stuff I 100 worry about it and so the I mean the uh you know some of these uh fears of AGS and how this will plan out I mean these will be like very powerful entities probably at some point and so um for a long time they're going to be tools in the hands of humans uh you know people talk about like alignment of agis and how to make the problem is like even humans are not aligned uh so uh how this will be used and what this is going to look like is um yeah it's troubling so do you think it'll happen so slowly enough that we'll be able to as a human civilization think through the problems yes that's my hope is that it happens slowly enough and in an open enough way where a lot of people can see and participate in it just figure out how to deal with this transition I think which is going to be interesting I draw a lot of inspiration from nuclear weapons because I sure thought it would be it would be fucked once they develop nuclear weapons but like it's almost like uh when the when the systems are not so dangerous they destroy human civilization we deploy them and learn the lessons and then we quickly if it's too dangerous we're quickly quicker we might still deploy it uh but you very quickly learn not to use them and so there'll be like this balance that you humans are very clever as a species it's interesting we exploit the resources as much as we can but we don't we avoid destroying ourselves it seems like well I don't know about that actually I hope it continues um I mean I'm definitely like concerned about nuclear weapons and so on not just as a result of the recent conflict even before that uh that's probably like my number one concern for society so if Humanity uh destroys itself or destroys you know 90 of people that would be because of nukes I think so um and it's not even about full destruction to me it's bad enough if we reset society that would be like terrible it would be really bad and I can't believe we're like so close to it yeah it's like so crazy to me it feels like we might be a few tweets away from something like that yep basically it's extremely unnerving but and has been for me for a long time it seems unstable that world leaders just having a bad mood can like um take one step towards a bad Direction and it escalates yeah and because of a collection of bad moods it can escalate without being able to um stop yeah it's just it's a huge amount of uh Power and then also with the proliferation and basically I don't I don't actually really see I don't actually know what the good outcomes are here uh so I'm definitely worried about that a lot and then AGI is not currently there but I think at some point we'll more and more become uh something like it the danger with AGI even is that I think it's even less likely worse in a sense that uh there are good outcomes of AGI and then the bad outcomes are like an absolute way like a tiny one way and so I think um capitalism and humanity and so on will drive for the positive uh ways of using that technology but then if bad outcomes are just like a tiny like flipping minus sign away uh that's a really bad position to be in a tiny perturbation of the system results in the destruction of the human species it's a weird line to walk yeah I think in general what's really weird about like the Dynamics of humanity and this explosion was talked about is just like the insane coupling afforded by technology yeah and uh just the instability of the whole dynamical system I think it's just it doesn't look good honestly yes that explosion could be destructive and constructive and the probabilities are non-zero in both both senses I'm going to have to I do feel like I have to try to be optimistic and so on and yes I think even in this case I still am predominantly optimistic but there's definitely me too uh do you think we'll become a multiplayer species probably yes but I don't know if it's dominant feature of uh future Humanity uh there might be some people on some planets and so on but I'm not sure if it's like yeah if it's like a major player in our culture and so on we still have to solve the drivers of self-destruction here on Earth so just having a backup on Mars is not going to solve the problem so by the way I love the backup on Mars I think that's amazing you should absolutely do that yes and I'm so thankful uh and would you go to Mars uh personally no I do like Earth quite a lot okay uh I'll go to Mars I'll go for you unless I'll tweet at you from there maybe eventually I would once it's uh safe enough but I don't actually know if it's on my lifetime scale unless I can extend it by a lot I do think that for example a lot of people might disappear into um virtual realities and stuff like that and I think that could be the major thrust of um sort of the cultural development of humanity if it survives uh so it might not be it's just really hard to work in Physical Realm and go out there and I think ultimately all your experiences are in your brain yeah and so it's much easier to disappear into digital Realm and I think people will find them more compelling easier safer more interesting so you're a little bit captivated by Virtual Reality by the possible worlds whether it's the metaverse or some other manifestation of that yeah yeah it's really interesting it's uh I'm I'm interested just just talking a lot to Carmack where's the where's the thing that's currently preventing that yeah I mean to be clear I think what's interesting about the future is um it's not that I kind of feel like the variance in The Human Condition grows that's the primary thing that's changing it's not as much the mean of the distribution it's like the variance of it so there will probably be people on Mars and there will be people in VR and they're all people here on Earth it's just like there will be so many more ways of being and so I kind of feel like I see it as like a spreading out of a human experience there's something about the internet that allows you to discover those little groups and you you gravitate each other something about your biology likes that kind of world and that you find each other yeah and we'll have transhumanists and then we'll have the Amish and they're gonna everything is just gonna coexist you know the cool thing about it because I've interacted with a bunch of Internet communities is um they don't know about each other like you can have a very happy existence just like having a very close-knit community and not knowing about each other I mean even even since this just having traveled to Ukraine there's they they don't know so many things about America you you like when you travel across the world I think you experience this too there are certain cultures they're like they have their own thing going on they don't and so you can see that happening more and more and more and more in the future we have little communities yeah yeah I think so that seems to be the that seems to be how it's going right now and I don't see that Trend like really reversing I think people are diverse and they're able to choose their own like path and existence and I sort of like celebrate that um and so will you spend so much time in the meters in the virtual reality or which Community are you are you the physicalist uh the the the physical reality enjoyer or uh do you see drawing a lot of uh pleasure and fulfillment in the digital world yeah I think well currently the virtual reality is not that compelling I do think it can improve a lot but I don't really know to what extent maybe you know there's actually like even more exotic things you can think about with like neural links or stuff like that so um currently I kind of see myself as mostly a team human person I love nature yeah I love Harmony I love people I love Humanity I love emotions of humanity um and I I just want to be like in this like solar Punk little Utopia that's my happy place yeah my happy place is like uh people I love thinking about cool problems surrounded by a lush beautiful Dynamic nature yeah yeah and secretly high tech in places that count places like they use technology to empower that love for other humans and nature yeah I think a technology used like very sparingly uh I don't love when it sort of gets in the way of humanity in many ways uh I like just people being humans in a way we sort of like slightly evolved and prefer I think just by default people kept asking me because they they know you love reading are there particular books that you enjoyed that had an impact on you for silly or for profound reasons that you would recommend you mentioned the vital question many of course I think in biology as an example the vital question is a good one anything by McLean really uh life ascending I would say is like a bit more potentially uh representative as like a summary of a lot of the things he's been talking about I was very impacted by the selfish Gene I thought that was a really good book that helped me understand altruism as an example and where it comes from and just realizing that you know the selection is in the level of genes was a huge insight for me at the time and it sort of like cleared up a lot of things for me what do you think about the the idea that ideas of the organisms the means yes love it 100 [Laughter] are you able to walk around with that notion for a while that that there is an evolutionary kind of process with ideas as well there absolutely is there's memes just like genes and they compete and they live in our brains it's beautiful are we silly humans thinking that we're the organisms is it possible that the primary organisms are the ideas yeah I would say like the the ideas kind of live in the software of like our civilization in the in the minds and so on we think as humans that the hardware is the fundamental thing I human is a hardware entity yeah but it could be the software right yeah yeah I would say like there needs to be some grounding at some point to like a physical reality yeah but if we clone an Andre the software is the thing like is this thing that makes that thing special right yeah I guess I you're right but then cloning might be exceptionally difficult like there might be a deep integration between the software and the hardware in ways we don't quite understand well from the evolution point of view like what makes me special is more like the the gang of genes that are writing in my chromosomes I suppose right like they're the they're the replicating unit I suppose and no but that's just for you the thing that makes you special sure wow the reality is what makes you special is your ability to survive based on the software that runs on the hardware that was built by the genes um so the software is the thing that makes you survive not the hardware all right yeah it's just like a second layer it's a new second layer that hasn't been there before the brain they both they both coexist but there's also layers of the software I mean it's it's not it's a it's a abstraction that's uh on top of abstractions but okay so selfish Gene um a neckline I would say sometimes books are like not sufficient I like to reach for textbooks sometimes um I kind of feel like books are for too much of a general consumption sometime and they just kind of like uh they're too high up in the level of abstraction and it's not good enough yeah so I like textbooks I like the cell I think the cell was pretty cool uh that's why also I like the writing of uh McLean is because he's pretty willing to step one level down and he doesn't uh yeah he's sort of he's willing to go there but he's also willing to sort of be throughout the stack so he'll go down to a lot of detail but then he will come back up and I think he has a yeah basically I really appreciate that that's why I love college early college even high school but just textbooks on the basics yeah of Computer Science and Mathematics of of biology of chemistry yes those are they condense down like uh uh it's sufficiently General that you can understand the both the philosophy and the details but also like you get homework problems and you you get to play with it as much as you would if you weren't yeah programming stuff yeah and then I'm also suspicious of textbooks honestly because as an example in deep learning uh there's no like amazing textbooks and I feel this changing very quickly I imagine the same is true and say uh synthetic biology and so on these books like this cell are kind of outdated they're still high level like what is the actual real source of truth it's people in wet Labs working with cells yeah you know sequencing genomes and yeah actually working with working with it and uh I don't have that much exposure to that or what that looks like so I sold them fully I'm reading through the cell and it's kind of interesting and I'm learning but it's still not sufficient I would say in terms of understanding well it's a clean summarization of the mainstream narrative yeah but you have to learn that before you break out yeah towards The Cutting Edge yeah what is the actual process of working with these cells and growing them and incubating them and you know it's kind of like a massive cooking recipe so making sure your self slows and proliferate and then you're sequencing them running experiments and uh just how that works I think is kind of like the source of truth of at the end of the day what's really useful in terms of creating therapies and so on yeah I wonder in the future AI textbooks will be because you know there's a artificial intelligence a modern approach I actually haven't read if it's come out the recent version the recent there's been a recent Edition I also saw there's a science a deep learning book I'm waiting for textbooks that worth recommending worth reading it's It's tricky because it's like papers and code code honestly papers are quite good I especially like the appendix appendix of any paper as well it's like it's like the most detail it can have it doesn't have to be cohesive to connected to anything else you just describe me a very specific way you solved a particular thing yeah many times papers can be actually quite readable not always but sometimes the introduction in the abstract is readable even for someone outside of the field uh not this is not always true and sometimes I think unfortunately scientists use complex terms even when it's not necessary I think that's harmful I think there there's no reason for that and papers sometimes are longer than they need to be in this in the parts that don't matter yeah appendix would be long but then the paper itself you know look at Einstein make it simple yeah but certainly I've come across papers I would say in say like synthetic biology or something that I thought were quite readable for the abstract and the introduction and then you're reading the rest of it and you don't fully understand but you kind of are getting a gist and I think it's cool what uh advice you give advice to folks interested in machine learning and research but in General Life advice to a young person High School um Early College about how to have a career they can be proud of or a life they can be proud of yeah I think I'm very hesitant to give general advice I think it's really hard I've mentioned like some of the stuff I've mentioned is fairly General I think like focus on just the amount of work you're spending on like a thing uh compare yourself only to yourself not to others that's good I think those are fairly General how do you pick the thing uh you just have like a deep interest in something uh or like try to like find the art Max over like the things that you're interested in ARG Max at that moment and stick with it how do you not get distracted and switch to another thing uh you can if you like um well if you do an ARG Max repeatedly every week it doesn't converge it doesn't it's a problem yeah you can like low pass filter yourself uh in terms of like what has consistently been true for you um but yeah I definitely see how it can be hard but I would say like you're going to work the hardest on the thing that you care about the most also a low pass filter yourself and really introspect in your past were the things that gave you energy and what are the things that took energy away from you concrete examples and usually uh from those concrete examples sometimes patterns can merge I like I like it when things look like this when I'm these positions so that's not necessarily the field but the kind of stuff you're doing in a particular field so for you it seems like you were energized by implementing stuff building actual things yeah being low level learning and then also uh communicating so that others can go through the same realizations and shortening that Gap um because I usually have to do way too much work to understand the thing and then I'm like okay this is actually like okay I think I get it and like why was it so much work it should have been much less work and that gives me a lot of frustration and that's why I sometimes go teach so aside from the teaching you're doing now uh putting out videos aside from a potential uh Godfather part two uh with the AGI at Tesla and Beyond uh what does the future for Android kapati hold have you figured that out yet or no I mean uh as you see through the fog of war that is all of our future um do you do you start seeing silhouettes of the what that possible future could look like the consistent thing I've been always interested in for me at least is is AI and um uh that's probably where I'm spending my rest of my life on because I just care about a lot and I actually care about like many other problems as well like say aging which I basically view as disease and uh um I care about that as well but I don't think it's a good idea to go after it specifically I don't actually think that humans will be able to come up with the answer I think the correct thing to do is to ignore those problems and you solve Ai and then use that to solve everything else and I think there's a chance that this will work I think it's a very high chance and uh that's kind of like the the way I'm betting at least so when you think about AI are you interested in all kinds of applications all kinds of domains and any domain you focus on will allow you to get insights to the big problem of AGI yeah for me it's the ultimate mental problem I don't want to work on any one specific problem there's too many problems so how can you work on all problems simultaneously you solve The Meta problem which to me is just intelligence and how do you automate it is there cool small projects like archive sanity and and so on that you're thinking about the the the the world the ml world can anticipate there's some always like some fun side projects yeah um archive sanity is one basically like there's way too many archive papers how can I organize it and recommend papers and so on uh I transcribed all of your yeah podcasts what did you learn from that experience uh from transcribing the process of like you like consuming audiobooks and and podcasts and so on and here's the process that achieves um closer to human level performance and annotation yeah well I definitely was like surprised that uh transcription with opening eyes whisper was working so well compared to what I'm familiar with from Siri and like a few other systems I guess it works so well and uh that's what gave me some energy to like try it out and I thought it could be fun to random podcasts it's kind of not obvious to me why whisper is so much better compared to anything else because I feel like there should be a lot of incentive for a lot of companies to produce transcription systems and that they've done so over a long time whisper is not a super exotic model it's a Transformer it takes smell spectrograms and you know just outputs tokens of text it's not crazy uh the model and everything has been around for a long time I'm not actually 100 sure why yeah it's not obvious to me either it makes me feel like I'm missing something I'm missing something yeah because there's a huge even at Google and so on YouTube uh transcription yeah um yeah it's unclear but some of it is also integrating into a bigger system yeah that so the user interface how it's deployed and all that kind of stuff maybe running it as an independent thing is eat much easier like an order of magnitude easier than deploying to a large integrated system like YouTube transcription or um anything like meetings like Zoom has trans transcription that's kind of crappy but creating uh interface where it detects the different individual speakers it's able to um display it in compelling ways Run in real time all that kind of stuff maybe that's difficult but that's the only explanation I have because like um I'm currently paying uh quite a bit for human uh transcription human caption right annotation and like it seems like uh there's a huge incentive to automate that yeah it's very confusing and I think I mean I don't know if you looked at some of the whisper transcripts but they're quite good they're good and especially in tricky cases yeah I've seen uh Whispers performance on like super tricky cases and it does incredibly well so I don't know a podcast is pretty simple it's like high quality audio and you're speaking usually pretty clearly and so I don't know it uh I don't know what open ai's plans are yeah either but yeah there's always like fun fun projects basically and stable diffusion also is opening up a huge amount of experimentation I would say in the visual realm and generate generating images and videos and movies videos now and so that's going to be pretty crazy uh that's going to that's going to almost certainly work and it's going to be really interesting when the cost of content creation is going to fall to zero you used to need a painter for a few months to paint a thing and now it's going to be speak to your phone to get your video so if Hollywood will start using that to generate scene means which completely opens up yeah so you can make a like a movie like Avatar eventually for under a million dollars much less maybe just by talking to your phone I mean I know it sounds kind of crazy and then there'd be some voting mechanism like how do you have a like would there be a show on Netflix that's generated completely uh automatedly potentially yeah and what does it look like also when you can just generate It On Demand and it's uh and there's Infinity of it yeah oh man all the synthetic content I mean it's humbling because we we treat ourselves as special for being able to generate art and ideas and all that kind of stuff if that can be done in an automated Way by AI yeah I think it's fascinating to me how these uh the predictions of AI and what it's going to look like and what it's going to be capable of are completely inverted and wrong and the Sci-Fi of 50s and 60s was just like totally not bright they imagined AI is like super calculating theore improvers and we're getting things that can talk to you about emotions they can do art it's just like weird are you excited about that feature just ai's like hybrid systems heterogeneous systems of humans and AIS talking about emotions Netflix and chill with an AI system legit where the Netflix thing you watch is also generated by AI I think it's uh it's going to be interesting for sure and I think I'm cautiously optimistic but it's not it's not obvious well the sad thing is your brain and mine developed in a time where um before Twitter before the before the internet so I wonder people that are born inside of it might have a different experience um like I maybe you can will still resist it uh and the people born now will not well I do feel like humans are extremely malleable yeah and uh you're probably right what is the meaning of life Andre we we talked about sort of the universe having a conversation with us humans or with the systems we create to try to answer for the university for the creator of the universe to notice us we're trying to create systems that are loud enough just answer back I don't know if that's the meaning of life that's like meaning of life for some people the first level answer I would say is anyone can choose their own meaning of life because we are conscious entity and it's beautiful number one but uh I do think that like a deeper meaning of life if someone is interested is uh or along the lines of like what the hell is All This and like why and if you look at the into fundamental physics and the quantum field Theory and a standard model they're like very complicated and um there's this like you know 19 free parameter parameters of our universe and like what's going on with all this stuff and why is it here and can I hack it can I work with it is there a message for me am I supposed to create a message and so I think there's some fundamental answers there but I think there's actually even like you can't actually really make dent in those without more time and so to me also there's a big question around just getting more time honestly yeah that's kind of like what I think about quite a bit as well so kind of the ultimate or at least first way to sneak up to the why question is to try to escape uh the system the universe yeah and then for that you sort of uh backtrack and say okay for that that's going to be take a very long time so the why question boils down from an engineering perspective to how do we extend yeah I think that's the question number one practically speaking because you can't uh you're not gonna calculate the answer to the deeper questions in the time you have and that could be extending your own lifetime or extending just the lifetime of human civilization of whoever wants to not many people might not want that but I think people who do want that I think um I think it's probably possible uh and I don't I don't know that people fully realize this I kind of feel like people think of death as an inevitability but at the end of the day this is a physical system some things go wrong uh it makes sense why things like this happen evolutionarily speaking and uh there's most certainly interventions that uh that mitigate it that would be interesting if death is eventually looked at as as a fascinating thing that used to happen to humans I don't think it's unlikely I think it's I think it's likely and it's up to our imagination to try to predict what the world without death looks like yeah it's hard to I think the values will completely change could be I don't I don't really buy all these ideas that oh without that there's no meaning there's nothing as I don't intuitively buy all those arguments I think there's plenty of meaning plenty of things to learn they're interesting exciting I want to know I want to calculate uh I want to improve the condition of all the humans and organisms that are alive yet the way we find meaning might change we there is a lot of humans probably including myself that finds meaning in the finiteness of things but that doesn't mean that's the only source of meaning yeah I do think many people will will go with that which I think is great I love the idea that people can just choose their own adventure like you you are born as a conscious free entity by default I'd like to think and um you have your unalienable rights for Life uh in the pursuit of happiness I don't know if you have that in the nature the landscape of happiness you can choose your own adventure mostly and that's not it's not fully true but I still am pretty sure I'm an NPC but um an NPC can't know it's an NPC there could be different degrees and levels of consciousness I don't think there's a more beautiful way to end it uh Andre you're an incredible person I'm really honored you would talk with me everything you've done for the machine learning world for the AI world to just inspire people to educate millions of people it's been it's been great and I can't wait to see what you do next it's been an honor man thank you so much for talking today awesome thank you thanks for listening to this conversation with Andre karapathi to support this podcast please check out our sponsors in the description and now let me leave you with some words from Samuel Carlin the purpose of models is not to fit the data but to sharpen the questions thanks for listening and hope to see you next time