Chris Lattner: The Future of Computing and Programming Languages | Lex Fridman Podcast #131
the following is a conversation with Chris lner his second time in the podcast he's one of the most brilliant engineers in modern Computing having created llvm compiler infrastructure project the clang compiler the Swift programming language a lot of key contributions to tensor flown tpus as part of Google he served as vice president of autopilot software Tesla was a software innovator and leader at Apple and now is at SciFi as senior vice president of platform engineering looking to revolutionize chip design to make it faster better and cheaper quick mention of each sponsor followed by some thoughts related to the episode first sponsor is blinkist an app that summarizes key ideas from thousands of books I use it almost every day to learn new things or to pick which books I want to read or listen to next second is neuro the maker of functional sugar-free gum and mints that I use to supercharge my mind with caffeine alanine and B vitamins third is master class online courses from the best people in the world on each of the topics covered from Rockets to game design to Poker to writing and to guitar and finally cash app the app I use to send money to friends for food drinks and unfortunately lost bets please check out the sponsors in the description to get a discount and to support this podcast as a side note let me say that Chris has been in inspiration to me on a human level because he is so damn good as an engineer and leader of Engineers and yet he's able to stay humble especially humble enough to hear the voices of disagreement and to learn from them he was supportive of me in this podcast from the early days and for that I'm forever grateful to be honest most of my life no one really believed that I would amount to much so when another human being looks at me and makes me feel like I might be someone special it can be truly inspired ing that's a lesson for educators the weird kid in the corner with a dream is someone who might need your love and support in order for that dream to flourish if you enjoy this thing subscribe on YouTube review it with five stars and apple podcast follow on Spotify support on patreon or connect with me on Twitter at Lex Freedman and now here's my conversation with Chris lner what are the strongest qualities of Steve Jobs Elon Musk and the Great and Powerful Jeff Dean since you've gotten the chance to work with each you're starting with an easy question there um these are three very different people I guess you could do maybe a pair wise comparison between them instead of a group comparison so if you look at Steve Jobs and Elon um I worked a lot more with Elon than I did with Steve um they have a lot of commonality they're both um Visionary in their own way they're both very demanding in their own way um my sense is Steve Steve is much more human factor focused where Elon is more technology focused what does human factor mean Steve is trying to build things that feel good that people love that affect people's lives how they live he's looking into into the future a little bit in terms of um what people want um where I think the Elon focuses more on uh learning how exponentials work and predicting the development of those Steve worked with a lot of Engineers that was one of the things that reading the biography and how how can a designer essentially talk to engineers and like get their respect I think so I did not work very closely with Steve I'm not an expert at all my sense is that he uh pushed people really hard but then when he got an explanation that made sense to him then he would let go and um he did actually have a lot of respect for engineering and but he also knew when to push and you know when you can read people well you can know when they're holding back and when you can get a little bit more out of them and I think he was very good at that I mean if you if you compare the other the other folks so Jeff Dean right Jeff Dean's an amazing guy he's super smart um as as are the other guys um Jeff is a really really really nice guy well-meaning he's a classic googler he uh uh wants people to be happy he combines it with Brilliance so he can pull people together in a in a really great way he's definitely not a CEO type I don't think he would even want to be that um do you know if he still programs oh yeah definitely programs Jeff is an amazing engineer today right and that has never changed so um it's really hard to compare Jeff to to either of those two um he uh I think that Jeff leads through technology and building it himself and then pulling people in and inspiring them and so I think that that's um a one of the amazing things about Jeff but each of these people you know with their pros and cons all are really inspirational and have achieved amazing things so it's been a it's been I've been very fortunate to work with these guys for yourself you've LED large teams you've done so many incredible difficult technical challenges is there something you've picked up from them about how to lead yeah well so I mean I think leadership is really hard it really depends on what you're looking for there um I think you really need to know what you're talking about so being grounded on the product on the technology on the business on the mission is really important being uh understanding what people are looking for for why they're there one of the most amazing things about Tesla is the unifying Vision right people are there because they believe in clean energy and El electrification all these kinds of things um uh the others is to understand what really motivates people how to get the best people how to how to build a plan that actually can be executed right there's so many different aspects of leadership and it really depends on the time the place the problems you know you know there's a lot of issues that don't need to be solved and so if you focus on the right things and prioritize well that can really help move things two interesting things you mentioned one is you really have to know what you're talking about how you've uh you've worked on a lot of very challenging technical things sure so I kind of assume you were born uh technically Savvy but assuming that's not the case uh how did how did you develop technical expertise like even at Google you worked on I don't know how many projects but really challenging very varied compilers tpus Hardware Cloud stuff bunch of different things um the thing that I've become comfortable as I've more comfortable with as I've uh gained experience is uh being okay with not knowing and so a major part of leadership is actually it's not about having the right answer it's about getting the right answer and so if you're working in a team of amazing people right and many of these places many of these companies all have amazing people it's the question of how do you get people together how do you get how do you build trust how do you get people to open up how do you people get people to you know be vulnerable sometimes with an idea that maybe isn't good enough but it's the start of something beautiful how do you um how do you provide an environment where you're not just like top down Thou shalt do the thing that I tell you to do right but you're encouraging people to be part of the solution and uh and providing a safe space where if you're not doing the right thing they're willing to tell you about it right so you're okay asking dumb questions yeah dumb questions are my specialty yeah well I so I've been in the hardware room recently and I don't know much at all about how chips are designed I know a lot about using them I know some of the principles and the ARs technical level of this but level it turns out that if you ask a lot of dumb questions you get smarter really really quick and when you're surrounded by people that want to teach and learn themselves uh it can be a beautiful thing uh so let's talk about programming languages if it's okay at the High this absurd philosophical level cuz I I don't get romantic on me Lex I I will forever get romantic and uh Tor here I apologize uh why do programming languages even matter okay well thank you very much so you're saying why should why why should you care about anyone programming language or why do why do we care about programming computers or no why why do we why do we care about programming language design creating effective programming languages uh choosing a you know one programming languages versus another programming language why we keep struggling and improving through the evolution of these programming languages sure sure sure okay so so I mean I think you have to come back to what what are we trying to do here right so we have these these uh beasts called computers that are very good at specific kinds of things and we think it's useful to have them do it for us right uh now you have this question of how best to express that because you have a human brain still that has an idea in its head and you want to achieve something right so well there's lots of ways of doing this you can go directly to the machine and speak Assembly Language and then you can express directly what the computer understands that's fine um you can then have higher and higher and higher levels of exraction up until machine learning and you're designing an neural net to do the work for you um the question is where where along this way do you want to stop and what benefits do you get out of doing so and so programming languages in general you have C you have Fortran and Java and a Pascal Swift you have lots of different things um they all have different trade-offs and they're T tackling different parts of the problems now one of the things that most programming languages do is they're trying to make it so that you have pretty basic things like portability across different Hardware so you've got I'm going to run on an Intel PC I'm going to run on RIS 5 PC I'm going to run on a arm phone or something like that fine um I want to write one program and have it portable and this is something the assembly doesn't do now when you start looking at the space of programming languages this is where I think it's fun because programming languages all have trade-offs and most people will walk up to them and they look at the surface level of syntax and say oh I like Curly braces or I like tabs or I like you know semicolons or not or whatever right subjective fairly subjective very shallow things but programming languages when done right can actually be very powerful and the the benefit they bring is expression okay and if you look at programming languages there's really kind of two different levels to them one is the down in the dirt nuts and bolts of how do you get the computer to be efficient stuff like that how they work type systems compiler stuff things like that the other is the UI and the UI for programming language is really a design problem and a lot of people don't think about it that way and the UI you mean all that stuff with the braces and yeah all that stuff to the UI and what it is and UI means user interface um and so what what's really going on is it's the interface between the guts and the human and humans are hard right humans have feelings they have things they like they have things they don't like and a lot of people treat programming languages as though humans are just kind of abstract creatures that cannot be predicted but it turns out that actually there are there is better and worse like people can tell when a program language is good or when it was an accident right and uh one of the things with Swift in particular is that a tremendous amount of time by a t tremendous number of people have been put into really polishing and Mak it feel good but it also has really good nuts and bolts underneath it you said that uh Swift makes a lot of people feel good how do you get to that point so how do you predict that um you know tens of thousands hundreds of thousands of people are going to enjoy using this the user experience of this programming language well you can you can look at it in terms of better and worse right so if you have to write lots of boiler plate or something like that you will feel unproductive and so that's a bad thing you can look at it in terms of safety if like C for example is what's called a memory unsafe language and so you get dangling pointers and you get all these kind of bugs that then you have to spend tons of time debugging and it's a real pain in the butt and you feel unproductive and so by subtracting these things from the experience you get um you know happier people but uh uh again keep interrupting I'm sorry uh but so hard to deal with if you look at the people people that are most productive on stack Overflow they are uh they have a set of priorities yeah that may not always correlate perfectly with the experience of the majority of users you know like if you look at the most uploaded uh quote unquote correct answer on stack Overflow it usually really um sort of uh prioritizes like safe code proper code stable code uh you know that kind of stuff as opposed to like if I want to use go-to statements in my basic right uh I'm I want to use go-to State like what if 99% of people want to use go-to statements or use completely improper you know unsafe syntax I I don't think that people actually like if you boil it down and you get below the surface level people don't actually care about go-tos or if statements for things like this they care about achieving a goal yeah right so the real question is I want to set up a web server and I want to do a thing and I whatever like how how quickly can I achieve that right and so the from a programming language perspective there's really two things that that matter there one is what libraries exist and then how quickly can you put it together and what are the tools around that look like right and uh and when you want to build a library that's missing what do you do okay now this is where you see huge Divergence in the force between worlds okay and so you look at python for example python is really good at assembling things but it's not so great at building all the libraries and so what you get because of performance reasons other things like this is you get python layered on top of C MH for example and that means that doing certain kinds of things well it doesn't really make sense to do in Python instead you do it in C and then you rapid and then you have you're living in Two Worlds and Two Worlds never is really great because tooling and the the debugger doesn't work right and like all these kinds of things can you clarify a little bit what uh what you mean by python is not good at building libraries meaning doesn't make certain kinds of libraries no but just the actual meaning of the sentence yeah uh meaning like it's not conducive to developers to come in and add libraries or it's it's or the langu or is it the The Duality of the it's a dance between Python and c and Python's amazing Python's a great language I did not mean to say that python is is bad for libraries what what I meant to say is um there python there are libraries that Python's really good at they you can write in Python but there are other things like if you want to build a machine learning framework you're not going to build a machine learning framework in Python because of performance for example or you want GPU acceleration or things like this instead what you do is you write a bunch of C or C++ code or something like that and then you talk to it from python right and so this is because of decisions that were made in the python design and um and those decisions have other counterbalancing forces but but the trick when you start looking at this from a programming language perspective is you start say okay cool how do I build this catalog of libraries that are really powerful and how do I make it so that then they can be assembled into ways that feel good and they generally work the first time because when you're talking about building a thing you have to include the debugging the fixing the turnaround cycle the development cycle all that kind of stuff in in into the process of building the thing it's not just about pounding out the code and so this is where things like um you know catching bugs at compile time is valuable for example um but if you dive into the details on this Swift for example has certain things like value semantics which is this fancy way of saying that when you uh treat a treat a variable like a value um uh it acts like a mathematical object would okay so in you have used pytorch a little bit in pytorch you have tensors tensors are uh ND nend dimensional grid of numbers very simple you can do plus and other operators on them it's all totally fine but why do you need to clone a tensor sometimes have you ever run into that uh yeah okay and so why is that why do you need to clone a tensor it's the usual object thing that's in Python so in Python and just like with Java and many other languages this isn't unique to python in Python it has a thing called reference semantics which is the nerdy way of explaining this and what that means is you actually have a pointer to a thing instead of the thing okay now this is due to a bunch of implementation details that you don't want to go into but in Swift you have this thing called value sematics and so when you have a tensor in Swift it is a value if you copy it it looks like you have a unique copy and if you go change one of those copies then uh it doesn't update the other one because you just made a copy of this thing right so that that's like highly error prone in uh at least computer science math Centric disciplines about python that like the the thing you would expect to behave like math like math it doesn't behave like math and in fact uh quietly doesn't behave like math and then can ruin the entirety of your math exactly well and then it puts you in debugging land again yeah right now now you just want to get something done and you're like wait wait a second where you need where do I need to put clone in what level of this stack which is very complicated which I thought I was reusing somebody's library and now I need to understand it to know where to clone a thing thing right and hard to debug by the way exactly right and so this is where programming languages really matter right so in Swift having value sematics so that um both you get the benefit of math working like math right but also the efficiency that comes with certain advantages there certain implementation details there really benefit you as a programmer right you clarify the value sematics like how how do you know that a thing should be treated like a value yeah so so Swift uh has a pretty strong culture and good language support for defining values and so if you have an array so tensors are one example of that the machine learning folks are very used to um just think about arrays same thing where you have an array you put uh you create an array you put two or three or four things into it and then you pass it off to another function mhm what happens if that that uh function adds some more things to it well you'll see it on the side that you pass it in right this is called reference semantics now what if you pass an array off to a function it Scrolls it away in some dictionary or some other data structure somewhere right well it thought that you just handed it that array then you return back and that that that reference to that array still exists in the callar and they go and put more stuff in it right the the person you handed it off to may have thought they had the only reference to that and so they didn't know what they that this was going to change underneath the covers and so this is where you end up having to do clone so like I was past a thing I'm not sure if I have the only version of it so now I have to clone it so what value sematics does is it allows you to say hey I have a so in Swift it defaults to value sematics for oh so defaults to value sematics and then because most things should Valu then it makes sense for that to be the default and one of the important things about that is that arrays and dictionaries and all these other collections or aggregations of other things also have value semantics and so when you pass this around uh to different parts of your program you don't have to do these defensive copies and so this is this is great for two sides right it's great because you define away the bug which is a big deal for productivity the the number one thing most people care about but it's also good for performance because when you're doing a clone so you pass the array down to the thing it's like I don't know if anybody else has it I have to clone it well you just did a copy of a bunch of data it could be big and then it could be that the thing that called you is not keeping track of the old thing so you just made a copy of it and you may not have had to yeah and so the way the value sematics work is in Swift is it uses this thing called copy on right which means that you get you get the benefit of safety cool and performance and it has another special trick because um if you think of certain languages like Java for example they have immutable strings and so what they're trying to do is they provide value semantics by having pure immutability functional languages have pure immutability in lots of different places and this provides a much safer model than it provides valics um the problem with this is if you have immutability everything everything is expensive everything requires a copy um for example in Java if you have a string X and A String y you pen them together we have to allocate a new string to hold XY oh if they're immutable well and strings strings in Java are immutable and if there's there's optimizations for short ones and it's it's complicated but but generally uh think about them as a separate allocation and so when you append them together you have to go allocate a third thing mhm because somebody might have a pointer to either of the other ones right and you can't go change them so you have to go allocate a third thing um because of the beauty of how the Swift value SM system works out if you have a string and Swift and you say hey put in X right and they say append on y z w what it knows that there's only one reference to that and so it can do an inpl update and so you're not allocating tons of stuff on the side you're not you don't have all those problems when you pass it off you can know you have the only reference if you pass it off to multiple different people but nobody changes it they can all share the same thing so you get a lot of the benefit of of purely mutable design and so you get a really nice sweet spot that I haven't seen in other languages yeah that's like I thought I thought there was going to be a a philosophical like narrative here that you're going to have to pay a cost for it CU it sounds like uh I think value semantics is beneficial for easing of debugging or minim izing the risk of Errors like bringing the errors closer to the source um bringing the symptom of the air closer to the source of the air however you say that and but you're saying there's not a performance cost either if you implement correctly well so there there's trade-offs with everything and so if you are doing very low-level stuff then sometimes you can noce cost but then what you're doing is you're saying what is the right default so um coming back to you interface when you when you talk about programming languages one of the ma ma major things that Swift does that makes people love it that is not obvious when it comes to designing language is this UI principle of progressive disclosure of complexity okay so Swift like many languages is very powerful the question is when do you have to learn the power as a user so Swift like python allows you to start with like print hello world MH right certain other languages uh start with like public static void Main like all the ceremony right and so you go to teach you teach a new person hey W welcome to this new thing let's talk about Public Access Control classes wait what's that string system.out do printland like packages like God right and so instead if you take this and you say hey we need you need we need packages you know modules we need we need powerful things like classes we need data structures we need like all these things the question is how do you factor after the complexity and how do you make it so that the normal case scenario is you're dealing with things that work the right way the right way give you good performance the right by default but then as a power user if you want to dive down to it you have full c c performance full control over low-l pointers you can call Malik if you want to call Malik this is not recommended on the first page of every tutorial but it's actually really important when you want to get work done right and so being able to have that is really the design in programm language design design and design is really really hard it's something that I think a lot of people kind of um outside of UI again a lot of people just think is uh subjective like there's nothing you know it's just like Curly braces or whatever it's just like somebody's preference but actually good design is something you can feel and uh how many people are involved with good design so if we looked at Swift but look at historically I mean this might touch like uh it's almost like a Steve jobs question too like how much dictatorial decision- making is required versus um collaborative and we'll talk about how all that can go wrong or right but yeah well Swift so I can't speak to in general all design everywhere uh so the way it works with swift is that um there's a core team and so a core team is uh six or seven people is something like that that is people that have been working with swift since very early days and so I and by early days is not that long ago okay yeah so it's it it became public in 2014 so it's been six years public now but um but still that's enough time that there's a story arc there okay and there's mistakes have been made that then get fixed and you learn something and then you you know and so uh what the core team does is it provides continuity and so you want to have a okay well there's a big hole that we want to fill we know we want to fill it so don't do other things that invade that space until we fill the hole right there there's a boulder that's missing here we want to do we will do that Boulder even though it's not today keep keep out of that space and the whole team remembers of the remembers the myth of the boulder that's there yeah yeah there's a general sense of what the future looks like in Broad strokes and a shared understanding of that combined with a shared understanding of what has happened in the past that worked out well and didn't work out well um the next level out is you have the uh what's called the Swift Evolution community and you've got in that case hundreds of people that really care passionately about the way Swift evolves and that's like an amazing thing to again uh the court team doesn't necessarily need to come up with all the good ideas you got hundreds of people out there that care about something and they come up with really good ideas too and that provides this like tumbling rock tumbler for ideas and so the the evolution process is you know a lot of people in a discourse form they're like hashing it out and trying to like talk about okay well would should we go left or right or if we did this what would be good and you know here you're talking about hundreds of people so you're not going to get consensus necessarily not obvious consensus and so there's a proposal process that uh then allows the core team and the community to work this out and what the core team does is it aims to get consensus out of the community and provide gu guard rails but also provide long-term make sure we're going the right direction kind of things so does that group represent like the how much people will love the user interface like you think to capture that well I mean it's something we talk about a lot something we care about how well we how well we do that Up For Debate but I think that we've done pretty well so far is the beginner in mind like cuz you said the progressive disclosure complex yeah so we care a lot about uh a lot about that a lot about power a lot about efficiency a lot about there are many factors to good design and you have to figure out a way to kind of work your way through that and so if you like think about like a language I love is lisp probably still because I use Zac but I haven't done anything any serious working list but it has a ridiculous amount of parentheses yeah um I've also you know with Java and C++ uh the braces um you know I I like I I I enjoyed the comfort of being between braces you know python is really sorry to interrupt just like and last thing to me as a design if I was a language designer uh God for bit is I would be very surprised that python with no braces would nevertheless somehow be comforting also so like I can see Arguments for all of these but look at this this is evidence that it's not about braces versus tevs right exactly you're good that's a good point right so like you know there there's there's evidence that but see like it's one of the most argued about things oh yeah of course just like tabs and spaces which it does I mean there's one obvious right answer but it doesn't it doesn't actually matter what's that come on we're friends like come on what are you trying to do to me here people are going to yeah half the people are going to tune out yeah um so so do you're able to identify things that don't really matter for the experience well no no no it's it's it's always a really hard so the easy decisions are easy right I mean you you C fine those are not the interesting ones the hard ones are the ones that are most interesting right the hard ones are the places where hey we want to do a thing everybody agrees we should do it there's one proposal on the table but it has all these bad things associated with it well okay what are we going to do about that do we just take it do we delay it do we say hey well maybe there's this other feature that if we do that first this will work out better um how does this if if we do this are we painting ourselves into a corner right and so this is where again you're having that core team of people that uh has some continuity and has perspective has some of the historical understanding is really valuable because you get um it's not just like one brain you get the power of multiple people coming together to make good decisions and then you get the best out of all these people and you also can harness the the community around it what about like the decision of whether like in Python having one type or having you know uh strict typing yeah yeah let's talk about this so so um I I like how you put that by the way like so so many people would say that python doesn't have types doesn't have types yeah I've listened to you enough to where okay I'm I'm a fan of yours and listened to way too many podcast and videos talking about this oh yeah so I would argue that python has one type and so um so like when you import python into Swift which by the way works really well you have everything comes in as a python object now here there trade-offs because um uh you know it depends on what you're optimizing for and python is a super successful language for a really good reason um because it has one type uh you get duck typing for free and things like this but also you're pushing you're making it very easy to to pound out code on the one hand but you're also making it very easy to introduce uh complicated bugs the have debug and you pass the string into something that expects an integer and it doesn't immediately die it goes all the way down the stack trace and you find yourself in the middle of some code that you really didn't want to know anything about and it blows up and you're just saying well what did I do wrong right and so types are good and bad and they have trade-offs they're good for performance and certain other things depending on where you're coming from but it's it's all about trade-offs and so this is this is what design is Right design is about weighing tradeoffs and trying to understand the ramifications of the the things that you're weighing like types or not or one type or many types um but also within many types how powerful do you make that type system is another very complicated question uh with lots of trade-offs it's very interesting by the way uh but uh but that's like one one dimension and there's a bunch of other dimensions jit compiled versus static compiled garbage collected versus reference counted versus memory man manual memory management versus you know like in like all these different trade-offs and how you balance them are what make a programm language good currency y so and all those things I guess uh when you're designing the language you also have to think of how that's going to get all compiled down to if you care about performance yeah well and go back to list right so list also I would say JavaScript is another example of a very simple language right and so one of the so I also love lisp I don't use it as much as maybe you do or you did no I think we're both everyone who loves lisp it's like you love it's like I don't know I love Frank Sinatra but like how often do I seriously listen to Franks sure but but but you look at that or you look at JavaScript which is another very different but relatively simple language and there's certain things that don't exist in the language but there's there is inherent complexity to the problems that we're trying to model and so what happens to the complexity in the case of uh both of them for example you say well what about about large scale software development okay well you need something like packages neither language has a like language affordance for packages and so what you get is patterns you get things like npn you get things like you know like these ecosystems that get built around and I'm a believer that if you don't uh model at least the most important inherent complexity in the language then what ends up happening is that complexity gets pushed elsewhere and when it gets pushed elsewhere sometimes that's great because often building things as libraries is very flexible and very powerful and allows you to evolve and things like that but often it leads to a lot of uh unnecessary Divergence in the force and fragmentation and and when that happens you just get kind of a mess yeah and so the question is how do you how do you balance that uh don't put too much stuff in the language because that's really expensive and makes things complicated but how do you model enough of the inherent complexity of the problem that um you provide the framework and the structure for people to think about Al so so the the the the key thing to think about with uh with programming languages and you think about what a programming language is there for is it's about making a human more productive right and so like there's an old I think it's Steve Jobs quote about um it's a bicycle for the mind right you can you can you can definitely walk but you'll get there a lot faster if you can bicycle on your way and a programming language is a bicycle for the mind yeah is basically a wow that's a really interesting way to think about it by by raising the level of abstraction now you can fit more things in your head by being being able to just directly leverage somebody's Library you can now get something done quickly um in the case of Swift swift UI is this new framework that Apple has released recently for doing UI programming and it has this declarative programming model which defines away entire classes of bugs it's make it builds on value sematics and many other nice Swift things and what this does allows you just get way more done with way less code and now your productivity as a developer is much higher right and so that that's really the what programming languages should be about is it's not about tabs versus spaces or curly braces or whatever it's about how productive do you make the person and you can only see that when you have libraries that were built with the right intention that the language was designed for and with Swift I think we're still a little bit early um but Swift UI and many other things that are coming out now are really showing that and I think that they're opening people's eyes it's kind of interesting to think about like how that you know then knowledge of something of how good the bicycle is how people learn about that you know so I've used C++ now this is not going to be a trash talking session about C++ but use C++ for a really long go there if you want I have the scars I I feel like I spent many years without realizing like there's languages that could for my particular LIF style brain style thinking style there's languages that that could make me a lot more productive uh in the debugging stage in the just the development stage and thinking like the bicycle for the mind I can fit more stuff into my Python's a great example of that right I mean a machine learning framework in Python is a great example of that it's just very high abstraction level and so you can be thinking about things on a like very high level Al algorithmic level instead of thinking about okay well am I copying this tensor to a GPU or not right it's not it's not what you want to be thinking about and as I was telling you I mean I guess I guess the question I head is uh you know how does a person like me or in general people discover more productive uh you know languages like how I was as I've been telling you offline I've been looking for like a project to work on in Swift so I can really uh try it out as I mean my intuition was like doing a hello world is not going to get me there uh to to to get me to experience the power of the language you need a few weeks to change your metabolism exactly I put uh that that's one of the problems with people with diets like I I'm I'm actually currently to go in parallel but in a small tangent is I've been recently eating only meat okay okay and okay so most people are like uh think that's horribly unhealthy or whatever you have like a million It Whatever the science is it just doesn't sound right well so so back when I was in college we did the Atkins diet that was that was a thing similar and but if you you have to always give these things a chance I mean with dieting always not dieting but just the things that you like if I eat personally if I eat meat just everything I could be super F or more focused than usual I just feel great I I've been I've been running a lot you know doing push-ups and pull-ups and so on I mean python is similar in that sense for me where you going with this I mean literally I just I felt I had like a stupid smile on my face when I first started using python I could uh code up really quick things like I like I I would see the world I'll be empowered to write a script to to um you know to do some basic data processing to rename files on my computer yeah right like Pearl didn't do that for me uh uh I mean kind a little bit well and again like none of these are about which which is best or something like that but there there's definitely better and worse here but it clicks right well yeah it if you look at Pearl for example you get bogged down in uh scalers versus arrays versus hashes versus type Globs and like all that kind of stuff and and Python's like yeah let's not do this right and some of is debuging like everyone has different priorities but for me it's could I create systems for myself that Empower me to debug quickly like I've always been a big fan even just crude like asserts like always uh stating things that should be true uh which in Python I found myself doing more because of type all these kinds of stuff well you could think of types in a programming language as being kind of assert yeah they get check at compile time right um so how do you learn a new thing well so this or how do how do people learn new things right this this is hard uh people don't like to change people generally don't like change around them either and so uh we're all very slow to adapt and change and usually there's a catalyst that's required to to force yourself over the over over the so for learning a programming language it really comes down to finding an excuse like build a thing that that's that the language is actually good for that the ecosystem is ready for um and so um and so if you were to write an IOS app for example that would be the easy case obviously you would use Swift for that right there are other Android Swift runs on Android oh does it oh yeah yeah Swift runs in lots of places so uh okay so Swift Swift swift is built on top of lvm lvm runs everywhere lvm for example builds the Android kernel oh wow okay so um okay I didn't realize this yeah so Swift swift is very portable runs on Windows there's it runs on lots of different things and Swift side Swift UI and then there's a thing called UI kit so can I build an app with Swift uh well so that that's the thing is the ecosystem is what matters there so Swift UI and uiit are Apple Technologies okay got it and so they happen to like Swift eii happens to be written in Swift but it's an apple proprietary framework that um Apple loves and wants to keep on its platform which makes total sense you go go to Android and you don't have that Library yeah right and so Android has a different ecosystem of things that hasn't been built out and doesn't work as well with Swift and so you can totally use Swift to do uh like arithmetic and things like this but building a UI with Swift on Android is not not not a not a great experience right now so so if I wanted to uh so learn Swift what's the pro I mean the one practical different version of that is um Swift for tensorflow for example and one of the inspiring things for me with both tensorflow and pytorch is how quickly the community can like switch from different libraries y like you could see some of the communi switching to pytorch now but it could it's very easy to see and then tensor flow is really stepping up its game and then there's no reason why I think it the way it works basically there has to be one GitHub repo like one paper steps up to get gets people excited gets people excited and they're like ah I have to learn this Swift for what what Swift again like and then they learn and they fall in love with I mean that's what happen PTO has there has be a reason a catalyst yeah and so and and there I mean people don't like change but it turns out that once you've worked with one or two programming languages they're the basics are pretty similar and so one of the fun things about learning programming languages even even maybe list I don't know if you agree with this is that when you start doing that you start learning new things because you have a new way to do things and you're forced to do them and that forces you to explore and it puts you in learning mode and when you get in learning mode your mind kind of opens a little bit and you can you can see things in a new way even when you go back to the old place right yeah it's totally well Lis is functional yeah uh stuff but I wish there was a kind of window maybe you can tell me if there is uh there you go this this a question uh to ask what is the most beautiful feature in a programming language before I ask it let me say like with python I remember when I saw list comprehensions okay was like when I like really took it in yeah it I don't know I just loved it it was like fun to do like it was fun to do that kind of um uh yeah there was something about it to be able to filter through a list and to create a new list all in a single line was elegant I could all get into my head and it just made me um fall in love with the language so is there let me ask you a question uh is there what do use the most beautiful feature in uh in a programming languages that you've ever encountered in Swift maybe and then outside of Swift I think the thing that I like the most from a programming language so so I think the thing you have to think about with the programming language again what is the goal you're trying to get people to get things done quickly and so you need libraries you need high quality libraries and then you need a user base around them that can assemble them and do cool things with them right and so to me the question is what enables high quality libraries okay yeah and there's a huge divide in the world between libraries who enable highquality libraries versus um the ones that put special stuff in the language so programming languages that enable high quality libr high quality libraries got it so so and what I mean by that is expressive libraries that then feel like a natural integrated part of the language itself MH so um an example of this in Swift is that int and float and also array and string things like this these are all part of the library like int is not hard-coded into Swift and so what that means is that because int is just a Library Thing defined in the standard Library along with strings and arrays and all the other things that come with the standard Library um well hopefully you do like int but anything that any language features that you needed to Define int you can also use in your own types so if you want to define a uh querian or something like this right um well it doesn't come in the standard Library um there's a very special set of people that care a lot about this but those people are also important it's not it's not about classism right it's not about the people who care about instant floats are more important than the people who care about querian and so to me the beautiful things about about programming languages is when you allow those communities to to build high quality libraries that feel native that feel like they're built into the built into the compiler without having to be what does it mean for the int to be part of not hardcoded in so is it like how so what is an what is an INT okay int is just a integer in this case it's like a you know like a 64-bit integer or something like this but so like the 64-bit is hardcoded or no no none of that's hardcoded so int int if you go look at how it's implemented is it's just a struct and Swift and so it's a struct and then how do you add two structs well you define plus and so you can Define Plus on int well you can Define Plus on your thing too you can Define uh int has like an is OD method or something like that on it and so yeah you can add methods onto things yeah uh so you can you can def find operators like how it behaves yeah that to is beautiful when there there's something about the language which enables others to create libraries which are um not hacky yeah that feel that feel native and so one of the best examples of this is lisp MH right because in lisp all like all the libraries are basically part of the language right you write term rewrite systems and things like this and so can you as a counter example provide what makes it difficult to write a library that's native is it the python C well so well so one example I'll give you two examples um Java and C++ or Java and C um they both allow you to Define your own types um but int is hard code in the language okay well why well in in Java for example coming back to this whole reference semantic value semantic thing um int gets passed around by value yeah but if you if you make if you make like a pair or something like that a complex number right it's a it's a class in Java and now it gets passed around by reference by pointer and so now you lose value sematics right you lost math okay well that's not great right if if you can do something with in why can't I do it with my type yeah right so that's that's the the negative side of the thing I find beautiful is when you can solve that when you can have full expressivity where where you as a user of the language have as much or almost as much power as the people who implemented all the standard built-in stuff because what that enables is that enables truly beautiful libraries you know it's kind of weird cuz I've gotten used to that uh that's one I guess other aspect of program language design you have to think you know the old uh first principles thinking like why are we doing it this way by the way I mean I remember cuz I was thinking about the wallers operator and I'll ask you about it later but it it hit me that like the equal sign for assignment yeah like why are we using the equal sign for assignment and that's not the only solution right so if you look at Pascal they use colon equals for assignment and equals for um for equality and they use like less than greater than instead of the not equal thing like there are other answers here so but like and yeah like ask you all but how do you then decide uh to break convention to say you know what this everybody's doing it wrong we're gonna do it right yeah so so it's like an Roi like return on investment tradeoff right so if you do something weird let's just say like not like colon equal instead of equal for assignment that would be weird with today's aesthetic right and so you'd say cool this is theoretically better but is it better in which ways like what do I get out of that do I Define away class of bugs well one of the class of bugs that c has is that you can use like you know if x equals without equals equals f x equals y yeah right well turns out you can solve that problem in lots of ways clang for example GCC all these compilers will detect that as a as a likely bug produce a warning do they yeah I feel like they didn't or clang do GCC didn't and it's like one of the important things about programming language design is like you're literally creating suffering in the world okay like like I feel I mean one way to see it is the bicycle for the mine but the other way is to like minimizing suffering well you have to decide if it's worth it right and so let's come back to that okay but um but if you if you look at this and again this is where there's a lot of detail that goes into each of these things um uh equal and C returns a value y that's messed up that allows you to say xal yals Z like that works in C yeah um is it messed up you know most people think it's messed up by think uh it it is very by messed up what I mean is it is very rarely used for good and it's often used for bugs yeah right and so that's a good definition of up yeah you could use you know it's it's a in hindsight this was not such a great idea right now one of the things with swift that is really powerful and one of the reasons it's actually good um versus it being full of good ideas is that um when when we launched Swift one we announced that it was public people could use it people could build apps but it was going to change and break okay when Swift 2 came out we said hey it's open source and there's this open process which people can uh help evolve and direct the language so the community at large like Swift users can now help shape the language as it is and what happened is that part as part of that process is a lot of really bad mistakes got taken out so for example Swift used to have the C style Plus+ and minus minus operators like what does it mean when you put it before versus after right well that got cargo culted from C into Swift early on what's cargo culted cargo culted means uh brought forward without really considering considering it okay um this is maybe not the most py term but um have to look it up an urban dictionary yeah yeah so it got pulled it got pulled into C without or it got pulled into Swift without very good consideration and we went through this process and one of the first things got ripped out was plus plus and minus minus because they lead to confusion they have very low value over saying you know X plus equals 1 and X Plus equal 1 is way more clear and so when you're optimizing for teachability and Clarity and bugs and this multi-dimensional space that you're looking at um things like that really matter and so being uh first principles on where you're coming from and what you're trying to achieve and being anchored on the objective is really important well let me ask you about uh the most uh sort of this this uh this this podcast isn't about information it's about drama so let me talk to you about some drama so you mentioned Pascal and colon equals uh there's something that's called the wallrus operator okay and uh python uh in Python 3.8 added the walus operator and the reason I think it's interesting uh it's not just because of the feature it does it's it has the same kind of expression feature you can mention to see that it Returns the value of the assignment and maybe you can comment on that in general but on the other side of it it's also the thing that that uh toppled the dictator uh so okay it finally drove Guido to uh step down from edfl the toxicity of the community so maybe um what do you think about the wallus operator in in Python is there an equivalent thing in Swift that really uh stress tested the community and uh and then on the flip side what do you think about AGA stepping down over it yeah if well if like if I look past the details of the W walrus operator one of the things that makes it most polarizing is that it's syntactic sugar okay what do you mean by syntactic Sugar it means you can take something that already exists in the language and you can express it in a more concise way so okay I'm going to play Do's advocate so uh this is great uh is that objective or subjective statement like can you can you argue that basically anything is syntactic sugar or no uh no you not everything is is syntactic sugar so for example um the type system like can you have classes versus uh versus uh like do you have types or not right so so one type versus many types is not something that affects syntactic sugar and so if you say I want to have the ability to Define types I have to have all this like language mechanics to Define classes and oh now I have to have inheritance and I have like have all this stuff that's just making the language more complicated mhm that's not that's not about sugaring it um Swift has sugar so like Swift has this thing called IFL and it has uh various operators that used to conisy uh specific use cases so the problem with syntactic sugar when you're talking about hey I have a thing that takes a lot to write and I have a new way to write it you have this like horrible trade-off which becomes almost completely subjective which is how often does this happen and does it matter and one of the things that is true about human psychology particularly when you're talking about introducing a new thing is that uh people over overestimate the burden of learning something and so it looks foreign when you haven't gotten used to it but if it was there from the beginning of course it's just part of python like unquestionably like this is this is just a thing I know and it's not a new thing that you're worried about learning it's just part of part of the deal now with Guido uh I I don't know Guido well um yeah have you a pass cross much yeah I've met him a couple of times but I don't know Guido well but the the sense that I got out of that whole dynamic was that he had put the not just the decision maker weight on his shoulders but it was so tied to his personal identity that um he took it personally and he felt the need and he kind of put himself in the situation of being the person instead of building a base of support around him I mean he this is probably not quite literally true but by too much so there's too much too much concentrated on him right and so and that can wear you down well yeah particularly because people then say Guido you're a horrible person I hate this thing blah blah blah blah blah blah blah and sure it's like you know maybe 1% of the community that's doing that but Python's got a big community and 1% of of millions of people is a lot of hate mail and that just from human factor will just wear on you what to to clarify it looked from just what I saw in the messaging for the let's not look at the million python users but at the python core developers it feels like the majority the big majority on a vote were opposed to it okay I'm not that close to it so I don't knowen so so this okay so the situation is like literally uh yeah I mean the majority of the core developers are against so I and they weren't they weren't even like against it it was uh there was a feel well they were against it but the they against it wasn't like this is a bad idea they were more like we don't see why this is a good idea and what that results in is there's a stalling feeling like you you just slow things down now from my perspective now you could argue this and I think it's very it's very interesting if we look at politics today and the way Congress works it slowed down everything it's a dampener yeah it's a dampener but like that's a dangerous thing too because if it dampens things like you know dampening results what are you talking about like it's a low pass filter but if you need billions of dollars injected into the economy or trillions of dollars then suddenly stuff happens right and so for sure so you're talking I'm not defending our political situation just to be clear but you're talking about like a a global pandemic I I was hoping we could fix like the Health Care system and the education like you know I I'm not I'm not a politics person I don't I don't I don't know um when it comes to languages the community is kind of right in terms of it's a very high burden to add something to a language so as soon as you add something you have a community of people building on it and you can't remove it okay and if there's a community of people that feel really uncomfortable with it then taking it slow I think is is is an important thing to do and there's no rush particularly if with something that's 25 years old and is very established and you know it's not like coming coming into its own um what about features so I I think that the issue with with Guido is that maybe this is a case where he realized it had outgrown him and it went from being or the language the language so python I mean Guido is amazing but but python isn't about Guido anymore it's about the users and to a certain extent the users own it and you know py Guido spent years of his life a significant fraction of his career on Python and from his perspective I imagine he's like well but this is my thing I should be be able to do the thing I think is right but you can also understand the users where they feel like you know this is my thing I use this like and um and I don't know it's it's a hard it's a hard thing but what if we could talk about leadership in this cuz it's so interesting to me I'm going to I'm going to make I'm going to wear hopefully somebody makes it if not I'll make it a w operator shirt because I think it represents to me maybe it's my Russian roots or something uh you know it's the burden of leadership like I feel like to push back I feel like progress can only like most difficult decisions just like you said there'll be a lot of divis divisiveness over especially in the passionate Community it just feels like leaders need to take those risky decisions that that if you like listen that with some nonzero probability maybe even a high probability would be the wrong decision but they have to use their gut and make that decision well this this this is like one of the things where you see uh amazing Founders the founders understand exactly what's happened and what how the company got there and are willing to say to we have been doing thing X the the last 20 years but today we're going to do thing why and they make a major pivot for the whole Company the company lines up behind them they move and it's the right thing but then when the founder dies the successor doesn't always feel that that um agency to be able to make those kinds of decisions yeah even though they're a CEO they could theoretically do whatever there's two reasons for that in my opinion or in many cases it's always different but um one of which is they weren't there for all the decisions that were made and so they don't know the principles in which those decisions were made and once the principles change you're you should be obligated to change what you're doing and change direction right and so if you don't know how you got to where you are it seems like gospel and you know you're not going to question it you may not understand that it really is the right thing to do so you just may not see it that's so brilliant I never thought of it that way like it's it's so much higher burden when as a leader you step into a thing that's already worked for a long time yeah yeah well and if you change it and it doesn't work out now you're the the person who screwed it up people always second guess that yeah and the second thing is that even if you decide to make a change even if you're theoretically in charge you're just you're just a person that thinks they're in charge meanwhile you have to motivate the troops you have to explain it to them in terms of understand you have to get them to buy into and believe in it because if they don't then they're not going to be able to make the turn even if you tell them you know their bonuses are going to be curtailed they're just not going to like buy into it you know and so there's only so much power you have as a leader and you have to understand what that what those limitations are are you still bdfl you've been bdfl of some stuff uh you're very heavy on the be the benevolent uh benevolent dictated for Life uh I guess lvm you're still so I still lead the lvm world uh I mean what's the role of uh so then on Swift you said that there's a group of people yeah so if you contrast python with Swift right one of the reasons so everybody on the core team takes the role really seriously and I think we all really care about where Swift goes but you're almost delegating the final decision- making to the wisdom of the group and so it doesn't become personal and also when you're talking with the community so yeah some people are very annoyed at certain decisions get made um there's a certain faith in the process because it's a very transparent process and when a decision gets made a full rationale is provided things like this these are almost defense mechanisms to help both guide future discussions and provide case law kind like Supreme Court does about this decision was made for this reason and here's the rationale and what we want to see more of or less of um but it's a way to provide a defense mechanism so that when somebody's griping about it they're not saying that person did the wrong thing they're saying well this this thing sucks and and later they move on and they they get over it yeah the analogy of the Supreme Court I think is really is really good but then okay not to get person on the Swift team but like is there is there div like it just seems like it's impossible for their for division not to emerge well each each of the humans on the the Swift core team for example are different and the membership of the Swift core team changes slowly over time which is I think a healthy thing and so each of these different humans have different opinions trust me it's not it's not a sing singular consciousness of by any stretch of the imagination you've got three major organizations including Apple Google and sci-fi all kind of working together and um it's a small group of people but you need High trust you need again it comes back to the principles of what you're trying to achieve and understanding you know what what you're optimizing for and I think that starting with strong principles and working towards decisions is always a good way to both make wise decisions in general but then be able to communicate them to people so that they can buy into them and that that is hard and so you mentioned lvm lvm is uh going to be 20 years old uh this December so it's it's showing its own age you have like like a like a like a dragon cake plant or you have a no we should definitely do that yeah if we can have a uh pandemic cake pandemic cake everybody gets a slice of cake and it gets you know sent through email um but the uh uh but lvm has had tons of its own challenges over time too right and one of the challenges that um the lvm community has in my opinion is that it has a whole bunch of people that um have been working at lvm for 10 years right because this happen some somehow and lvm has always been one way but it needs to be a different way right and they've worked on it for like 10 years is a long time to work on something and you know you you suddenly can't see the faults in the thing that you're working on and lvm has lots of problems and we need to address them and we need to make it better and if we don't make it better then somebody else will come up with a better idea right and so it's just kind of of that age where the community is like in danger of getting too calcified and um and so I'm happy to see new projects joining and new things mixing it up you know Fortran is now a new a new thing in the Elum Community which is hilarious and good I've been trying to find uh on this little tangent find people who program in Cobalt or Fortran Fortran especially to talk to they're hard to find yeah yeah look to the uh scientific Community they still use forun quite a bit interesting thing you kind of mentioned with lvm or just in general that if something evolve you're not able to see the faults so do you uh fall in love with the thing over time or do you start hating everything about the thing over time well so so my my my personal Folly is that um I see maybe not all but many of the faults and they gr on me and I don't have time to go fix them yeah and they get magnified over time well and they may not get magnified but they never get fixed it's like sand underneath you you know it's just like raiding against you and it's like s underneath your fingernails or something it's just like you know it's there you can't get rid of it um and so the the problem is that if other people don't see it right nobody ever get like I can't go I don't have time to go write the code and fix it anymore but then uh people are resistant to change and so you say hey we should go fix this thing they're like oh yeah that sounds risky well is it the right thing or not are the challenges uh the group dynamics or is it also just technical I mean some of these features like yeah I think uh as an observer is almost like a fan in in the uh you know as a spectator of the whole thing it I don't often think about you know some things might actually be technically difficult to implement an example of this is we we built this new compiler framework called ml yes ml is this a whole new framework it's not many people think it's about machine learning the ml stands for multi-level because compiler people can't name things very well I guess can we can we dig into what ml IR is yeah so when you look at compilers compilers have historically been solutions for a given space so lvm is a it's really good for dealing CPUs let's just say at a high level you look at um Java Java has a jvm the jvm is very good for garbage collected languages that need Dynamic compilation and it's very optimized for specific space and so hotspot is one of the compilers that gets used in that space and that compiler is really good at that kind of stuff um usually when you build these domain specific compilers you end up building whole thing from scratch for each domain uh what's a domain so what what we what's this what's the scope of a domain Al so here I would say like if you look at Swift there's several different parts to the Swift compiler um one of which is covered by um the LM part of it there's also a highle piece that's specific to Swift and there's a huge amount of redundancy between those two different infrastructures and a lot of re reimplemented stuff that is similar but different what is llvm Define lvm is effectively an infrastructure so you can mix and match it in different ways it's built libraries you can use it for different things but it's really good at CPUs and gpus CPUs and like the tip of the iceberg on gpus it's not really great at gpus okay um but it turns out languages that that then use it to talk to CPUs it um and so it turns out there's a lot of Hardware out there that is custom accelerators so machine learning for example there are a lot of uh Matrix multiply accelerators and things like this there there's a whole world of Hardware synthesis so we're using ml to build circuits okay and so you're compiling for a domain of transistors and so what ml does is it provides a tremendous amount of compiler infrastructure that allows you to build these domain specific compilers in a much faster way and have the result be good if we're if we're thinking about the future now we're talking about like as6 like so anything yeah yeah so if we project into the future it's very possible that the number of these kinds of as6 very specific um infrastructure thing architecture things uh like multiplies exponentially I hope so yeah so that's ml so what ml what ml does is it allows you to build these compilers very efficiently right now one of the things that coming back to the lvm thing and then we'll go to Hardware is um lvm is is a specific compiler for specific domain mlr is now this very general very flexible thing that can solve lots of different kinds of problems so lvm is a subset of what ml does so m is I mean it's an ambitious project then yeah it's a very ambitious project yeah and so to make it even more confusing ml has joined the lvm umbrella project so it's part of the lvm family right um but where this comes full circle is now folks that work on the lvm part the classic part that's 20 years old um aren't aware of all the cool new things that have been done in the new the new thing that you know mlr was built by me and many other people that knew a lot about lvm and so we fixed a lot of the mistakes that lived in LV so now you have this community Dynamic where it's like well there's this new thing but it's not familiar nobody knows it it feels like it's new and so let's not trust it and so it's just really interesting to see the cultural social Dynamic that comes out of that and and you know I think it's super healthy because we're seeing the ideas percolate and we're seeing the technology diffusion happen as people get more comfortable with it they start to understand things in their own terms and this just gets to the it takes a while for ideas to propagate even though um they may be very different than what people are used to so maybe let's talk about that a little bit the world of Asic and well actually you're uh you're you have a new role at sci-fi what's that place about what is the vision sure uh for their vision for I would say the future of computing yeah so I lead the engineering and product teams at SciFi sci5 is a company who's was founded with this architecture called risk 5 risk 5 is a new instruction set instruction sets are the things inside of your computer that tell how to run things um x86 from Intel and arm from the arm company and things like this or other instruction sets I've talked to sorry interrupt I've talked to Dave Patterson who's super excited about risk 5 Dave Dave is awesome he's brilliant yeah yeah the uh risk five is distinguished by not being proprietary MH and so xa6 can only be made by Intel and AMD arm can only be made by arm they sell licenses to build arm ships to other companies things like this myips is another instruction set that is owned by the myips company now wave and it gets licensed out things like that um and so RIS 5 is an open standard that anybody can build chips for and so SciFi was founded by three of the founders of RIS 5 that designed and built it in Berkeley working with Dave um and so that was the The Genesis of the company scii today has some of the world's best r five cores and we're selling them and that's really great they're going into tons of products it's very exciting um so they're taking this uh thing that's open source and just being trying to be or are the best in the world at building these things yeah so here it's the specifications open source it's like saying tcpip is an open standard or C Is An Open standard but then you have to build an implementation of the standard and so sci5 on the one hand pushes forward and defined and pushes forward the standard on the other hand we have implementations that are best in class for different points in the space depending on if you want a really tiny CPU or if you want a really big beefy one that that uh is faster but it uses more area and things like this what about the actual manufacturer chip so like what where does that all fit I'm going to ask a bunch of dumb questions that's okay this is how we learn right uh and so uh what the the way this works is that there's generally a separation of the people who design the circuits than the people who manufacture them and so that you'll hear about Fabs like tsmc and Samsung and things like this that actually produce the chips but they take a design coming in and that design specifies how um how the you know you turn uh code for the chip into uh little rectangles that then use Photo lithography to make uh mask sets and then burn transistors onto a chip or onto a onto silicon rather well so and we're talking about Mass manufacturing so yeah they're talking about making hundreds of millions of parts and things like that yeah and so the the Fab handles the volume production things like that but um when you look at this problem um the interesting thing about the space when you look at it is that um these the steps that you go from designing a chip and writing the quote unquote code for it and things like verog and languages like that down to what you hand off to the Fab is a really well studied really old problem mhm um tons of people have worked on it lots of smart people have built systems and tools um these tools then have generally gone through Acquisitions and so they've ended up at three different major companies that build and sell these tools they're called Eda tools like for electronic design automation um the problem with this is you have huge amounts of fragmentation you have loose standards um and the tools don't really work together so you have tons of duct tape and you have tons of uh lost productivity now these are uh these are tools for Designing so the risk five is a instruction like what is risk five like how deep does it go how how how much does it touch the hardware how much does it Define how much of the hardware is yeah so RIS RIS five is all about um given a CPU so the the the processor and your computer how does the the compiler like Swift compiler the C compiler things like this how does it make it work so it's what is the assembly code and so you write risk five assembly instead of xa6 assembly for example but it's a set of instructions as opposed to set of instructions yeah why why do you say it tells you how the compiler works the sorry it's what the compiler talks to okay yeah and then uh the tooling you mentioned the disperate tools are for what for for when you're building a specific chip so RIS five in Hardware in Hardware yeah so so RIS five you can buy rist 5 Core from scif and say Hey I want to have a certain number of run a certain number of gigahertz I want it to be this big I want to be have these features I want to have um like I want floating point or not for example um and then what you get is you get a description of a CPU with those characteristics now if you want to make a chip you want to build like an iPhone chip or something like that right you have to take both the CPU but then you have to talk to memory you have to have timers iOS a GPU other components and so you need to pull all those things together into what's called an Asic an application specific grade circuit so a custom chip and then you take that design and then you have to transform it into something that the Fabs like tsmc for example know how to turn take to production got it so but yeah okay and and so that process I will I can't help but see it is is a big compiler okay it's a whole bunch of compilers written without thinking about it through that lens isn't isn't the universe a compiler in that like comp compilers do two things they represent things and transform them yeah and so there's a lot of things that end up being compilers but this is this is a space where we're talking about design and usability and the way you think about things the way things composed correctly it matters a lot and so sci-fi is investing a lot into that space and we think that there's a lot lot of benefit that can be made by allowing people to design chips faster get them to Market quicker and um scale out because um you know it the alleged more end of Mor's law uh you've got this problem of uh you're not getting free performance just by waiting another year for a faster CPU and so um you have to find performance in other ways and one of the ways to do that is with custom accelerators and other things and hardware and and so well we'll talk a little about uh a little more about as6 but um do you see that a lot of people a lot of companies will try to have a like different sets of requirements that this whole process to go for so like like almost different car companies might use different uh and like different uh PC manufacturers like so is this like is risk 5 um in this whole process is it potentially the future of all Computing devices yeah I think that so if you look at risk 5 and step back from the Silicon side of things RIS 5 is an open standard and one of the things that has happened over the course of decades if you look over the long Arc of computing somehow became decades old yeah is that you have uh companies that come and go and you have instruction sets that come and go like one example of this out of many is uh uh sun with spark yeah Sun one away spark still lives on it Fujitsu but we have uh HP had this instruction set called PA risk so P risk was its big server business and had tons of customers they decided to move to this called itanium from Intel yeah this didn't work out so well yeah right and so you have this issue of you're making many billion doll Investments on instruction sets that are owned by a company and even companies as big as Intel don't always execute as well as they could they have their own issues um HP for example decided that it wasn't in their best interest to continue investing in the space because it was very expensive and so they make technology decisions or they make their own business decisions and this means that a customer what do you do you've sunk all this time all this engineering all the software work all these you've built other products around them and now you're stuck right what risk 5 does is it provides you more optionality in the space because if you buy uh an implementation of RIS five from SciFi and you should they're the best ones yeah um uh but if something bad happens to SciFi in 20 years right well great you can turn around and buy r five core from somebody else and there's an ecosystem of people people that are all making different risk five cores with different trade-offs which means that if you have more than one requirement if you have a family of products you can probably find something in the rist five space that fits your needs whereas with if you're talking about xa6 for example it's Intel's only going to bother to make certain classes of devices right I see so uh maybe a weird question but like if SciFi is uh like infinitely successful in the next 20 30 years what does the world look like so like how does the world of computing change so too much diversity in Hardware instruction sets I think is bad like we have a lot of people that are using um lots of different instruction sets particularly in the embedded the like very tiny microcontroller space the thing in your toaster um that uh that are just weird and different for historical reasons and so the compilers and the tool chains and the languages on top of them uh aren't there right and so the Developers for that software have to use really weird tools because the ecosystem that supports is not big enough so I expect that will change right people will have better tools and better languages better features everywhere that then can service many different points in the space um and I think RIS 5 will progressively um eat more of the ecosystem because it can scale up it can scale down sideways left right it's very flexible and very well considered welld designed and instruction set um I think when you look at sci-fi tackling silicon and how people build chips which is a very different space um that's where you say I think we'll see a lot more custom chips and that means that you get much more battery life you get better better tuned solutions for your iot thingy you get you get people that move faster you get the ability to have faster time to market for example so how many custom so first of all on iot of things do you see the number of smart toasters increasing exponentially so uh and and if you do like how much customization per toaster is there do all toasters in the world run the same uh silicon like the same design or is it different companies have different design like how how much customization is possible here well a lot of it comes down to cost right and so the way that chips work is you end up paying by the one one of the factors is the the size of the Chip And so what ends up happening just from an economic perspective is there's only so many chips that get made in any year of a given design and so often what customers end up having to do is they end up having to pick up a chip that exists that was built for somebody else so they can then ship their product and the reason for that is they don't have the volume of the iPhone they can't afford to build a custom chip however what that means is they're now buying an off-the-shelf chip that isn't really good that isn't a perfect fit for their needs and so they're paying a lot of money for it because they're buying silicon that they're not using well if you now reduce the cost of Designing the chip now you get a lot more chips and the more you reduce it the the easier it is to design chips um The More The World Keeps evolving and we get more AI accelerators we get more other things we get more uh standards to talk to we get 6G right you get you get you get changes in the world that you want to be able to talk to these different things there's more diversity in the cross product of features that people want and um that drives differentiated chips in different in another Direction and so nobody really knows what the future looks like but um but I think that there's a lot of silicon in the future speaking of the future uh you said Mo's law allegedly is dead so do you think do you agree with uh uh Dave Patterson and and many folks that Mo's law is dead or do you agree with Jim Keller who says uh who's uh standing at the Helm of the pirate ship saying it's uh still alive it's still alive yeah also I agree with what they're saying and different people are interpreting the anor's law in different ways yeah so Jim would say you know there's another thousand X left in physics and we can we can continue to squeeze the stone and make it faster and smaller and smaller geometries and all that kind of stuff uh he's right so Jim Jim is is absolutely right that there's a ton of ton of progress left and we're not at the limit of physics yet um uh that's not really what mors law is though if you look at what mors law is is that it's a very simple uh evaluation of okay well you look at the cost per um I think it was cost per area and the most economic point in that space and if you go look at the the the now quite old paper that describes this um mors law has a specific economic aspect to it and I think this is something that Dave and others often point out and so on a technicality that's right um I look at it from so I can acknowledge both of those viewpoints they're both right they're both right I'll give you a third wrong yeah Viewpoint that may be right in its own way which is um single threaded performance doesn't improve like it used to and it used to be back when you got a uh you know a pennium 66 or something and the year before you had a pennium 33 and now it's twice as fast MH right well it was twice as fast at doing exactly the same thing okay like literally the same program ran twice as fast you just wrote a check okay and waited a year year and a half well so that's what a lot of people think about Moors law and I think that is dead and so what we're seeing instead is we're pushing we're pushing people to write software in different ways and so we're pushing people to write Cuda so they can get GPU compute and the the thousands of cores on GPU we're talking about C programmers having to use P threads because they now have you know 100 100 threads or 50 cores in a machine or something like that um you're now talking about machine learning accelerators they're now domain specific and when you look at these kinds of use cases you can still get performance um and Jim will come up with cool things that uh utilize the Silicon in new ways for sure but you're also going to change the programming model right and now when you start talking about changing the programming model that's when you come back to languages and things like this too because often what you see is um like you take the C programming language right the C programming language is designed for CPUs and so if you want to talk to a GPU now you're talking to its cousin Cuda okay Cuda is a different thing with a different set of tools a different world a different way of thinking and we don't have one world that scales and I think that we can get there we can have one world that scales in a much better way on a small tangent then I think most programming languages are designed for CPUs for single core even just in their Spirit even if they allow for paralyzation so what does it look like for programming language to have um paralyzation or massive parallelization as it's like first principle so the canonical example of this is the hardware design world so verog vhdl these kinds of languages they're what's called a uh highle synthesis language this is the thing people design chips in and when you're designing a chip it's kind of like a brain where you have infinite parallelism like you've got you're you're you're like laying down transistors transistors are always running okay yeah and so you're not saying run run this transistor then this transistor than this transistor it's like your brain like your neurons are always just doing something they're not clocked right they're they're just they're just doing they're they're doing their thing and so when you design a chip or when you design a CPU when you design a CPU when you design when you're laying down the transistors uh similarly you're talking about well okay well how do these things communicate and so these languages exist verog is um a kind of mixed example of that none of these languages are really great either very low level yeah yeah they're very low level and abstraction is necessary here and there's different different approaches at that and it's a it's itself a very complicated world but um but it's implicitly parallel and so having that as a as the domain that you uh program towards makes it so that by default you get parallel systems if you look at Cuda Cuda is a point halfway in the space where in Cuda when you write a Cuda kernel for your GPU it feels like you're writing a scaler program so you're like you have ifs you have for Loops stuff like this you're just writing normal normal code but what happens outside of that in your driver is that it actually is running you on like a thousand things at once right and so it's it's parallel but it has pulled it out of the programming model and so so now you as a programmer are working at a in a simpler world and it's solved that for you right how do you take the language like Swift um you know if we we think about gpus but also ASX maybe if we can dance back and forth between hardware and software uh is you know how do you design for these features to be able to program make it a first class citizen to be able to do like Swift for tensor flow to be able to do machine learning on current Hardware but also future Hardware like uh dpus and all kinds of as6 that I'm sure will be popping up more yeah well so so a lot of this comes down to this whole idea of having the nuts and bolts underneath the covers that work really well so you need if you're talking to tpus you need you know ml or xlaa or one of these compilers that talks to tpus to build on top of okay and if you're talking to circuits you need to figure out how to lay down the transistors and how to organize it and how to set up clocking and like all the domain problems that you get with uh circuits then you have to decide how to explain it to a human what is the UI right and if if you do it right that's a library problem not a language problem and that works if you have a library or a language which allows your library to write things that feel native in The Language by implementing libraries because then you can innovate in programming models without having to change your syntax again and like have to invent new code formatting tools and like all the other things that languages come with and this this gets really interesting and so um if you look at the space the interesting thing once you separate out syntax becomes what is that programming model and so do you want the Cuda style I write one program and it runs many places the um do you want the implicitly parallel model how do you reason about that how do you give developers you know chip Architects the the ability to express their intent and that comes into this whole design question of how do you detect bugs quickly so you don't have to tape out a chip to find out it's wrong ideally right how do you and and you know this is a spectrum how do you make it so that people feel productive so their turnaround time is very quick all these things are really hard problems and um in this world I I think that not a lot of effort has been put into that design problem and thinking about the layering in other pieces well you've uh on the topic of concurrency you've written the Swift concurrency Manifesto I think it's it's kind of interesting anything that uh has the word manifesto in is very interesting can you summarize the key ideas of U each of the five parts you written about so what is a Manifesto yes how about we start there uh so in the Swift Community we have this um problem which is on the one hand you want to have relatively small proposals that you can kind of fit in your head you can understand the details at a very fine grain level that move the world forward but then you also have these big arcs okay and often when you're working on something that is a big Arc but you're tackling in small pieces you have this question of how do I know I'm not doing a random walk where are we going like how does this add up furthermore when you start that first the first small step what terminology do you use how do we think about it what is better and worse in the space what are the principles what are we trying to achieve and so what a Manifesto in the Swift Community does is it starts to say hey well let's step back from the details of everything let's paint a broad picture to talk about how what we're trying to achieve let's give an example design Point let's try to paint the big picture so that then we can zero in on the individual steps and make sure that we're making good progress and so the Swift concurrency Manifesto is something I wrote three years ago it's been a while maybe maybe more um trying to do that for for Swift and concurrency and it starts with some fairly uh simple things like making the observation that when you have multiple different computers or multiple different threads that are communicating it's best for them to be asynchronous right and so you need things to be able to run separately and then communicate with each other and this means asynchrony and this means that uh you need a way to modeling asynchronous communication uh many languages have features like this uh asyn a weight is a popular one and so that's what I think is very likely in Swift um but as you start building this Tower of abstractions it's not just about how do you write this you then reach into the how do you get memory safety because you want correctness you want debuggability and Sanity for developers and how do you get uh that memory safety into um into the language so if you take a language like go or uh C or any of these languages you get what's called a race condition when two different threads or go routines or whatever touch the same point in memory right this is a huge like maddening problem to debug because uh it's not reproducible generally and so there's tools there's a whole ecosystem of solutions that built up around this but it's it's a huge problem when you're writing concurrent code and so with Swift uh this whole value sematics thing is really powerful there because it turns out that math and copies actually work even in concurrent worlds and so um you get a lot of safety just out of the box but there are also some hard problems and it talks about some of that um when you start building up to the next level up and you start talking Beyond memory safety you have to talk about what is a programmer model how does a human think about this so a developer that's trying to build a program think about this and it proposes a really old model with a new spin called actors actors are about saying we have islands of single threaded logically so you write something that feels like it's one programming one program running in a unit and then it communicates asynchronously with other other things and so making that expressive and natural feel good be the first thing you reach for and being safe by default is a big big part of the design of that proposal when you start going beyond that now you start to say cool well these things that communicate asynchronously they don't have to share memory well if they don't have to share memory and they're sending messages to each other why do they have to be in the same process these things should be able to be in different processes on your machine and why just processes well why not different machines and so now you have a very nice gradual transition towards distributed programming and of course when you start talking about the the big the big future the the manifesto doesn't go into it but uh accelerators are asyn things you talk to asynchronously by sending messages to them and how do you program those well that that gets very interesting um that's not that's not in the proposal so but and uh how much do you want to make that explicit like the control of that whole process explicit to the programmer yeah good question so when when you're designing any of these kinds of features or language features or even libraries you have this really hard trade-off that you have to make which is how much is it magic or how much is it in the human's control how much can they predict and control it what do you do when the default case is the wrong case okay and so when you're designing a system um uh I won't name names but there there are systems where um you it's really easy to get started and then you you jump so let's pick like logo okay so something like this so it's really easy get start it's really designed for uh teaching kids but as you get into it you hit a ceiling yeah and then you can't go any higher and then what do you do well you have to go switch to a different world and rewrite all your code and this logo is a silly example here this exists in many other languages uh with python you would say uh uh like concurrency right so python has the global interpreter lock so threading is challenging in Python and so if you if you start writing a large scale application in Python and then you need concurrency you're kind of stuck with the series of bad trade-offs right um uh there's other ways to go where you say like voice all the all the complexity on the user all at once right and that's also bad in a different way and so what what I what I prefer is building a simple model that you can explain that then has an escape hatch so you get in you have guard rails you uh memory safety works like this in Swift where you can start with you like by default if you use all the standard things it's memory safe you're not going to shoot your foot off but if you want to get a uh a c-level pointer to something you can explicitly do that but by default it's uh there's guard rails there's guard rails okay so but like you know uh whose job is it to figure out which part of the code is paralyzable um so in the case of the proposal it is the human's job so they decide how to architect their application and then uh the runtime in the compiler is very predictable and so this this is in contrast to um like there's a long body of work including on Fortran for auto parallelizing compilers and um this is an example of a bad thing and my so as a compiler person I can rag on compiler people um often compiler people will say cool since I can't change the code I'm going to write my compiler that then takes this unmodified code and makes go way faster on this machine MH okay application develop and so it does pattern matching it does like really deep analysis compiler people are really smart and so they like want to like do something really clever and tricky and you get like 10x speed up by taking like an array of structures and turn it into a structure of arrays or something because it's so much better for memory like there's bod like tons of Tricks yeah um they love optimization yeah you love optimization everyone loves optimization everyone loves it well and it's it's just this promise of build with my compiler and your thing goes fast yeah right but here here's the problem Lex you write you write program M you run it with my compiler it goes fast you're very happy wow it's so much faster than the other compiler yeah then you go and you add a feature to your program or you refactor some code and suddenly you got a 10x loss in performance well why what just happened there what just happened there is you the theistic the the the pattern match and the compiler whatever analysis it was doing just got defeated because you didn't inline a function or or or something right as a user you don't know you don't want to know that was the whole point you don't want to know how the compiler works you don't want to know how how the memory hierarchy works you don't want to know how it got parallelized across all these things you wanted that abstractor away from you but then the magic is lost as soon as you did something and you fall off a performance cliff and now you're in this funny position where what do I do I don't change my code I don't fix that bug it cost 10 10x performance now what do I do well this is the problem with unpredictable performance right if if you care about performance predictability is a very important thing and so um and so what the what the proposal does is it provides a architecture patterns for being able to lay out your code gives you full control over that makes it really simple so you can explain it and then um and then if you want to scale out in different ways you have full control over that so in your sense the intuition is for a compiler too hard to do automated parallelization like you know cuz the compilers do stuff automatically that's incredibly impressive for other things right but for parallelization we're not even we're not close to there well it it depends on the programming model so compile there's many different kinds of compilers and so if you talk about like a c compiler or a swift compiler or something like that where you're writing imperative code parallelizing that and reasoning about all the pointers and stuff like that is very is a very difficult problem now if you switch domains so there's this cool thing called machine learning right so the machine the machine learning nerds among other endearing things like you know solving cat detectors and other things like that um have done this amazing breakthrough of producing a programming model operations that you compose together mhm that has raised level of abstraction high enough that suddenly you can have autop paralyzing compilers you can write and model using tensor flow and have it run on 1,24 nodes of a TPU yeah that's true I didn't even think about like you know CU there's so much flexibility in the design of architectures that ultimately boil down to a graph that's paralyzable for you par for you and and if you think about it that's pretty cool and you think about batching for example as a way of being able to exploit more parallelism yeah like that's a very simple thing that now is very powerful that didn't come out of the programming language nerds right those people like that came out of people that are just looking to solve a problem and use a few gpus and organically developed by the community of people focusing on machine learning and it's an incredibly power powerful abstraction layer that enables the compiler people to go and exploit that and you can drive supercomputers from python that's that's pretty cool that's amazing so just to pause on that I cuz I'm not sufficiently low level I forget to admire the beauty and power of that but um maybe just to linger on it like what what does it take to run a neural network fast like how hard is that compilation it's really hard um so we just skipped you said like it's amazing that that's a thing but yeah how hard is that of a thing it's it's hard and I I would say that not all the systems are really great including the ones I help build so there's a lot of work left to be done there is it the compiler nerds working on that or is it a whole new group of people well it's it's a full stack problem including compiler people um in including apis so like Caris and the the the the module API and pytorch and Jack and there's a bunch of people pushing on all the different parts of these things because when you look at it is it's both how do I express the computation do I stack up layers well cool like setting up a linear sequence of layers is great for the simple case but how do I do the hard case how do I do reinforcement learning well now I need to integrate my application logic in this right then it's you know the next level down of how do you represent that for the runtime how do you get Hardware abstraction and then you get to the next level down of saying like forget about abstraction how do I get the Peak Performance out of my TPU or my iPhone accelerator or whatever right all these different things and how and so this is a layered problem with a lot of really interesting uh design and work going on in the space and a lot of really smart people working on it uh machine learning is a very well-funded area of investment right now and so there's a lot of progress being made so how much Innovation is there on the lower level so closer to the to the as6 so redesigning the hardware or redesigning concurrently compilers with that Hardware is that like if you were to predict the biggest uh you know the equivalent of Moors law improvements in the inference in the training of your own networks and just all of that where is that going to come from you think sure you get scalability have different things and so you get um you know Jim Keller shrinking process technology you get 3 nanometer instead of five or seven or 10 or 28 or whatever um and so that that marches forward that provides improvements you get uh architectural level performance and so the you know a TPU with a matrix multiply unit and a systolic array is much more efficient than having a scaler core doing multiplies and adds and things like that you then get um uh uh system level improvements so how you talk to memory how you talk across a cluster of machines how you scale out how you have fast interconnects between machines you then get system level programming models so now that you have all this Hardware how to utilize it you then have algorithmic breakthroughs where you say hey wow cool instead of training in uh you know resonant 50 and uh a week I'm now training it in you know 25 seconds yeah and Comin It's a combination of uh you know new new optimizers and new new new just training regimens and different different approaches to train and and all of these things come together to to push the world forward that that was a a beautiful exposition of but if you were to uh Force to bet all your money on one of these would you why do we have to that's unfortunately we have people working on all this it's an exciting time right so I mean you know open the eye did this little paper showing the algorithmic Improvement you can get has been you know improving exponentially uh I haven't quite seen the same kind of analysis on other layers of the stack I'm sure it's also improving significantly I just it's a it's a nice intuition Builder I mean there's a reason why Moore's Law that's the beauty of Mo's law is somebody writes a paper that makes a ridiculous prediction yeah and it you know becomes reality in a sense there's there's something about these narratives when you uh uh when Chris L on a silly little podcast makes bets all his money on a particular thing somehow it can have a ripple effect of actually becoming real that's an interesting aspect of it cuz like it might have been uh you know we focus with Mor's law most of the Computing industry really really focused on the hardware I mean software Innovation I don't know how much software Innovation there was in terms of Intel giveth Bill takes away right yeah I mean compiler has improved significantly also right well not not really so actually I mean so I'm joking about how uh software's gotten slower pretty much as fast as Hardware got better at least through the 90s um there's another joke another law in compilers which is called uh I think it's called probstein law which is uh compilers double the performance of any given code every 18 years so they move slowly yeah well so well well yeah it's exponential also yeah you're making progress but but there again it's not about um the the power of compilers is not just about how do you make the same thing go faster it's how do you unlock the new hardware right a new chip came out how do you utilize it you say oh the programming model how do we make people more productive how do we how do we uh like have better error messages even such mundane things like how do I generate a very specific error message about your code actually makes people happy because then they know how to fix it right it comes back to how do you help people get their job done yeah and yeah and then in this world of exponentially increasing smart toasters how do you uh expand Computing to uh to all all these kinds of devices do you see this world where just everything's a Computing surface you see that possibility just everything a computer yeah I don't see any reason that that couldn't be achieved it turns out that sand goes into glass and glass is pretty useful too and you know like why not why not so uh very important question then if um if we're living in a simulation and the simulation is running a computer like what what's the architecture of that computer do you think so you're saying is it is it a Quantum system is it a yeah like this whole Quantum discussion is it needed or can can we run it on a on a you know with a risk 5 architecture uh a bunch of CPUs I think it comes down to the right tool for the job okay and so and what's the compiler yeah exactly that's that's my question how do I get that job be the universe compiler um uh and so there as far as we know Quantum Quantum Quantum systems are the bottom of the T pile of so far yeah and so we don't know efficient ways to implement Quantum systems without using quantum computers yeah and that's totally outside of everything we've talked about Quantum but who runs that quantum computer yeah right so if it if it if we really are living in a simulation then is it bigger quantum computers is it different ones like how how does that work out how does that scale well it's it's the same size it's the same size but then but then the thought of the simulation is you don't have to run the whole thing that you know we humans are cognitively very limited checkpoints checkpoints yeah and uh and if we the point at which we human so you basically do minimal amount of uh what is it uh Swift does um on right copy copy on right yeah so you only you only adjust the simulation par parallel universe theories right and so and so every time a a decision is made somebody opens the Shor in your box then there's a fork this could happen and then uh thank you for uh for considering the possibility but yeah so it may not require you know the entirety of the universe to simulate it but it's um interesting to think about uh as we create this this higher and higher Fidelity systems but I do want to ask on the on the quantum computer side because everything we've talked about with uh with you work with sci-fi with every with compilers none of that includes quantum computers right that's true so have you ever thought about uh what a you know this whole serious engineering work of quantum computers looks like of compilers of architectures all of that kind of stuff so I've looked at a little bit I know almost nothing about it which means that at some point I will have to find an excuse to get involved because that's how do you think do you think that's a thing to be like is was your little senses of the timing of when to be involved is it not yet well so so the thing I do really well is I jump into messy systems and figure out how to make them figure out what the truth in the situation is try to figure out what um what the unifying theory is how to like Factor the complexity how to find a beautiful answer to a problem that um has been well studied and lots of people have bashed their heads against it I don't know that quantum computers are mature enough and accessible enough to be um figured out yet right and um the uh I think the open question with quantum computers is is there a useful problem that gets solved with a quantum computer that makes it worth the economic cost of like having one of these things and having having Legions of people that that that uh set it up you go back to the 50s right and there's the projections of the world can will only need seven seven computers right well and part of that was that people hadn't figured out what they're useful for what are the algorithm we want to run what are the problems to get solved and this comes back to how do we make the world better either economically or making somebody's life better or like solving a problem that wasn't solved before things like this and um I think that just we're a little bit too early in that development cycle because it's still like literally a science project not any negative connotation right it's literally a science project and um the progress there is amazing and so I don't know if it's 10 years away if it's 2 years away exactly where that breakthrough happens but um you look at uh machine learning it we went through a few winners um before the Alex net transition and then suddenly it had its breakout moment and that was the Catalyst that then drove the talent flocking into it that's what drove the economic applications of it that's what drove the um the technology to go faster because you now have more Minds thrown at the problem this is what caused uh like a serious knee and uh deep learning and the algorithms that we're using and um and so I think that's what Quantum needs to go through and so right now it's in that that formidable finding itself getting the the like literally the physics figured out and um and and then has to figure out the application that makes this useful like right now I'm I'm not skeptical that I think that will happen I think it's just you know 10 years away something like that I forgot to ask what programming language do you think the simulation is written in O probably lisp so not sft like if you were to bet you were to bet uh I'll just leave it at that so I mean we've mentioned that you work with all these companies we we've talked about all these projects it's kind of if we just step back and zoom out about the way you did that work and we look at covid times this pandemic we're living through that may if I look at the way Silicon Valley folks are talking about it the way MIT is talking about it this might last for a long time uh not just the virus but the the remote nature the economic impact I all it yeah yeah it's it's going to be a mess do you think uh what's your prediction I mean from sci-fi to Google to uh uh to just all the places you worked in just Silicon Valley you're in the middle of it what do you think is how is the whole place going to change yeah so I mean I I really can only speak to the tech perspective I am in that bubble um I think it's going to be really interesting because the you know the zoom culture of being remote and on video chat all the time has really interesting effects on people so on the one hand it's a great normalizer it's a normalizer that I think will help communities of people that have traditionally been underrepresented uh because now you're taking in some cases a face off you don't have to have a camera going right and so you can have conversations without physical appearance being part of the part of the dynamic which is pretty powerful you're taking remote employees that have already been remote and you're saying you're now on the same level and foot footing as everybody else nobody gets whiteboards you're not going to be the one person that doesn't going to be participating in the Whiteboard conversation and that's pretty powerful um you've got uh you're forcing people to think uh asynchronously in some cases because it's harder to just just get people physically together and the bumping into each other forces people to find new ways to solve those problems s and I think that that leads to more inclusive Behavior which is good um on the other hand it's also it just sucks right and so um the the nature the the actual communication or just sucks being not in with people like on a daily basis and collaborating with them yeah all of that right I mean everything this whole situation is terrible um what I meant primarily was the um I think that that most humans like working physically with humans I think this is something that not everybody but many people are programmed to do and I think we get something out of that that it's very hard to express at least for me and so maybe this isn't true of everybody but um and so the question to me is you know when you get through that time of adaptation right you get out of March and April and you get into December and you get into next march if it's not changed right it's already terrifying well you you think about that and you think about what is the nature of work yeah right how do how do we adapt and humans are very adaptable species right we can we can learn things and when we're forced to and there's a catalyst to make that happen and so what is it that comes out of this and are we better or worse off right I think that you know you look at the Bay Area housing prices are insane well why well there's a high incentive to be physically located because if you don't have proximity you end up paying for it and commute right and there's there has been huge social social pressure in terms of like you will be there for the meeting right or whatever scenario it is and I think that's going to be way better I think it's going to be much more than Norm to have remote employees and I think this is going to be really great do you uh do you have friends or do you hear of people moving yeah I I know one family friend that moved they moved back to Michigan and uh you know they were a family with three kids living in a small apartment and like we're going insane right and they're in Tech uh husband works for Google so first of all friends of mine have are in the process of or are have already lost the business the thing that represents their passion their dream it could be small entrepreneur projects but it could be large businesses like people that run gyms like do restaurants like tons of things yeah so but also people like look them at themselves in the mirror and ask the question of like what do I want to do in life for some reason they don't they haven't done it until Co like they really asked that question and that results often in moving or leaving the company you're with starting your own business or transitioning to different company do you think we're going to see that a lot like in um I I well I can't speak to that I mean we're definitely going to see it at a higher frequency than we did before um just because I think what you're trying to say is there are decisions that you make yourself and big life decisions that you make yourself and like I'm going to like quit my job and start a new thing there's also decisions that get made for you like I got fired from my job what am I going to do right and that's not a decision that you think about but you're forced to act okay and so I think that those you're forced to act kind of moments where like you know Global pandemic comes and wipes out the economy and now you're business doesn't exist I think that does lead to more reflection right because you're less anchored on what you have and it's not a what do I have to lose versus what do I have to gain AB comparison it's more of a fresh slate cool well I could do anything now do I want to do the same thing I was doing did that make me happy is this now time to go back to college and take a class and learn learn a new skill is this is this a time to uh spend time with family if you can afford to do that is this time to like you know literally move in with parents right I mean all these things that were not normative before suddenly become I think uh very the value system has changed and I think that's actually a good thing in the short term at least because um it leads to you know there's kind of been an over optimization along one one set of priorities for the world and now maybe we'll get to a more balanced and more interesting world where the people are doing different things I think it could be good I think there could be more Innovation that comes out of it for example what do you think about the all the social chaos we're in the middle of like it sucks you think it's uh let me ask you I hope you think it's all going to be okay well I think Humanity survive um the from an existential like we're not all going to kill Yeah well yeah I don't think the virus is going to kill all all the humans um I don't think all the humans are going to kill all the humans I think that's unlikely but um I I look at it as uh um progress requires a catalyst right so so you need you need a reason for people to be willing to do things that are uncomfortable I think that the US at least but I think the world in general is a pretty uh uh unoptimal place to live in for a lot of people and I think that what we're seeing right now is we're seeing a lot of unhappiness and because because of all the pressure because of all the the Badness in the world that's coming together it's really kind of igniting some of that debate that should have happened a long time ago right I mean I think that we'll see more progress you're asking about offline you're asking about politics and wouldn't be great if politics move faster because there's all these problems in the world and we can move it well people are intentional or inherently uh conservative and so if you're talking about conservative people particularly if they have heavy burdens on their shoulders because they represent literally thousands of people um it makes sense to be conservative but on the other hand when you need change how do you get it the global pandemic will probably lead to some change and it's not a directed it's not a directed plan but I think that it leads to people asking really interesting questions and some of those questions should have been asked a long time ago well let me know if if you observed this as well something that's bothered me in the machine Learning Community I'm guessing it might be prevalent in other places is um something that feels like in 2020 increase level of toxicity like people are just quicker to pile on they just be they're just harsh on each other to to like mob uh pick a person that screwed up and like make it a big thing yeah and uh is there something that we can like have you observed that in other places is there is there some way out of I think there's a inherent thing in humanity that's kind of an Us Versus Them thing which is that you want to succeed and how do you succeed well it's relative to somebody else and so what what's happening in at least in some part is that with the internet and with online communication the world's getting smaller right and so we're having some of the the social ties of like my NE My Town versus your Town's football team right turn into much larger larger and yet shallower problems and uh people don't have time the incentives are clickbait and like all these things kind of really really feed into this machine and I don't know where that goes um yeah I mean the reason I think about that I I mentioned to you this offline a little bit but uh you know I have uh a few difficult conversations scheduled some of them political related some of them within the community uh difficult personalities that went through some stuff I mean one of them I've talked before I will talk again is Yan laon he got of a little bit of crap on Twitter for uh for uh talking about a particular paper and the bias within a data set and then there's been a huge uh in my view and I'm willing comfortable saying it uh irrational ere exagger ated pylon on his comments because uh he made pretty basic comments about the fact that if there's bias in the data there's going to be bias in the results so we should not have bias in the data but people piled on to him because he said he trivialize the problem of bias like it's a lot more than just bias and the data but like yes that's a very good point but that's that's not what he was saying that's not what he was saying and the response like the imply response that he's basically sexist and racist um is uh is something that completely drives away the possibility of nuance discussion one nice thing about like a podcast long form uh conversation is you can talk it out you can lay your reasoning out and even if you're wrong you can still show that you're a good human being underneath it you know your point about you can't have a productive discussion well how do you get to that point where people can turn they can learn they can listen they can think they can engage versus just being a a shallow like like and then keep moving right and I don't think that that uh progress really comes from that right and I don't think that um one should expect that I think that you you'd see that as reinforcing individual circles and the US versus them thing and I think that's fairly divisive yeah I think uh there's a big role in like the people that bother me most on Twitter when I observe things is not the people who get very emotional angry like over the top it's the people who like prop them up it's all the it's it's that I think what should be the we should teach each other is to be sort of empathetic the the thing that it's really easy to forget particularly on like Twitter or the internet or in email is that sometimes people just have a bad day yeah right you have a bad day or you're like I've been in the situation where it's like between meetings like fire off a quick response to an email because I want to like help get something unblocked phrase it really objectively wrong I screwed up and suddenly this is now something that sticks with people and it's not because they're bad it's not because you're bad it's just psychology of like you said a thing um it sticks with you you didn't mean it that way but it really impacted somebody because the way they interpreted it and this is just an ECT of working together as humans and I have a lot of optimism in the long term the very long term about what we as Humanity can do but I think that's going to be it's just always a rough ride and you you came into this by saying like what do Co and all the the social Strife that's happening right now mean and I think that it's really bad in the short term but I think it'll lead to progress and for that I'm very thankful yeah it's painful in the short term though well yeah I mean people are out of jobs like some people can't eat like it's horrible and um but but you know it's progress so we'll see we'll see what happens I mean the the real question is when you look back 10 years 20 years 100 years from now how do we evaluate the decisions are being made right now I think that's really the way you can frame that and look at it and you say you know you integrate across all the short-term horribleness that's happening and you look at what that means and is the you know Improvement across the world or the regression across the world uh significant enough to make it a good or bad thing I think that's the question yeah and for that it's good to study history I'm one of the big problems for me right now is I'm reading the rise and fall of the Third Reich Light reading so it's everything is just I just see parallels and every I mean it's it's you have to be really careful not to overstep it but just the the thing that worries me the most is the pain that people feel when of com when a few things combined which is like economic depression which is quite possible in this country and then just being disrespected yeah uh by in some kind of way which the German people were really disrespected by most of the world uh like in a way that's over the top that something can it can build up and then all you need is a charismatic leader uh to to go either positive or negative and both work as long as they're charismatic and there it's taking advantage of again that that inflection point that the world's in and what they do with it could be good or bad and so it's a good way to think about times now like on an individual level what we decide to do is when when history is written you know 30 years from now what happened in 2020 probably history's going to remember 2020 yeah I think so either for good or bad and it's like up up to us to write it so it's good well one of the things I've observed that I find F is most people act as though the world doesn't change you make decision knowingly right you make a decision where you're predicting the future based on what you've seen in the recent past and so if something's always been H it's rained every single day then of course you expect it to rain today too right on the other hand the world changes all the time yeah constantly like for better and For Worse right so the question is if you're interested in something that's not right what is the INF point that led to a change and you can look to history for this like what is what is the Catalyst that led to that that explosion that led to that bill that led to the like you you can kind of work your way backwards from that and maybe if you pull together the right people and you get the right ideas together you can actually start driving that change and doing in a way that's productive and hurts fewer people yeah like a single person single event can turn all of absolutely everything starts somewhere and often It's a combination of multiple factors but but yeah this is these these things can be engineered that's actually the optimistic view that I'm I'm a long-term Optimist on pretty much everything and human nature you know we can look to all the negative things that that Humanity has all the pettiness and all the like self self-serving and the um just the the cruelty right the the biases the just humans can be very horrible but on the other hand we're capable of amazing things and um and the progress across you know hundred-year chunks is striking and even across decades it's we've come a long ways and there's still a long ways to go but that doesn't mean that we've stopped yeah the kind of stuff we've done in the last 100 years is is unbelievable it's kind of scary to think what's going to happen next 100 year it's scary like exciting like scary in a sense that it's kind of sad that the kind of technology is going to come out in 10 20 30 years will probably too old to really appreciate cuz you don't grow up with it it'll be like kids these days with their virtual and their uh their Tik toks and stuff like this like how do this thing and like come on give me my uh you know static photo you know my Commodore 64 yeah yeah exactly okay uh sorry we kind of skipped over but let me ask on um you know the machine learning world has been kind of inspired their imagination captivated with gpt3 and these language models I thought it'd be cool to get your opinion on it what what's your thoughts on this exciting world of um it connects to computation actually uh is of language models that are huge yeah and take multip many many computers not just the train but to also do inference on sure well I mean it depends on what you're speaking to there but I mean I think that there's been a pretty well understood maximum in deep learning that if you make the model bigger and you shove more data into it assuming you train it right and you have a good model architecture that you'll get a better model out and so on the one hand gpg 3 was not that surprising um on the other hand a tremendous amount of engineering went into making it possible um the implications of it are pretty huge I think that when gpt2 2 came out there was a very provocative blog post from open AI talking about you know we're not going to release it because of the social damage it could cause if it's misused um I think that's still a concern I think we need to look at how um technolog is applied and you know well-meaning tools can be applied in very horrible ways and they can have very profound impact on that um uh I think the gpt3 is a huge technical achievement and what will GPT 4 be will probably be bigger and more expensive to train really cool uh architectural tricks do what do you think is there um I don't know how much thought you've done on distributed computing uh is there is there some technical challenges that are interesting that you're hopeful about exploring in terms of you know a system that like a piece of code that you know GPT 4 uh that might have I don't know uh hundreds of trillions of parameters we have to run on thousands of computers is there some is there some hope that we can make that happen yeah well I mean today you can you can write a check and get access to 1,000 TPU cores and do really interesting large scale training and inference and things like that um in Google Cloud for example right and um so I don't think it's a question about scale it's a question about utility and when I look at the Transformer series of architectures that that the GPT series is based on it's really interesting to look at that because they're actually very simple simple designs they're not recurrent um the training regen are pretty simple um and so they don't really reflect like human brains MH right um but they're really good at learning language models and they're unrolled enough that you get you can simulate some recurrence right and so the question I think about is where does this take us like so we can just keep scaling it have more parameters more data more things we'll get a better result for sure but are there architectural techniques that can lead to progress at a faster Pace right this is when you know how do you get uh instead of just like making it constant time bigger how do you get like an algorithmic improvement out of this right and whether it be a new training regimen if it becomes um uh sparse sparse networks for example the human brain is sparse all these networks are dense um the connectivity patterns can be very different I think the this is where I get very interested and I'm way out of my league on the Deep learning side of this but I think that could lead to Big breakthroughs when you talk about uh large scale networks one of the things that Jeff Dean likes to talk about and he's uh uh giv a few talks on is this idea of having a sparsely gated mixture of experts kind of a model where you have um you know different Nets that are trained and are really good at certain kinds of tasks and so you have this distributor across a cluster and so you have a lot of different computers that end up being kind of locally specialized in different domains and then when a query comes in you you gate it and you use learn techniques to route to different parts of the network and then you utilize the compute resources of the entire cluster by having specialization within it and I don't know where that goes or if it starts to when it starts to work but I think things like that could be really interesting as well and then on the data side too if you can think of data selection as a kind of programming yeah I mean at the essentially if you look at like Kathy talked about software 2.0 I mean that in a sense data is the programming yeah yeah so I I just so let me try to summarize Andre's position really quick before I disagree with it yeah um so Andre Kathy is amazing so this is nothing nothing personal with him he's he's he's an amazing engineer and and also a good uh blog post writer yeah well he's a great communicator I he's just an amazing person he's he's also really sweet um so his his basic premise is that uh software is suboptimal I think we can all agree to that uh he also points out that uh deep learning and other learning based techniques are really great because you can solve problems in uh more structured ways uh with less like ad hoc code that people write out and don't write test cases for in some cases and so they don't even know if it works in the first place um and so if you start replacing systems of uh imperative code with deep learning models then you get better a better result okay and I think that he argues that software 2.0 is a per pervasively learned set of models and you get away from writing code and he's given talks where he talks about you know swapping over more and more and more parts of a code being learned and um driven that way I think that works and if you're pre predisposed to liking machine learning then I think that that's that's that's definitely a good thing I think this is also good for accessibility in many ways because certain people are not going to write C code or something and so having a data driven approach to do this kind of stuff I think can be very valuable on the other hand there are huge trade-offs and it's not clear to me that software 2.0 is um the answer and probably Andre wouldn't argue that it's the the answer for every problem either but um I look at machine learning as not a replacement for software 1.0 I look at it as a new programming Paradigm and so programming paradigms when you look across across domains is you know structured programming where you go from go-tos to if then else or functional programming from lisp and you start talking about higher order functions and values and things like this or you talk about objectoriented programming you talk about encapsulation subclassing inheritance you start talking about generic programming where you start talking about code reuse through um through uh specialization in different type instantiations um when you start talking about differentiable programming something that I am very excited about in the context of machine learning talking about taking functions and generating uh variance like the derivative of another function like that's a programming Paradigm that's very useful for solving certain classes of problems machine learning is amazing at solving certain classes of problems like you're not going to write a you know a cat detector or even a language translation system by writing C code that's not going to that's not a very productive way to do things anymore and so machine learning is absolutely the right way to do that in fact I would say that learn models are really the one of the best ways to work with the human world in general and so anytime you're talking about sensory input of different modalities anytime that you're talking about um generating things in a way that makes sense to a human I think that learn models are really really useful and that's because humans are very difficult to character okay and so this is a very powerful Paradigm for solving classes of problems but on the other hand uh imperative code is two you're not going to write a Bootloader for your computer in with a deep learning model deep learning models are very uh Hardware intensive they're very energy intensive because you have a lot of parameters and you can provably Implement any function with a learned model like this has been shown uh but that doesn't make it efficient and so if you're talking about carrying about a few orders of magnitudes worth of energy usage then it's useful to have other tools in the toolbox what also robustness too I mean yeah exactly all the problems of dealing with data and bias and data all the problems of uh you know software 2.0 and one of the great things that Andre is is uh arguing towards which I completely agree with him is that when you start uh implementing things with deep learning you need to learn from software 1.0 in terms of testing continuous integration how you deploy how do you validate all these things and building Building Systems around that so that you're not just saying like o it seems like it's good ship it right well what happens when I regress something what happens when I make a classification that's wrong and now I uh hurt somebody right I mean all these things you have to reason about yeah but at the same time the bootloader that works for our for us humans is uh looks awfully a lot like a new network right so it's it's it's messy and you can cut out different parts of the brain there's a lot of this neuroplasticity work that shows that it's going to adjust it's a I mean it's a really interesting question how much of the world programming could be replaced by software 2.0 like with oh could well I mean it's provably true that you could replace all of it right so then it's question anything that's a function you can so it's not a question about if I think it's a economic question it's a what kind of talent can you get what kind of trade-offs in terms of Maintenance right those kind of questions I think what kind of data can you collect I think one of the reasons that I'm most interested in uh machine learning is a programming Paradigm is that one of the things that we've seen across Computing in general is that being laser focused on one Paradigm often put you in a box it's not super great and so you look at object ear programming like it was all the rage in the early 80s and like everything has to be objects and people forgot about functional programming even though came first and and then people rediscovered that hey if you mix functional and object oriented and structure like you mix these things together you can provide very interesting tools that are good at solving different problems and so the question there is how do you get the best way to solve the problems it's not about whose tribe should win right it's not about you know that that that shouldn't be the question the question is how do you make it so that people can solve those problems the fastest and they have the right uh Tools in their box to build good libraries and they can solve these problems and when you look at that that's like you know you look at reinforcement learning as one really interesting subdomain of this reinforcement learning often you have to have the integration of a of a learn model combined with your Atari or whatever the other scenario it is that you're you're working in you have to combine that that thing with the robot control for the arm right and so now it's not just about that one uh Paradigm it's about integrating that with all the other systems that you have including often Legacy systems and things like this right and so to me I think that the interesting interesting thing to say is like how do you get the best out of this domain and how do you enable people to achieve things that they otherwise couldn't do without excluding all the good things we already know how to do right but okay this is just a crazy question but we talked a little about gpt3 but do you think it's possible that these language models that uh in essence in the language domain software 2.0 could replace some aspect of compilation for example or do program synthesis replace some aspect of programming yeah absolutely so I think the that learn models in general are extremely powerful and I think the people underestimate them um maybe you can suggest what I should do so of uh you know access to the gpt3 API would I be able to generate Swift code for example do you think that could do something interesting and so gpt3 is not probably not trained on the right Corpus so it probably has the ability to generate some Swift I bet it does um it's probably not going to generate a large enough body of Swift to be useful but but like take it a next step further like if if you had the goal of training something like gpt3 and you wanted to train it to generate source code right it could definitely do that now the question is um how do you express the intent of what you want filled in you can definitely like write write scaffolding of code and say fill in the hole and sort of put in some for Loops open put some classes or whatever and and the power of these mods is impressive but there's an unsolved question at least unsolved to me which is how do I express the intent of what to fill in right and kind of what you'd really want to have and I don't know that that these models are up to the task is you want to be able to say um here's a scaffolding and here are the assertions at the end and the assertions always pass and so you want a generative model on the one hand yes oh that's fascinating yeah right but you also want some loop back some reinforcement learning system or something where you're actually saying like I need to hill climb towards something that is more correct and I don't know that we have that so it would generate not only a bunch of the code but like the checks that do the testing it would generate the tests I think I think the humans would generate the tests right because the the test be fascinating if well the tests are the requirements yes but the okay so because you're have you have to express to the model what you want to you don't just want gibberish code look look at how compelling this code looks you want a story about four horned unicorns or something well okay so exactly but that's human requirements but then I thought it's a compelling idea that the gp4 model could generate uh checks like that are more um High Fidelity that check for correctness because uh the coded generates like say I ask it to generate a function that um gives me the Bacci sequence sure I don't like so so decompose the problem right so you have you have two things you have you need the ability to generate syntactically correct Swift Code that that's interesting right I think GPT series of model architectures can do that but then you need the ability to add the requirements so generate Fibonacci yeah the human needs to express that goal we don't have that language that I know of no I mean it can generate have you seen with gpt3 can generate you can say I mean there's uh interface stuff like it can generate HTML it can generate uh basic for Loops that give you like right but pick HTML how do I say I want google.com well no you could say or not not literally google.com how do I say I want a web page that's got a shopping cart and this and that that does that I mean so okay so just uh I don't know if you've seen these demonstrations but you type in I want a red button with the text that says hello and you type that in natural language and it generates the correct HTML done this demo it's it's kind of compelling so you have to uh uh prompt it with similar kinds of mappings of course it's probably handpicked like have to experiment they probably but the fact that they can do that once even out of like 20 yeah is uh is quite impressive again that's very basic uh like the HTML is kind of messy and and bad sure sure but yes the intent is the idea is the intent to specifi the natural language okay and so I've have not seen that that's really cool yeah yeah yeah but the question is uh the correctness of that like visually you can check oh the button is red but the for more uh for more complicated functions where the intent is harder to check this goes into like MP completeness kind of things like I want to know that this code is correct and gener it's a giant thing that uh does some kind of calculation it seems to be working it it's interesting to think like should the system also try to generate checks for itself for correctness yeah I don't know and this this is way beyond my experience the uh uh the thing that I think about is that there doesn't seem to be a lot of equational reasoning going on there's a lot of pattern matching and filling in and kind of propagating patterns that have been seen before into the future and into the generator result and so if you want to get correctness you kind of need theorem proving kind of things and like higher level logic and I don't know that um you could talk to Yan about that um and see and see what uh the the bright minds are thinking about right now but I don't think the GPT is in that that vein it's still really cool yeah and surpris who knows you know maybe reasoning is is uh is overrated yeah is over right I mean do we reason how do how do you tell right are we just pattern matching based on what we have and then reverse justifying it to ourselves exactly the reverse so like I think what the neural networks are missing and I think GPT for might have is to be able to uh tell stories to itself about what it did well that's what humans do right I mean you talk about uh like Network explainability right and we give noral Nets a hard time about this but humans don't know why we make decisions we have this thing called intuition and then we try to like say this feels like the right thing but why right and you know you wrestle with that when you're making hard decisions and is that science not really let me ask about a few highle questions I guess is um you've done a million things in your life and been very successful a bunch of young folks listen to this ask for advice from successful people like you uh if you were to give advice to uh somebody you know another a graduate student or some high school student about uh pursuing a career in Computing or just advice about life in general is there sure is there some words of wisdom you can give them so I think you come back to change and you know profound leaps happen because people are willing to believe that change is possible and that um the world does change and are willing to do the hard thing that it takes to make change happen and whether it be implementing a new programming language or implementing a new system or implenting a new research paper designing a new thing moving the world forward in science and philosophy whatever it really comes down to somebody who's willing to put in the work right and you have the the work is hard for a whole bunch of different reasons one of which is um you uh it it's work right and so you have to have the space in your life in which you can do that work which is why going to grad school can be a beautiful thing for certain people um but also there's a self-doubt that happens like you're two years into a project is it going anywhere right right well what do you do do you do you just give up because it's hard well no I mean some people like suffering um and so you plow through it the the secret to me is that you have to love what you're doing and and follow that passion because if when you get to the hard times that's when you know if you if you love what you're doing you're willing to kind of push through and um this is really uh hard because it's it's hard to know what you will love doing until you start doing a lot of things and so that's why I think that particularly early in your career it's good to experiment do a little bit of everything go go go take the the survey class on you know four different the first half of every class in your upper division you know lessons and um just get exposure to things because certain things will resonate with you and you'll find out wow I'm really good at this I'm really smart at this well it's just because it's it works with the way your brain and when something jumps out I mean that's one of the things that people often ask about is like well I think there's bunch of cool stuff out there like how do I pick the thing like uh yeah how do you how do you hook in your life how did you just hook yourself in and stuck with it well I got lucky right I mean I think that many people uh forget that a huge amount of it or most of it is luck right so um let's not forget that um so for me I fell in love with computers early on because I'm they they spoke to me I guess uh what language did they speak basic basic yeah um but the uh uh but then it was just kind of following a set of logical progressions but also um deciding that something that was hard was worth doing and and a lot of fun right and so I think that that is also something that's true for many other domains which is if you find something that you love doing that's also hard if you invest yourself in it and add value to the world then it will mean something generally right and again that can be a research paper that can be a software system that can be a new robot that can be that there's many things that that is that can be but a lot of it is like real value comes from doing things that are hard and that doesn't mean you have to suffer but um it's hard I mean you don't often hear that message we talked about it last time a little bit but I I it's one of my f not enough people talk about this this it's uh it's beautiful to hear a successful person well and self-doubt and impostor syndrome and the these are all things that uh successful people suffer with as well particularly when they put themselves in a point of being uncomfortable which um I like to do now and then just because it puts you in learning mode like if you want to if you want to grow as a person put yourself in a room with a bunch of people that know way more about whatever you're talking about than you do and ask dumb questions and guess what smart people love to teach often not always but often and if you listen if you're prepared to listen if you're prepared to grow if you're prepared to make connections you can do some really interesting things and I think a lot of progress is made by people who kind of hop between domains now and then because they bring uh they bring a perspective into a field that nobody else has if people have only been working in that field themselves we mentioned that the universe is kind of like a compiler of you know the entirety of it the whole evolution is kind of a kind of compilation maybe our us human beings are kind of compilers um let me ask the the old absur question that I didn't ask you last time which is uh what's the meaning of it all is there a meaning like if you asked a compiler why what would a compiler say what's the meaning of life what's the meaning of life uh you know I'm prepared for it not to mean anything here we are all biological things programmed to survive and and propagate our our DNA um and maybe the is just a just a computer and you you just go until entropy takes over the world and or takes over the universe and then you're done um I don't think that's a very productive way to live your life if so and so I prefer to bias towards the other way which is saying the world has the universe has a lot of value and I take uh I take happiness out of other people and a lot a lot of times part of that's having kids but also the relationships you build with other people and so uh the way I try to live my life is like what can I do that has value how can I move the world forward how can I take what I'm good at and like bring it bring it into the world and how can I I'm one of these people that likes to work really hard and be very focused on the things that I do and so if I'm going to do that how can it be in a domain that actually will matter right because a lot of things that we do we find ourselves in the cycle of like okay I'm doing a thing I'm very familiar with it I've done it for a long time I've never done anything else but I'm not really learning I I'm not really I'm keeping things going but there's a there's a younger generation that that can do the same thing maybe even better than me right maybe if I actually step out of this and jump into something I'm less comfortable with it's scary but on the other hand um it gives somebody else a new opportunity it also then put you back in learning mode and that can be really interesting and one of the things I've learned is that uh when you go through that that first you're deep into impostor syndrome but when you start working your way out you start to realize hey well there's actually a method to this and and now I'm able to add new things because I bring different perspective and this is one of the the the good things about bringing different kinds of people together diversity of thought is really important and um if you can pull together people that are coming at things from different directions you often get Innovation and I I love to see that that aha moment where you're like we've like really cracked this this is something never nobody's ever done before and then if you can do it in a context where it adds value other people can build on it it helps move the world then that's what that's what really excites me so the that kind of description of the magic of The Human Experience do you think we'll ever create that in like an AGI system you think we be able to create uh give uh give AI systems a sense of meaning where they operate in this kind of world exactly in the way you've described which is they interact with each other they interact with us humans sure sure well so I mean I why why are you being so speciest right all right so so agis versus bionet or you know versus bi um you know uh what are we but machines right we're just programmed to run our we have our objective function that we optimized for right and so we're doing our thing we think we have purpose but do we really yeah right I'm not prepared to say that th those new fangled agis have no soul just because we don't understand them right and I think that would be um when they when they exist uh that would be very premature to uh uh look at a new thing through your own lens without fully understanding it um you might be just saying that because AI systems in the future will be listening to this and then oh yeah yeah exactly you don't want to say anything please be nice to me you know when Skynet Skynet kills everybody please spare me Wise Wise uh look ahead thinking yeah but I mean I I think that people spend a lot of time worrying about this kind of stuff and I think that what we should be worrying about is how do we make the world better and the thing that I'm most scared about with agis is not that um that necessarily the Skynet will start shooting everybody with lasers and stuff like that to to use us for our calories the thing that I'm worried about is that um Humanity I think needs a challenge and if we get into a mode of not having a personal challenge not having a personal contribution whether that be like you know your kids and seeing what they grow into and helping helping guide them whether it be um your community that you're engaged in you're driving forward whether it be your work and the things that you're doing and the people you're working with and the products you're building and the contribution there if people don't have a objective I'm afraid what that means and um I think that this would lead to a rise of the worst part of people right instead of people striving together and trying to make uh the world better it could degrade into a very uh unpleasant world but but I don't know I mean we hopefully have a long ways to go before we discover that unfortunately we have pretty on the ground problems with the pandemic right now and so I think we should be focused on that as well yeah ultimately just as you said you're optimistic I think it helps for us to be optimistic that's uh fake it until you make it yeah well and why not what's what's the other side right so I mean uh uh I I'm not personally a very religious person but I've heard people say like oh yeah of course I believe in God of course I go to church because if God's real you know I want to be on the right side of that and if it's not real it doesn't matter doesn't matter and so you know that's that's a fair way to do it um yeah I mean the same thing with uh with nuclear deterrence all you know global warming all these things all these threats natural engineer pandemics all these threats we face I think it's uh uh it's paralyzing to be terrified of all the possible ways we could destroy ourselves I think it's much better uh or at least productive to be hopeful and to engineer defenses against these things to uh engineer a future where like you know see like a positive future and engineer that future yeah well and I think that's other another thing to think about as you know a human particularly if you're young and trying to figure out what it is that you want to be when you grow up like I am um I'm always looking for that uh the the question then is how do you want to spend your time and right now there seems to be a norm of being a consumption culture like I'm going to watch the news and and revel in how horrible everything is right now I'm going to go find out about the latest atrocity and find out all the details of like this the terrible thing that happened and be outraged by it um you can spend a lot of time watching TV and watching the new sitcom or whatever people watch these days I don't know um uh but that's a lot of hours right and those are hours that if you're tur into being productive learning growing experiencing uh you know when the pandemic's over going exploring right it leads to more growth and I think it leads to more optimism and happiness because you're you're you're building right you're building yourself you're building your capabilities you're building your viewpoints you're building your perspective and um I think that a lot of the cons the consuming of other people's me messages leads to kind of a negative Viewpoint which you need to be aware of what's happening because that's also important but there's a balance that um I think focusing on creation is is a very valuable thing to do yeah so what you're saying is people should focus on uh working on the sexiest feel of them all which is compiler design exactly well hey you can go work on machine learning and be crowded out by the the thousands of graduates popping out of school that all want to do the same thing or you could work in the place that people overpay you because there's not enough smart people working in it and uh here at the end of Mor's law According to some people uh actually the software is the hard part too yeah uh I mean optimization is is truly uh truly beautiful and also on the YouTube side or education side uh you know it's there's um it'd be nice to have some material that shows the beauty of compilers yeah yeah that's that's something so that's a call for uh for people to create that kind of content as well Chris uh you're one of my favorite people to talk to I it's such a huge honor that you would waste your time talking to me uh I've always appreciate it thank you so much today the the the the truth of is you spend a lot of time talking to me just on you know walks and other things like that so it's it's great to catch up thanks man thanks for listening to this conversation with Chris lner and thank you to our sponsors blinkist an app that summarizes key ideas from thousands of books neuro which is a maker of functional gum and mints that supercharge my mind masterclass which are online courses from World experts and finally cash app which is an app for sending money to friends please check out these sponsors in the description to get a discount and to support this podcast if you enjoy this thing subscribe on YouTube review it with f stars on Apple podcast follow on Spotify support on patreon or connect with me on Twitter at Lex Friedman and now let me leave you some words from Chris latner so much of language design is about trade-offs and you can't see those tradeoffs unless you have a community of people that really represent those different points thank you for listening and hope to see you next time