George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387
- What possible ideas do you have for how human species ends? - Sure. So, I think the most obvious
way to me is wire heading. We end up amusing ourselves to death. We end up all staring
at that infinite TikTok and forgetting to eat. Maybe it's even more benign than this. Maybe we all just stop reproducing. Now, to be fair, it's probably
hard to get all of humanity. - Yeah. The interesting thing about
humanity is the diversity in it. - Oh yeah.
- Organisms in general. There's a lot of weirdos out there. Two of them are sitting here. - Yeah, I mean, diversity in humanity is- - With due respect. (both chuckling) - I wish I was more weird. - The following is a
conversation with George Hotz, his third time on this podcast. He's the founder of comma.ai that seeks to solve autonomous driving and is the founder of a new
company called tiny corp that created tinygrad, a neural network framework
that is extremely simple with the goal of making
it run on any device by any human easily and efficiently. As you know, George
also did a large number of fun and amazing things,
from hacking the iPhone to recently joining Twitter for a bit as a "intern" in quotes, making the case for refactoring
the Twitter code base. In general, he's a fascinating
engineer and human being, and one of my favorite people to talk to. This is the Lex Fridman Podcast. To support it, please
check out our sponsors in the description. And now, dear friends, here's George Hotz. You mentioned something in a stream about the philosophical nature of time. So let's start with the wild question. Do you think time is an illusion? - You know, I sell phone
calls to comma for $1,000 and some guy called me
and it's $1,000 dollars, you can talk to me for half an hour. And he is like, "Yeah, okay. So time doesn't exist and I really wanted to
share this with you." I'm like, "Oh, what do you
mean time doesn't exist?" I think time is a useful model
whether it exists or not. Does quantum physics exist? Well, it doesn't matter. It's about whether it's a useful
model to describe reality. Is time maybe compressive? - Do you think there
is an objective reality or is everything just useful models? Like underneath it all? Is there an actual thing that
we're constructing models for? - I don't know. - I was hoping you would know.
- I don't think it matters. - I mean, this kinda
connects to the models of constructive reality
with machine learning. Right?
- Sure. - Is it just nice to have
useful approximations of the world such that we
can do something with it? - So there are things that are real. Column graph complexity is real. - Yeah.
- Yeah. - The compressive thing-
- Math. - [George] Math is real, yeah. - Should be a T-shirt. - And I think hard
things are actually hard. I don't think p equals np. - Ooh, strong words. - Well, I think that's the majority. I do think factoring is in p, but... - I don't think you're the
person that follows the majority in all walks of life, so it's good. - For that one, I do.
- Yeah. In theoretical computer science,
you're one of the sheep. (both chuckling) All right. But to you, time is a useful model. - Sure.
- Hmm. What were you talking about
on the stream with time, are you made of time? - If I remembered half the
things I said on stream. Someday, someone's gonna
make a model of all of it and it's gonna come back to haunt me. - Someday soon?
- Yeah, probably. - Would that be exciting to you or sad? That there's a George Hotz model? - I mean, the question is
when the George Hotz model is better than George Hotz. Like I am declining and
the model is growing. - What is the metric by which
you measure better or worse in that, if you are
competing with yourself? - Maybe you can just play a game where you have the George Hotz answer and the George Hotz model answer and ask which people prefer. - People close to you or strangers? - Either one. It will hurt more when
it's people close to me, but both will be overtaken
by the George Hotz model. - It'd be quite painful, right? Loved ones, family members
would rather have the model over for Thanksgiving than you. Or like, significant
others would rather sext (both chuckling) with the large language
model version of you. - Especially when it's fine
tuned to their preferences. - Yeah. Well, that's what we're doing
in a relationship, right? We're just fine tuning ourselves, but we're inefficient with it, 'cause we're selfish and greedy and so on. A language model is, can
fine tune more efficiently, more selflessly. - There's a "Star Trek Voyager" episode where Kathryn Janeway lost
in the delta quadrant, makes herself a lover on the holodeck. And the lover falls asleep on her arm and he snores a little bit and Janeway edits the
program to remove that. And then of course the
realization is, "Wait, this person's terrible." It is actually all
their nuances and quirks and slight annoyances that make this relationship worthwhile. But I don't think we're gonna realize that until it's too late. - Well, I think a large language model could incorporate the flaws and the quirks and all that kind of stuff. - Just the perfect amount
of quirks and flaws to make you charming
without crossing the line. - Yeah, yeah. And that's probably a good approximation of the percent of time. The language model should
be cranky or an asshole. or jealous or all this kind of stuff. - And of course, it can and it will. But all that difficulty at
that point is artificial. There's no more real difficulty. - Okay, what's the difference
between real and artificial? - Artificial difficulty is difficulty that's constructed or could
be turned off with a knob. Real difficulty is like,
you're in the woods and you gotta survive. - So, if something can not
be turned off with a knob, it's real? - Yeah, I think so. Or, I mean, you can't get out of this by smashing the knob with a hammer. I mean, maybe you kind of
can into the wild when, Alexander Supertramp, he
wants to explore something that's never been explored before. But it's the '90s
everything's been explored. So he's like, "Well, I'm
just not gonna bring a map." I mean, no, you're, you're not exploring. You should have brought a map, dude. You died. There was a bridge a mile
from where you were camping. - How does that connect to
the metaphor of the knob? - By not bringing the map,
you didn't become an explorer, you just smashed the thing. The difficulty is still artificial. - You failed before you started. What if we just don't
have access to the knob? - Well, that maybe is even scarier. We already exist in a world of nature and nature has been fine
tuned over billions of years. To have humans build something
and then throw the knob away in some grand romantic
gesture is horrifying. - Do you think of us humans as individuals that are like, born to die or are we just all part
of one living organism that is earth that is nature? - I don't think there's
a clear line there. I think it's all kinda
just fuzzy, I don't know. I mean, I don't think I'm conscious. I don't think I'm anything. I think I'm just a computer program. - So it's all computation? And the thing running in
your head is computation. - Everything running in the universe is computation, I think. I believe the extended
Church-Turing thesis. - Yeah, but there seems
to be an embodiment to your particular computation
like there's a consistency. - Well, yeah, but, I mean,
models have consistency too. Models that have been
RLHF will continually say, like, "Well how do I
murder ethnic minorities?" "Oh, well I can't let you do that, Hal." There's a consistency to that behavior. - It's all RLHF. Like we all RLHF each other. We find, (chuckles) we
provide human feedback and thereby fine tune these
little pockets of computation. But it's still unclear why that pocket of computation stays with you for years. Like you have this consistent
set of physics biology, whatever you call the neurons firing. The electrical signals,
the mechanical signals, all of that, that seems to stay there. And it contains information,
it stores information and that information
permeates through time and stays with you. There's like memory that's like sticky. - To be fair, a lot of the
models we're building today, even RLHF is nowhere near as complex as the human loss function. - Reinforcement learning
with human feedback. - When I talked about will GPT12 be AGI? My answer is no, of course not. I mean, cross-entropy loss
is never gonna get you there. You need probably RL in fancy environments in order to get something that
would be considered AGI-like. So to ask the question about why, I don't know, it's just
some quirk of evolution. I don't think there's
anything particularly special about where I ended up,
where humans ended up. - So, okay, we have
human level intelligence. Would you call that AGI? Whatever we have? AGI? - Look, actually, I don't
really even like the word AGI, but general intelligence is defined to be whatever humans have. - Okay. So why can GPT12 not get us to AGI? Can we just like linger on that? - If your loss function is
categorical cross-entropy. If your loss function is just
try to maximize compression. I have a SoundCloud, I rap
and I tried to get ChatGPT to help me write raps. And the raps that it wrote sounded like YouTube comment raps. You can go on any rap beat online and you can see what
people put in the comments. And it's the most
mid-quality rap you can find. - Is mid good or bad? - Mid is bad.
- Mid is bad. - [George] It's like mid, it's like... - Every time I talk to
you, I learn new words. (George chuckling) Mid.
- Mid, yeah. - Is it like basic? Is that what mid means? - Kind of. It's like middle of the curve. - So there's like that intelligence curve. And you have like the
dumb guy, the smart guy, and then the mid guy. Actually, being the mid guy is the worst. The smart guy is like, "I
put all my money in Bitcoin." The mid guy is like, "You
can't put money in Bitcoin. It's not real money." - And all of it is a genius meme. That's another interesting one, memes. The humor, the idea, the absurdity encapsulated in a single image and it just propagates virally
between all of our brains. I didn't get much sleep last night, so I sound like I'm high. I swear I'm not. Do you think we have
ideas or ideas have us? - I think that we're gonna
get super scary memes once the AI actually are superhuman. - Ooh, you think AI will generate memes? - Of course. - [Lex] You think it'll make humans laugh? - I think it's worse than that. So, "Infinite Jest," it's
introduced in the first 50 pages is about a tape that
once you watch it once, you only ever wanna watch that tape. In fact, you wanna watch the
tape so much that someone says, "Okay, here's a hack
saw, cut off your pinky and then I'll let you
watch the tape again," and you'll do it. So we're actually gonna
build that, I think. But it's not gonna be one static tape. I think the human brain is too complex to be stuck in one static tape like that. If you look at ant brains, maybe they can be stuck on a static tape. But we're going to build
that using generative models. We're going to build the TikTok that you actually can't look away from. - So TikTok is already pretty close there, but the generation is done by humans. The algorithm is just
doing their recommendation but if the algorithm is also
able to do the generation. - Well, it's a question
about how much intelligence is behind it. So the content is being
generated by let's say, one humanity worth of intelligence. And you can quantify a humanity. It's x of flops, yada flops. But you can quantify it. Once that generation is
being done by 100 humanities, you're done. - So it's actually scale
that's the problem. But also speed. Yeah. And what if it's manipulating the very limited human
dopamine engine for porn? Imagine just TikTok, but for porn. It's like a brave new world. - I don't even know what it'll look like. Like again, you can't
imagine the behaviors of something smarter than
you, but a super intelligent, an agent that just dominates
your intelligence so much will be able to completely manipulate you. - Is it possible that it
won't really manipulate, it'll just move past us? It'll just kinda exist
the way water exists or the air exists. - You see? And that's the whole AI safety thing. It's not the machine that's gonna do that. It's other humans using the machine that are gonna do that to you. - Yeah. 'Cause the machine is not
interested in hurting humans. - The machine is a machine. But the human gets the machine and there's a lot of humans
out there very interested in manipulating you. - Well, let me bring up Eliezer Yudkowsky who recently sat where you're sitting. He thinks that AI will
almost surely kill everyone. Do you agree with him or not? - Yes, but maybe for a different reason. - (chuckles) Okay. And then I'll try to get you to find hope or we could find a no to that answer. But why yes? - Okay. Why didn't nuclear weapons kill everyone? - That's a good question. - I think there's an answer. I think it's actually very hard to deploy nuclear weapons tactically. It's very hard to accomplish
tactical objectives. Great, I can nuke their country. I have an irradiated pile of rubble. I don't want that. - Why not? - Why don't I want an
irradiated pile of rubble? - [Lex] Yeah. - For all the reasons no one wants an irradiated pile of rubble. - Oh, 'cause you can't use
that land for resources. You can't populate the land. - Yeah, well, what you want,
a total victory in a war is not usually the
irradiation and eradication of the people there. It's the subjugation and
domination of the people. - Okay. So you can't use this
strategically, tactically in a war to help gain a military advantage. It's all complete destruction. All right. But there's egos involved.
It's still surprising. Still surprising that nobody
pressed the big red button. - It's somewhat surprising. But you see, it's the little red button that's gonna be pressed
with AI that's gonna... And that's why we die. It's not because the AI, if there's anything in the nature of AI, it's just the nature of humanity. - What's the algorithm
behind the little red button? What possible ideas do you have
for how human species ends? - Sure. So I think the most obvious
way to me is wire heading. We end up amusing ourselves to death. We end up all staring
at that infinite TikTok and forgetting to eat. Maybe it's even more benign than this. Maybe we all just stop reproducing. Now, to be fair, it's probably hard to get all of humanity. (Lex chuckling) - This always go, the interesting thing about
humanity is the diversity in it. Organisms in general. There's a lot of weirdos out there. Two of them are sitting here. - Yeah, I mean, diversity in humanity is- - With due respect. (both chuckling) - I wish I was more weird. No, like, look, I'm
drinking Smartwater, man. That's like a Coca-Cola product, right? - You want corporate George Hotz. - Yeah, I want corporate. No, the amount of diversity in humanity I think is decreasing. Just like all the other
biodiversity on the planet. - Oh boy, yeah. And social media's not helping. - Go eat McDonald's in China. No, it's the interconnectedness
that's doing it. - Oh, that's interesting. So everybody starts
relying on the connectivity of the internet? And over time, that reduces the diversity, the intellectual diversity. And then that gets you,
everybody into a funnel. There's still going to be a guy in Texas. - There is and yeah. - [Lex] A bunker. - To be fair, do I think AI kills us all? I think AI kills everything
we call society today. I do not think it actually
kills the human species. I think that's actually
incredibly hard to do. - Yeah, but society, like if
we start over, that's tricky. Most of us don't know
how to do most things. - Yeah but some of us do. And they'll be okay and they'll rebuild after the great AI. - What's rebuilding look like? How much do we lose? What has human civilization
done that's interesting. Combustion engine, electricity. So power and energy, that's interesting. Like how to harness energy. - Whoa, whoa, whoa, whoa. They're gonna be religiously against that. - Are they going to get back to like fire? - Sure, I mean they'll
be a little bit like, some kinda Amish looking
kind of thing I think. I think they're going to
have very strong taboos against technology. - Hmm. Like technology is almost
like a new religion, technology is the devil and nature is God. - Sure.
- So closer to nature. But can you really get away from AI if it destroyed 99% of the human species? Isn't it somehow have a
hold, like a stronghold? - Well, what's interesting
about everything we build, I think we're going to
build super intelligence before we build any sort
of robustness in the AI. We cannot build an AI that is capable of going out into nature
and surviving like a bird. A bird is an incredibly robust organism. We've built nothing like this. We haven't built a machine
that's capable of reproducing. - Yes, but I work with
Lego robots a lot though. I have a bunch of them. They're mobile. They can't reproduce. But all they need is, I guess you're saying they
can't repair themselves. If you have a large number, if you have like a
hundred million of them. - Let's just focus on them reproducing. Do they have microchips in them? Then do they include a fab? - [Lex] No. - Then how are they gonna reproduce? - Well, it doesn't have
to be all on board. They can go to a factory,
to a repair shop. - Yeah, but then you're really
moving away from robustness. All of life is capable of reproducing without needing to go to a repair shop. Life will continue to reproduce in the complete absence of civilization. Robots will not. So if the AI apocalypse happens, I mean the AI are gonna probably die out. 'Cause I think we're gonna get, again, super intelligence long
before we get robustness. - What about if you just
improve the fab to where you just have a 3D printer
that can always help you? - Well, that'd be very interesting. I'm interested in building that. - (laughs) Of course you are. You think, how difficult is that problem? To have a robot that
basically can build itself? - Very, very hard. - I think you've mentioned
this to me or somewhere where people think it's easy conceptually. - And then they remember that you're gonna have to have a fab. - Yeah, on board. - [George] Of course. - So 3D printer that prints a 3D printer. - Yeah.
- Yeah. On legs. Why's that hard? - Well, 'cause it's, I mean, a 3D printer is a very simple machine. You're gonna print chips, you're gonna have an atomic printer. How are you gonna dope the silicon? How are you gonna hatch the silicon? - You're gonna have to have a very interesting kind of fab if you wanna have a lot
of computation on board. But you can do structural
type of robots that are dumb. - Yeah, but structural type
of robots aren't gonna have the intelligence required to survive in any complex environment. - What about like ants type of systems where you have like trillions of them? - I don't think this works. I mean, again, like,
ants at their very core, are made up of cells that are capable of individually reproducing. - They're doing quite a lot of computation that we're taking for granted. - It's not even just the computation. It's that reproduction is so inherent. So, there's two stacks
of life in the world. There's the biological
stack and the silicon stack. The biological stack
starts with reproduction. Reproduction is at the absolute core. The first proto-RNA organisms
were capable of reproducing. The silicon stack, despite
as far as it's come, is nowhere near being able to reproduce. - Yeah. So the fab movement, digital fabrication, fabrication in the full
range of what that means, is still in the early stages. - [George] Yeah. - You're interested in this world? - Even if you did put
a fab on the machine. Let's say, okay, yeah, we can build fabs. We know how to do that as humanity. We can probably put all the precursors that build all the machines in the fabs also in the machine. So first off, this machine's
gonna be absolutely massive. I mean, we almost have a... Think of the size of the thing required to reproduce a machine today. Is our civilization
capable of reproduction? Can we reproduce our civilization on Mars? - If we were to construct a machine that is made up of humans like a company that can reproduce itself, I don't know, it feels like 115 people. - I think it's so much harder than that. - 120? (George laughing) I'm just looking for a number. - I believe that Twitter
can be run by 50 people. I think that this is gonna take most of... It's just most of society, right? We live in one globalized world. - No, but you're not
interested in running Twitter, you're interested in seeding. You want to seed a civilization 'cause humans can, like... - [George] Oh, okay.
- Have sex. - You're talking about, yeah, okay. So you're talking about
the humans reproducing and basically, what's the
smallest self-sustaining colony of humans? - [Lex] Yeah.
- Yeah. Okay, fine. But they're not gonna be
making five nanometer chips. - Over time, they will. I think we have to expand
our conception of time here. Going back to the original timescale. I mean across maybe 100 generations, we're back to making chips. If you seed the colony correctly. - Maybe, or maybe they'll watch
our colony die out over here and be like, "We're not making chips. Don't make chips." - No, but you have to seed
that colony correctly. - "Whatever you do, don't make chips. Chips are what led to their downfall." - Hmm. Well that is the thing that humans do. They come up, they construct a devil, a good thing and a bad thing, and they really stick by that. And then they murder each other over that. There's always one asshole in the room who murders everybody. (George laughing) And usually makes
tattoos and nice branding with flags and stuff.
- Do you need that asshole? That's the question, right? Humanity works really hard today
to get rid of that asshole, but I think they might be important. - Yeah, this whole
freedom of speech thing, the freedom of being an asshole
seems kind of important. - [George] That's right. - Man, this thing, this fab, this human fab that we constructed, this human civilization
is pretty interesting. And now, it's building
artificial copies of itself or artificial copies of
various aspects of itself that seem interesting like intelligence. And I wonder where that goes. - I like to think it's just
like another stack for life. Like we have like the bios stack life. We're a bios stack life and
then the silicon stack life. - But it seems like the ceiling or there might not be a ceiling or at least the ceiling is much higher for the silicon stack. - Oh no, I don't. We don't know what the ceiling
is for the bio stack either. The bio stack just seem to move slower. You have Moore's law, which is not dead despite
many proclamations. - [Lex] In the bio stack
or the silicon stack? - In the silicon stack. And you don't have anything
like this in the bio stack. So I have a meme that I posted, I tried to make a meme,
it didn't work too well. But I posted a picture of
Ronald Reagan and Joe Biden. And you look, this is
1980 and this is 2020. And these two humans
are basically the same. There's been no change in
humans in the last 40 years. And then I posted a computer from 1980 and a computer from 2020. Wow. - Yeah, with their early stages, right? Which is why you said,
when you said the fab, the size of the fab
required to make another fab is very large right now. But computers were very large 80 years ago and they got pretty tiny and people are starting to
wanna wear them on their face in order to escape reality. That's a thing. In order to live inside the computer. Put a screen right here, I don't have to see the
rest of the you, assholes. - I've been ready for a long time. - You like virtual reality? - [George] I love it. - Do you wanna live there?
- Yeah. - Yeah. Part of me does too. How far away are we, do you think? - Judging from what you can buy today? Far, very far. - I gotta tell you that
I had the experience of Meta's Codec Avatar, where it's a ultra high resolution scam. It looked real. - I mean, the headsets just are not quite at eye resolution yet. I haven't put on any
headset where I'm like, "Oh, this could be the real world." Whereas when I put good
headphones on, audio is there. And we can reproduce audio that I'm like, "I'm actually in a jungle right now." If I close my eyes, I can't tell I'm not. - Yeah, but then there's also smell and all that kind of stuff.
- [George] Sure. - I don't know. The power of imagination or the power of the mechanism in the human
mind that fills the gaps, that reaches and wants
to make the thing you see in the virtual world, real to you. I believe in that power. - Or humans wanna believe.
- Yeah. What if you're lonely? What if you're sad? What if you're really struggling in life and here's a world where you
don't have to struggle anymore? - Humans wanna believe so much that people think the large
language models are conscious. That's how much humans wanna believe. - Strong words. He's throwing left and right hooks. Why do you think large language
models are not conscious? - I don't think I'm conscious. - Oh, so what is consciousness
then, George Hotz? - What it seems to mean to people, it's just like a word that
atheists use for souls. - Sure, but that doesn't mean soul is not an interesting word. - If consciousness is a spectrum, I'm definitely way more conscious than the large language models are. I think the large language models are less conscious than a chicken. - When is the last time
you've seen a chicken? - In Miami, a couple months ago. - How? No, like a living chicken. - There's living chickens
walking around Miami, it's crazy. - Like on the street?
- Yeah. - Like, a chicken?
- A chicken. - All right. (both chuckling) I was trying to call you
out like a good journalist and I got shut down. But you don't think much about this kind of subjective feeling that it feels like something to exist. And then, as an observer,
you can have a sense that an entity is not only intelligent, but has a subjective
experience of its reality. Like a self-awareness that is capable of suffering, of hurting, of being excited by the environment in
a way that's not merely an artificial response,
but a deeply felt one. - Humans wanna believe so
much that if I took a rock and a sharpie and drew
a sad face on the rock, they'd think the rock is sad. - Yeah. And you're saying when
we look in the mirror, we apply the same smiley face with rock? - Pretty much, yeah. - Isn't that weird though
that you're not conscious? Is that?
- No. - But you do believe in consciousness? - Not really. - It's unclear. So, to you it's like a little, a symptom of the bigger thing
that's not that important. - Yeah, I mean, it's interesting that the human systems seem to
claim that they're conscious. And I guess it says
something in a straight up, even if you don't
believe in consciousness, what do people mean when
they say consciousness? And there's definitely meanings to it. - What's your favorite thing to eat? - Pizza. - Cheese pizza. What are the toppings? - [George] I like cheese pizza. I like pepperoni pizza.
- Don't say pineapple pizza. - [George] No, I don't like pineapple. - Okay, pepperoni pizza. - Unless they put any ham
on it. Oh, that's real bad. - What's the best pizza? What
are we talking about here? You like cheap, crappy pizza? - A Chicago deep dish cheese pizza. Oh, that's my favorite. - There you go. You bite into a deep dish,
Chicago deep dish pizza. So you were starving. You haven't eaten for 24 hours. You just bite in and you're
hanging out with somebody that matters a lot to you. And
you're there with the pizza. - [George] Oh, sounds real nice, man. - Yeah, all right. It feels like something. I'm George motherfucking Hotz eating a fucking Chicago deep dish pizza. There's just the full
peak living experience of being human. The top of the human condition. It feels like something
to experience that. Why does it feel like something? That's consciousness, isn't it? - If that's the word you want
to use to describe it, sure. I'm not gonna deny that
that feeling exists. I'm not gonna deny that I
experienced that feeling. I guess what I kind of take issue to is that there's some, how does
it feel to be a web server? Do 404s hurt? - Not yet. - How would you know what
suffering looked like? Sure, you can recognize a suffering dog because we're the same stack as the dog. All the bios stack stuff
kind of, especially mammals, it's really easy. - Game recognizes game.
- Yeah. Versus the silicon stack stuff. It's like, you have no idea. The little thing has learned to mimic. But then I realized that
that's all we are too. Oh, look, the little thing
has learned to mimic. - Yeah. I guess, yeah, 404 could be suffering, but it's so far from our
kind of living organism, our kind of stack. But it feels like AI can start maybe mimicking the biological
stack better, better, better 'cause it's trained. - [George] We trained it, yeah. - And so maybe that's the
definition of consciousness, is the bio stack consciousness. - The definition of consciousness is how close something looks to human. Sure, I'll give you that one. - No, how close something
is to the human experience. - Sure. It's a very anthropos-centric
definition, but... - Well, that's all we got.
- Sure. No, and I don't mean to, like, I think there's a lot of value in it. Look, I just started my second company, my third company will be AI Girlfriends. No, I mean it. - I wanna find out what your
fourth company is after that. - Oh, wow. - Because I think once you
have AI girlfriends, it's... Oh boy does it get interesting. Well, maybe let's go there. I mean, the relationships with AI, that's creating human-like organisms. And part of being human
is being conscious, is having the capacity to suffer, having the capacity to
experience this life richly in such a way that AI system
can empathize with you and you can empathize with it, or you can project your
anthropomorphic sense of what the other entity is experiencing. And an AI model would need to, yeah, to create that experience
inside your mind. And it doesn't seem that difficult. - Yeah, but, okay, so here's where it actually
gets totally different. When you interact with another human, you can make some assumptions. When you interact with
these models, you can't. You can make some assumptions that that other human
experiences suffering and pleasure in a pretty
similar way that you do. The golden rule applies. With an AI model, this isn't really true. These large language models
are good at fooling people because they were trained on
a whole bunch of human data and told to mimic it. - Yup. But if the AI system says,
"Hi, my name is Samantha," it has a backstory. Went to college there, here and there. Maybe you'll integrate
this in the AI system. - I made some chatbots. I give them backstories. It was lots of fun. I was so happy when LLaMA came out. - Yeah. Well, we'll talk about LLaMA,
we'll talk about all that, but the rock with a smiley face. It seems pretty natural for you to anthropomorphize that thing
and then start dating it. And before you know it,
you're married and have kids. - With a rock? (laughs) - With a rock. There's pictures on Instagram with you and a rock and smiley face. - To be fair, something that
people generally look for when they're looking for so much to date is intelligence in some form. And the rock doesn't
really have intelligence. Only a pretty desperate
person would date a rock. - I think we're all desperate deep down. - Oh, not rock level desperate. - All right. (chuckling) Not rock level desperate,
but AI level desperate. I don't know, I think all of
us have a deep loneliness. It just feels like the
language models are there. - Oh, I agree. And you know what? I won't
even say this so cynically, I will actually say
this in a way that like, I want AI friends. I do. I would love to. Again, the language models
now are still a little.. People are impressed with these GPT things or the copilot, the coding one. And I'm like, okay, this is
like junior engineer level, and these people are Fiverr
level artists and copywriters. Okay, great. We got Fiverr and junior engineers. Okay, cool. And this is just a start
and it will get better. I can't wait to have AI friends who are more intelligent than I am. - So Fiverr is just a
temporary, it's not the ceiling. - [George] No, definitely not. - Is it count as cheating when you're talking to an AI model? Emotional cheating? - That's up to you and your
human partner to define. - [Lex] Oh, you have to, all right. - You have to have that
conversation, I guess. - [Lex] All right. I mean, integrate that with
porn and all this stuff. - Well, no, I mean, it's similar to porn. I think people in relationships have different views on that. - Yeah, but most people don't have serious open conversations about
all the different aspects of what's cool and what's not. And it feels like AI is a really
weird conversation to have. - I mean, the porn one is
a good branching off point. Like these things, one of my scenarios
that I put in my chatbot is a nice girl named Lexi, she's 20, she just moved out to LA,
she wanted to be an actress but she started doing OnlyFans instead, and you're on a date with her, enjoy. - (laughs) Oh, man. Yeah, and so is that, if
you're actually dating somebody in real life, is that cheating? I feel like it gets a little weird. It gets real weird. It's like, what are you
allowed to say to an AI bot? Imagine having that conversation
with a significant other. - I mean, these are all
things for people to define in their relationships. What it means to be human is
just gonna start to get weird. - Especially online. How do you know? There'll be moments when you'll have what you think is a real
human you interacted with on Twitter for years and
you realize it's not. - I spread, I love this meme. Heaven banning. You hear about shadow banning? - [Lex] Yeah. - Shadow banning. You
post, no one can see it. Heaven banning, you
post, no one can see it but a whole lot of AIs are
spot up to interact with you. - Well, maybe that's what the
way human civilization ends is all of us heaven banned. - There's a great, it's called "My Little Pony Friendship is Optimal." It's a sci-fi story
that explores this idea. - Friendship is optimal. - Friendship is optimal. - Yeah. I'd like to have some, at least the on the intellectual realm, some AI friends that argue with me. But the romantic realm is weird. Definitely weird. But not out of the realm of the kind of weirdness
that human civilization is capable of, I think. - I want it. Look, I want it. If no one else wants it, I want it. - [Lex] Yeah, I think a lot
of people probably want it. There's a deep loneliness. - And I'll fill their loneliness and it just will only advertise
to you some of the time. - Yeah, Maybe the conceptions
of monogamy changed too. I grew up in a time like I value monogamy, but maybe that's a silly notion when you have arbitrary
number of AI systems. - Mm, yeah. This interesting path from
rationality to polyamory. Yeah, that doesn't make sense for me. - For you. But you're just a biological
organism who was born before the internet really took off. - The crazy thing is culture
is whatever we define it as. Do you think, is our problem
and moral philosophy, right? There's no, like, okay, what is might be that computers
are capable of mimicking girlfriends perfectly. They pass the girlfriend Turing test. But that doesn't say anything about ought. That doesn't say anything
about how we ought to respond to them as a civilization. That doesn't say we ought
to get rid of monogamy. That's a completely separate question, really a religious one. - Girlfriend Turing test. I wonder what that looks like. - [George] Girlfriend Turing test. - Are you writing that? Will you be the Allen
Turing of the 21st century that writes the girlfriend Turing test? - Well, no. I mean, of
course, my AI girlfriends, their goal is to pass the
girlfriend Turing test. - No, but there should be like a paper that defines the test. I mean, the question is if
it's deeply personalized or there's a common thing
that really gets everybody. - Yeah. I mean, look, we're a company. We don't have to get everybody. We just have to get a
large enough clientele to stabilize us. - I like's how you're
already thinking company. Before we go to company number three and company number four, let's
go to company number two. tiny corp. Possibly one of the
greatest names of all time for a company. (George chuckling) You've launched a new
company called tiny corp that leads the development of tinygrad. What's the origin story
of tiny corp and tinygrad? - I started tinygrad as like a toy project just to teach myself, okay,
what is a convolution? What are all these options
you can pass to them? What is the derivative of a convolution? Very similar to Karpathy wrote micrograd. I'm very similar. And then I started realizing, I started thinking about AI chips. I started thinking about chips that run AI and I was like, well,
okay, this is going to be a really big problem. If NVIDIA becomes a monopoly here, how long before NVIDIA is nationalized? - Hmm. So one of the reasons to start tiny corp is to challenge NVIDIA. - It's not so much to challenge NVIDIA. Actually, I like NVIDIA it's to make sure power
stays decentralized. - And here's computational power. And to you, NVIDIA's locking down the computational power of the world. - If NVIDIA becomes just like 10X better than everything else, you're
giving a big advantage to somebody who can secure
NVIDIA as a resource. In fact, if Jensen watches this podcast, he may wanna consider this. He may wanna consider making sure his company's not nationalized. - Do you think that's an actual threat? - Oh yes. - No, but there's so much... There's AMD. - Mm-hm. So we have NVIDIA and AMD, great. - But you don't think there's
a push towards selling? Like Google selling TPUs
or something like this? You don't think there's a push for that? - Have you seen it? Google loves to rent you TPUs. - You can't buy it at Best Buy? - No. So I started work on a chip. I was like, okay, what's it
gonna take to make a chip? And my first notions were
all completely wrong. About like how you could improve on GPUs. And I'll take this, this is from Jim Keller on your podcast. And this is one of my
absolute favorite descriptions of computation. So there's three kinds
of computation paradigms that are common in the world today. There's CPUs, and CPUs can do everything. CPUs can do add and multiply. They can do load and store, and they can do compare and branch. And when I say they can do these things, they can do them all fast. So compare and branch are unique to CPUs. And what I mean by they can do them fast is they can do things
like branch prediction and speculative execution. And they spend tons of transistors and these super deep reorder buffers in order to make these things fast. Then you have a simpler
computation model GPUs. GPUs can't really do compare and branch. I mean they can, but
it's horrendously slow. But GPUs can do arbitrary load and store. GPUs can do things like X de-reference Y. So they can fetch from
arbitrary pieces of memory. They can fetch from memory that is defined by the contents of the data. The third model of computation is DSPs. And DSPs are just add and multiply. They can do loads and stores, but only static load in stores. Only loads in stores that are
known before the program runs. And you look at neural networks today and 95% of neural networks
are all the DSP paradigm. They are just statically
scheduled adds and multiplies. So tinygrad really took this idea and I'm still working on it to extend this as far as possible. Every stage of the stack
has Turing completeness. Python has Turing completeness. And then we take Python, we go into C++ which is Turing complete. And then maybe C++ calls
into some CUDA kernels, which are Turing complete. The
CUDA kernels go through LVM, which is Turing complete, into PTX, which is Turing complete, into SAS, which is Turing complete on a Turing complete processor. I wanna get Turing completeness
outta the stack entirely. Because once you get rid
of turn completeness, you can reason about things. Rice theorem and the
Halting problem do not apply to ADMO machines. - Okay. What's the power and the value of getting Turing completeness out of... Are we talking about the
hardware or the software? - Every layer of the stack.
- Every layer. - Every layer of the stack. Removing Turing completeness allows you to reason about things. So the reason you need to do
branch prediction in a CPU and the reason it's prediction, and the branch predictors are, I think, they're like 99% on CPUs. Why do they get 1% of them wrong? Well, they get 1% wrong
because you can't know. That's the halting problem. It's equivalent to the
halting problem to say whether a branch is gonna be taken or not. I can show that, but the ADMO machine, the neural network runs the
identical compute every time. The only thing that changes is the data. So, when you realize
this, you think about, okay, how can we build a computer and how can we build a stack that takes maximal advantage of this idea? So what makes tinygrad different from other neural network libraries is it does not have a primitive operator even for matrix multiplication. And this is every single one. They even have primitive
operation interest, things like convolutions. - So no mat mul. - No mat mul. Well here's what a mat mul is. So I'll use my hands to talk here. So if you think about a cube, and I put my two matrices
that I'm multiplying on two faces of the cube, you can think about the matrix multiply as the n cubed. I'm gonna multiply for
each one in the cubed. And then I'm gonna do a sum, which is a reduce up to here
to the third phase of the cube. And that's your multiplied matrix. So what a matrix multiply is, is a bunch of shape operations. A bunch of per three shapes and
expands on the two matrices, A multiply, n cubed, a reduce, n cubed, which gives you an ns squared matrix. - What is the minimum number of operations that can accomplish that if you don't have mat mul as a primitive? - So tinygrad has about 20. And you can compare tinygrads offset or IR to things like XLA or PrimTorch. So XLA and PrimTorch are
ideas where like, okay, Torch has like 2000 different kernels. PyTorch 2.0 introduced
PrimTorch which has only 250. tinygrad has order of magnitude 25. It's 10x less than XLA or PrimTorch. And you can think about it as
kind of like RISC versus CISC. These other things are CISC-like systems. tinygrad is RISC. - And RISC one. - RISC architecture is
gonna change everything. 1995 hackers. - Wait, really? That's an actual thing? - Angelina Jolie delivers
the line RISC architecture is gonna change everything in 1995. And here we are with ARM in
the phones and ARM everywhere. - Wow. I love it when movies actually
have real things in them. Okay, interesting. So you're thinking of this
as the risk architecture of ML stack. 25. Can you go through the four op types? - Sure. Okay. So you have Unary ops
which take in a tensor and return a tensor of the same size and do some unary op to it. X, log, reciprocal sine. They take in one and they're point wise. - ReLU.
- Yeah, ReLU. Almost all activation
functions are unary ops. Some combinations of unary ops
together is still a unary op. Then you have binary ops. Binary ops are like pointwise
edition, multiplication, division, compare. It takes in two tensors of equal size and outputs one tensor. Then you have reduce ops. Reduce ops will like take
a three-dimensional tensor and turn it into a two-dimensional tensor or a three-dimensional tensor turn into zero dimensional tensor. Think like a sum or a max
are really common ones there. And then the fourth type is movement ops. And movement ops are
different from the other types because they don't actually
require computation. They require different
ways to look at memory. So that includes reshapes,
permutes, expands, flips. Those are the main ones probably. - So with that, you have
enough to make a mat mul. - And convolutions. And every convolution you can imagine. Dilated convolutions,
strated convolutions, transposed convolutions. - You're right on GitHub about
laziness showing a mat mul. Matrix multiplication. See how despite the style, it is fused into one kernel
with the power of laziness. Can you elaborate on
this power of laziness? - Sure, so if you type in
PyTorch A times B plus C, what this is going to do is
it's going to first multiply, add and B, A and B and store
that result into memory. And then it is going to add
C by reading that result from reading C from memory,
and writing that out to memory. There is way more loads
in stores to memory than you need there. If you don't actually do A
times B as soon as you see it, if you wait until the user
actually realizes that tensor until the laziness actually resolves, you can fuse that plus C. It's the same way Haskell works. - So what's the process of
porting a model into tinygrad? - So tinygrad's front end
looks very similar to PyTorch. I probably could make a perfect
or pretty close to perfect interop layer if I really wanted to. I think that there's some
things that are nicer about tiny grad syntax than PyTorch. But the front end looks very Torch-like. You can also load in Onyx models. We have more onyx tests
passing than kernel. We'll pass Onyx around time soon. - Well what about like
the developer experience with tiny grad? What it feels like versus PyTorch? - By the way, I really like PyTorch. I think that it's actually a
very good piece of software. I think that they've made
a few different tradeoffs. And these different
tradeoffs are where tinygrad takes a different path. One of the biggest differences
is it's really easy to see the kernels that are actually
being sent to the GPU. If you run PyTorch on the GPU, you like do some operation
and you don't know what kernels ran, you don't
know how many kernels ran, you don't know how many flops were used. You don't know how much
memory accesses were used. tinygrad type debug equals two. And it will show you in this
beautiful style every kernel that's run how many
flops and how many bytes. - So can you just linger on
what problem tinygrad solves? - tinygrad solves the problem of porting new ML accelerators quickly. One of the reasons tons
of these companies now, I think Sequoia marked Graphcore to zero. Cerebras, Tenstorrent Groq, all of these ML accelerator
companies, they built chips. The chips were good, the
software was terrible. And part of the reason is
because I think the same problem is happening with Dojo. It's really, really hard
to write a PyTorch port because you have to write 250 kernels and you have to tune
them all for performance. - What does Jim Keller
think about tinygrad? You guys hang out quite a bit. So he's involved with Tenstorrent. What's his praise and what's his criticism of what you're doing with your life? - Look... My prediction for Tenstorrent
is that they're gonna pivot to making RISC-V chips. CPUs. - CPUs. Why? - Because AI accelerators
are a software problem, not really a hardware problem. - Oh, interesting. So you think the diversity
of AI accelerators in the hardware space is
not going to be a thing that exists long term? - I think what's gonna
happen is if I can, okay. If you're trying to
make an AI accelerator, you better have the capability of writing a torch level performance
stack on NVIDIA GPUs. If you can't write a torch
stack on NVIDIA GPUs, and I mean all the way, I
mean down to the driver, there's no way you're
gonna be able to write it on your chip. 'Cause your
chip's worse than an NVIDIA GPU. The first version of
the chip you tape out, it's definitely worse. - Oh, you're saying writing
that stack is really tough. - Yes. And not only that, actually the chip that you
tape out almost always, 'cause you're trying to
get advantage over NVIDIA, you're specializing the hardware more. It's always harder to write software for more specialized hardware. Like a GPU is pretty generic. And if you can't write an NVIDIA stack, there's no way you can
write a stack for your chip. So my approach with tinygrad is first, write a performant NVIDIA stack. We're targeting AMD. - So you did say FU to NVIDIA
a little bit with Love. - With Love.
- With love. - It's like the Yankees,
you know I'm a Mets fan. - Oh, you're a Mets fan. A RISC fan and a Mets fan. What's the hope that AMD has? You did a build with
AMD recently that I saw. How does the 7900 XTX compare
to the RTX 4090 or 4080? - Well let's start with
the fact that the 7900 XTX kernel drivers don't work. And if you run demo apps and
loops, it panics the kernel. - [Lex] Okay, so this is a software issue. - Lisa Su responded to my email. Oh, I reached out. I was like, this is, you know, really? I understand if you are
seven by seven transposed winne grad com this slower than NVIDIA's, but literally when I
run demo apps in a loop, the kernel panics? - So just adding that loop. - Yeah, I just literally
took their demo apps and wrote like, while
true semicolon do the app semicolon done in a bunch of screens. This is like the most
primitive fuzz testing. - Why do you think that is? They're just not seeing a
market in machine learning? - They're changing. They're trying to change. They're trying to change. And I had a pretty positive
interaction with them this week. Last week, I went on YouTube. I was just like, "That's it. I give up on AMD. Their driver doesn't even,
like, I'm not gonna... I'll go with Intel GPUs, Intel GPUs have better drivers." - So you're spearheading
the diversification of GPUs? - Yeah, and I'd like to extend that diversification to everything. I'd like to diversify the... My central thesis about
the world is there's things that centralize power and
they're bad and there's things that decentralize power and they're good. Everything I can do to
help decentralize power, I'd like to do. - So you're really worried about the centralization of NVIDIA. That's interesting. And you don't have a fundamental hope for the proliferation of
ASICs except in the cloud. - I'd like to help them with software. No, the only ASIC that
is remotely successful is Google's TPU. And the only reason that's
successful is because Google wrote a machine learning framework. I think that you have to
write a competitive machine learning framework in order
to be able to build an ASIC. - Hmm. You think Meta with PyTorch
builds a competitor? - I hope so. They have one. They have an internal one. - Internal, I mean, public-facing with a nice cloud interface and so on. - I don't want a cloud. - You don't like cloud? - I don't like cloud. - What do you think is the
fundamental limitation of cloud? - Fundamental limitation of cloud is who owns the off switch? - So that's power to the people. - [George] Yeah. And you don't like the
man to have all the power? - Exactly.
- All right. And right now, the only way
to do that is with the GPU is if you want performance and stability. Interesting. It's a costly investment
emotionally to go with AMDs. Well, let me ask sort of
on a tangent to ask you, you've built quite a few PCs. What's your advice on how
to build a good custom PC let's say for the different applications that you use for gaming,
for machine learning? - Well you shouldn't build one. You should buy a box from the tiny corp. - I heard rumors, whispers
about this box in the tiny corp. What's this thing look like? What is it called? - It's called the tinybox.
- tinybox. - It's $15,000. - And it's almost a
paid a flop of compute. It's over 100 gigabytes of GPU ram. It's over five terabytes per
second of GPU memory bandwidth. I'm gonna put like four NVMe in RAID, you're gonna get like 20, 30 gigabytes per second
of drive read bandwidth. I'm gonna build the best
deep learning box that I can that plugs into one wall outlet. - Okay. Can you go
through those specs again a little bit from memory? - Yeah, so it's almost a
paid a flop of compute. - So, AMD, Intel? - Today I'm leaning toward AMD, but we're pretty agnostic
to the type of compute. The main limiting spec is
a 120 volt, 15 amp circuit. - Okay. - Well, I mean it. Because in order to like, there's a plug over there. You have to be able to plug it in. We're also gonna sell
the tiny rack, which, what's the most power you
can get into your house without arousing suspicion? And one of the answers is
an electric car charger. - Wait, where does the rack go? - Your garage.
- Interesting. The car charger. - A wall outlet is about 1,500 watts. A car charger is about 10,000 watts. - What is the most amount of
power you can get your hands on without arousing suspicion? - That's right.
- George Hotz. So the tinybox and you
said, NVMEs and RAID. I forget what you said about
memory, all that kind of stuff. So what about what GPUs? - Again, probably 7900 XTXs
but maybe 3090s, maybe A770s. Those are tells. - You're flexible or still exploring? - I'm still exploring. I wanna deliver a really
good experience to people and yeah, what GPUs I
end up going with again, I'm leaning toward AMD. We'll see. In my email, what I said to AMD is like, just dumping the code on
GitHub is not open source. Open source is a culture. Open source means that your issues are not all one-year-old stale issues. Open source means developing in public. And if you guys can commit to that, I see a real future for AMD
as a competitor to NVIDIA. - Well I'd love to get a tinybox to MIT. So whenever it's ready. Let's do it. - We're taking pre-orders. I took this from Elon. I'm like $100 fully refundable pre-orders. - Is it gonna be like the cyber truck's gonna take a few years or? - No, I'll try to do it fast on that. It's a lot simpler. It's a
lot simpler than a truck. - Well there's complexities
not to just the putting the thing together, but like shipping and all this kind of stuff. - The thing that I want to
deliver to people out of the box is being able to run 65-billion
parameter LLaMA in FP 16 in real time. In like a good like 10 tokens per second or five tokens per second or something. - Just it works. Time is running or something like LLaMA. - Experian or I think
Falcon is the new one. Experian's a chat with
the largest language model that you can have in your house. - Yeah, from a wall plug. - From a wall plug, yeah. Actually, for inference,
it's not like even more power would help you get more. - Even more power wouldn't get you more. - Well no, there's just
the biggest model released is 65 billion parameter
LLaMA as far as I know. - So it sounds like tinybox
will naturally pivot towards company number three. 'Cause you could just get
the girlfriend and I mean, or boyfriend. - That one's harder actually. - The boyfriend is harder? - The boyfriend's harder, yeah. - I think that's a very biased statement. I think a lot of people would disagree. Why is it harder to replace a boyfriend than a girlfriend with the artificial LLM? - Because women are
attracted to status and power and men are attracted to youth and beauty. (Lex laughing) No, I mean this is what I mean. - But both are could be mimicable easy through the language model. - No. No, machines do not have
any status or real power. - I don't know. I think you both... Well first of all, you're
using language mostly to communicate youth and
beauty and power and status. - But status fundamentally
is a zero sum game. Whereas youth and beauty or not. - No, I think status is a
narrative you can construct. I don't think status is real. - I don't know. I just think that that's why it's harder. Maybe it is my biases. - I think status is way easier to fake. - I also think that men
are probably more desperate and more likely to buy my products, so maybe they're a better target market. - Desperation is interesting. Easier to fool. I could see that. - I mean look, I know you can look at
porn viewership numbers. A lot more men watch porn than women. You can ask why that is. - Wow, there's a lot of
questions and answers you can get there. Anyway, with the tinybox. How many GPUs in tinybox? - Six. - (laughs) Oh man. - And I'll tell you why it's six. So AMD EPYC processors
have 128 lanes of PCIe. I wanna leave enough lanes for
some drives and I wanna leave enough lanes for some networking. - How do you do cooling
for something like this? - Ah, that's one of the big challenges. Not only do I want the cooling to be good, I want it to be quiet. I want the tinybox to be
able to sit comfortably in your room. - This is really going
towards the girlfriend thing. 'Cause you want to run the LLM. - I'll give a more, I mean, I can talk about how it
relates to company number one. - comma.ai.
- Yeah. - Well, but yes, oh, quiet because you maybe
potential wanna run in a car? - No, no, quiet because you wanna put
this thing in your house and you want it to coexist with you. If it's screaming, it's 60 dB, you don't want that in your house. You'll kick it out. - 60 dB, yeah. - I want like 40, 45. - So how do you make the cooling quiet? That's an interesting problem in itself. - A key trick is to actually make it big. Ironically, it's called the tinybox. But if I can make it big, a lot of that noise is generated because of high pressure air. If you look at like a 1U server, a 1U server has these
super high pressure fans, they're like super deep
they're like generates, versus if you have something that's big, well, I can use a big thing, you know they call them big ass fans. Those ones that are
like huge on the ceiling and they're completely silent. - So tinybox will be big. - I do not want it to be
large according to UPS. I want it to be shippable
as a normal package, but that's my constraint there. - Interesting. Well, the fan stuff, can't it be assembled down location or no? - No. - No, has to be. Well, you're- - No, look, I wanna give you a great out-of-the-box experience. I want you to lift this thing out. I want it to be like Mac. tinybox. - The Apple experience.
- Yeah. - I love it. Okay, and so tinybox would run tinygrad. What do you envision this
whole thing to look like? We're talking about like Linux with a full software
engineering environment and it's just not PyTorch but tinygrad. - Yeah, we did a poll. If
people want Ubuntu or Arch, we're gonna stick with Ubuntu. - Ooh, interesting. What's your favorite flavor of Linux? Ubuntu. I like Ubuntu MATE. However you pronounce that, MATE. So you've gotten LLaMA into tinygrad, you've gotten stable
diffusion in tinygrad. What was that like? Can you comment on what are these models? What's interesting about porting them? So yeah, what are the challenges? What's naturally, what's easy? All that kind of stuff. - There's a really simple
way to get these models into tinygrad and you can
just export them as Onyx. And then tinygrad can run Onyx. So the ports that I did
of LLaMA Stable Diffusion and now Whisper are more academic to teach me about the models. But they are cleaner than
the PyTorch versions. You can read the code. I think
the code is easier to read. It's less lines. This is a few things about the
way tinygrad writes things. Here's a complaint I have about PyTorch. nn.ReLU is a class. when you create an nn module, you'll put your nn.ReLUs as in a net. And this makes no sense. ReLU is completely stateless. Why should that be a class? - But that's more like a
software engineering thing, or do you think it has
a cost on performance? - Oh no, it doesn't have
a cost on performance. But yeah, no, I think, that's what I mean about
tinygrad's front end being cleaner. - Ah, I see. What do you think about Mojo? I don't know if you've
been paying attention the programming language that
does some interesting ideas that kinda intersect tinygrad. - I think that there's a
spectrum and like on one side, you have Mojo and on the
other side, you have ggml. ggml is this like, we're
gonna run LLaMA fast on Mac. "We're gonna expand out to a little bit, but we're gonna basically
like depth first." Mojo is like, "We're
gonna go breath first. We're gonna go so wide
that we're gonna make all of Python fast." And tinygrad's in the middle. "We are going to make
neural networks fast." - Yeah, but they try to
really get it to be fast, compile down to the specifics hardware and make that compilation step as flexible and resilient as possible. - [George] Yeah, but
they've turned completeness. - And that limits you. That's what you're saying
it's somewhere in the middle. So you're actually going to be
targeting some accelerators. Like some number, not one. - My goal is step one, build an equally performance
stack to PyTorch on NVIDIA and AMD but with way less lines. And then step two is, okay,
how do we make an accelerator? But you need step one. You have to first build the framework before you can build the accelerator. - Can you explain MLPerf? What's your approach in general to benchmarking tinygrad performance? - So, I'm much more of a
like build it the right way and worry about performance later. There's a bunch of things where I haven't even really
dove into performance. The only place where
tinygrad is competitive performance wise right
now is on Qualcomm GPUs. So tinygrad's actually used
an openpilot to run the model. So the driving model is tinygrad. - When did that happen? That transition? - Well, eight months ago now. And it's 2x faster than
Qualcomm's library. - What's the hardware that
openpilot runs the comma.ai? - It's a Snapdragon 845. So this is using the GPU. So the GPU's an Adreno GPU. There's different things. There's really good Microsoft
paper that talks about like mobile GPUs and why they're different from desktop GPUs. One of the big things is in a desktop GPU, you can use buffers. On a mobile GPU, image
texture's are a lot faster. - On a mobile GPU image,
textures and image, okay. And so you want to be
able to leverage that. - I wanna be able to leverage it in a way that it's completely generic. XiaoMi has a pretty
good open source library for mobile GPUs. It's called MACE where they can generate, where they have these kernels,
but they're all hand coded. So that's great, if you're
doing three by three comps, that's great if you're
doing dense mat muls. But the minute you go off
the beaten path a tiny bit, well, your performance is nothing. - Since you mentioned openpilot, I'd love to get it an
update in the company. Number one, comma.ai world. How are things going
there in the development of semi-autonomous driving? - You know, almost no one
talks about FSD anymore, and even less people talk about openpilot. We've solved the problem. Like we solved it years ago. - What's the problem exactly? Well, what does solving it mean? - Solving means how do you build a model that outputs a human policy for driving? How do you build a model that given reasonable set of sensors, outputs a human policy for driving? So you have companies
like Waymo and Cruise, which are hand coding these things that are like quasi-human policies. Then you have Tesla and maybe
even to more of an extent, comma, asking, "Okay, how do we just learn the
human policy and data?" The big thing that we're doing now, and we just put it out on Twitter. At the beginning of comma, we published a paper called
"Learning a Driving Simulator". And the way this thing worked was it was an auto encoder
and then an RNN in the middle. You take an autoencoder,
you compress the picture, you use an RNN, predict the next state. And these things were, it was a laugh it loop bad simulator. This is 2015 error machine
learning technology. Today we have VQ-VAE and transformers. We're building drive GPT basically. - Drive GPT, okay. So and it's trained on what? Is it trained in a self-supervised way? - Yeah. It's trained on all
the driving data to predict the next frame. - So really trying to
learn a human policy. What would a human do? - Well, actually, our simulator's
conditioned on the pose. So it's actually a
simulator you can put in like a state action pair
and get out the next state. And then once you have a simulator, you can do RL in the simulator and RL will get us that human policy. - So it transfers.
- Yeah. RL with a reward function. Not asking is this close
to the human policy, but asking what a human disengage
if you did this behavior. - Okay, let me think about
the distinction there. Would a human disengage? Would a human disengage? That correlates I guess
with a human policy, but it could be different. So it doesn't just say,
"What would a human do?" It says, "What would a
good human driver do?" And such that the experience is comfortable but also not annoying in that the thing is very cautious. So it's finding a nice balance. That's that's interesting, that's a nice- - It's asking exactly the right question. What will make our customers happy? A system that you never wanna disengage. - 'Cause usually disengagement
is just almost always a sign of, "I'm not happy with
what the system is doing." - Usually. There's some that are just, "I felt like driving," and
those are always fine too, but they're just gonna look
like noise in the data. - But even that felt like driving. - Maybe, yeah. - That's a signal. Like why do you feel like driving? You need to recalibrate your
relationship with the car. So that's really interesting. How close are we to solving self-driving? - It's hard to say. We haven't completely closed the loop yet. So we don't have anything
built that truly looks like that architecture yet. We have prototypes and there's bugs. So we are a couple bug fixes away. Might take a year, might take 10. - What's the nature of the bugs? Are these these major philosophical bugs? Logical bugs? What what kind of bugs
are we talking about? - Oh, they're just like stupid bugs. And also, we might just need more scale. We just massively expanded
our compute cluster at comma. We now have about two
people worth of compute. 40p of flops. - Well people are different. - Yeah, 20p of flops. That's a person. I mean it's just a unit. Horse are different too, but
we still call it a horsepower. - Yeah, but there's something
different about mobility than there is about perception and action in a very complicated world, but yes. - Yeah, of course not all
flops are created equal. If you have randomly initialized
weights, it's not gonna... - Not all flops are created. - Flops are doing way more
useful things than others. - Yup, yup. Tell me about it. So more data. Scale means more scale and compute or scale in scale of data? - Both.
- Diversity of data? - Diversity is very important in data. I mean, I think we have
like 5,000 daily actives. - How would you evaluate how FSD is doing? - Pretty well.
- In self-driving? - Pretty well. - [Lex] How's that race gone
between comma.ai and FSD? - Tesla's always one to
two years ahead of us. They've always been one
to two years ahead of us. And they probably always will be because they're not doing anything wrong. - What have you seen that's
since the last time we talked that are interesting
architectural decisions, training decisions like
the way they deploy stuff, the architectures they're
using in terms of the software, how the teams are run,
all that kind of stuff. Data collection, anything interesting? - I mean, I know they're moving toward more of an end-to-end approach. - So creeping towards
end-to-end as much as possible across the whole thing. The training, the data
collection, everything. - They also have a very fancy simulator. They're probably saying
all the same things we are. They're probably saying
we just need to optimize, what is the reward? Well you get negative
reward for disengagement. Everyone kinda knows this. It's just a question
who can actually build and deploy the system. - Yeah. I mean, this requires good software engineering, I think. And the right kind of hardware. - Yeah, and the hardware to run it. - You still don't believe
in cloud in that regard? - I have a compute cluster
in my oh, 800 amps. - tinygrad. - It's 40 kilowatts at
idle, our data center. Does seem crazy. Have 40 kilowatts is burning just when the computers are idle. Oh sorry, sorry, compute cluster. - [Lex] Compute cluster, I got it. - It's not a data center.
No data centers are clouds. We don't have clouds. Data centers have air conditioners. We have fans. That makes
it a compute cluster. - I'm guessing this is a
kind of a legal distinction that should- - Sure, yeah, we have a compute cluster. - You said that you don't
think LLMs have consciousness, or at least not more than a chicken. Do you think they can reason? Is there something interesting
to you about the word reason about some of the
capabilities that we think is kind of human to be able to integrate complicated information and
through a chain of thought, arrive at a conclusion that feels novel? A novel integration of disparate facts? - Yeah, I think that
they can reason better than a lot of people. - Isn't that amazing to you though? Isn't that like an incredible thing that a transformer can achieve? - I mean, I think that calculators can add better than a lot of people. - But language feels like
reasoning through the process of language, which looks
a lot like thought. - Making brilliancy in chess, which feels a lot like thought. Whatever new thing that AI can do everybody thinks is brilliant. And then 20 years go by and they're like, "Well, yeah, but chess,
that's like mechanical. Like adding, that's like mechanical." - So you think language
is not that special. It's like chess. - Its' like chess.
- I don't know. And because it's very human, we take it... Listen, there is something different between chess and language. Chess is a game that a
subset of population plays. Language is something we use nonstop for all of our human interaction. And human interaction is
fundamental to society. So it's like, holy shit. This language thing is
not so difficult to create in the machine. - The problem is if you go
back to 1960 and you tell them that you have a machine
that can play amazing chess, of course, someone in 1960 will tell you that machine is intelligent. Someone in 2010 won't. What's changed? Today, we think that these machines that have language are intelligent. But I think in 20 years
we're gonna be like, "Yeah, but can it reproduce?" - So reproduction. Yeah, we may redefine what
it means to be, what is it? A high performance
living organism on earth? - Humans are always gonna
define a niche for themselves. Well, we're better than the
machines because we can... When like they tried creative for a bit, but no one believes that one anymore. - But niche, is that delusional or is there some accuracy to that? Because maybe with chess,
you start to realize that we have ill conceived notions of what makes humans special? Like the apex organism on earth. - Yeah, and I think maybe
we're gonna go through that same thing with language and that same thing with creativity. - But language carries these
notions of truth and so on. And so we might be like, "Wait, maybe truth is not carried by language. Maybe there's like a deeper thing." - The niche is getting smaller. - Oh boy. - But no, no, no. You don't understand. Humans are created by God and machines are created by humans. Therefore, that'll be
the last niche we have. - So what do you think about just the rapid development of LLMs? If you could just like stick on that, it's still incredibly impressive. Like with ChatGPT. Just even ChatGPT, what are your thoughts
about reinforcement learning with human feedback on
these large language models? - I'd like to go back to when
calculators first came out or computers and like, I wasn't around. Look, I'm 33 years old. And to see how that affected society. - Maybe you're right. So I wanna put on the
big picture hat here. - Oh my god. A refrigerator, wow. - Refrigerator, electricity,
all that kind of stuff. But no, with the internet, large language models seeming human basically passing a Turing test. It seems it might have really at scale rapid transformative effects on society. But you're saying other
technologies have as well. So maybe calculator's not
the best example of that 'cause that just seems like maybe, well no, maybe, calculator- - But the poor milk man, the day he learned about refrigerators, he's like, "I'm done. You're telling me you can just
keep the milk in your house? You don't need me to deliver it every day. I'm done." - Well, yeah, you have to actually look at the practical impacts
of certain technologies that they've had. Yeah, probably electricity is a big one. And also how rapidly spread. Man, the internet is a big one. - [George] I do think it's
different this time though. - Yeah, it just feels like- - The niche is getting smaller. - The niche as humans.
- Yes. - That makes humans special.
- Yes. - It feels like it's getting
smaller rapidly though, doesn't it? Or is that just the feeling
we dramatize everything? - I think we dramatize everything. I think that you ask the milk
man when he saw refrigerators. "And they're gonna have one
of these in every home?" - Yeah, yeah, yeah, yeah. But boy, is it impressive. So much more impressive than seeing a chess world champion AI system. - I disagree actually. I disagree. I think things like MuZero and AlphaGo are so much more impressive because these things are playing beyond the highest human level. The language models are writing
middle school level essays and people are like,
"Wow, it's a great essay. It's a great five paragraph essay about the causes of the Civil War." - Okay, forget the Civil War
just generating code, codex. (George grunting) So you're you're saying
it's mediocre code? - Terrible. - I don't think it's terrible. I think it's just mediocre code. Often close to correct. Like for mediocre. - That's the scariest kinda code. I spent 5% of time typing
and 95% of time debugging. The last thing I want is
close to correct code. I want a machine that can
help me with the debugging, not with the typing. - Well, it's like L2, level two a driving similar kind of thing. Yeah, you still should
be a good programmer in order to modify, I
wouldn't even say debugging, it's just modifying the code, reading it. - Actually don't think it's
like level two driving. I think driving is not tool
complete and programming is. Meaning you don't use like the
best possible tools to drive. Cars have basically the same interface for the last 50 years. Computers have a radically
different interface. - Okay, can you describe the
concept of tool complete? - Yeah. So think about the difference
between a car from 1980 and a car from today. No difference really. It's got a bunch of pedals,
it's got a steering wheel. Great. Maybe now, it has a few ADAS features, but it's pretty much the same car. You have no problem getting
into a 1980 car and driving it. You take a programmer today
who spent their whole life doing JavaScript and you put
them in an Apple 2E prompt and you tell them about
the line numbers in basic, but how do I insert something
between line 17 and 18? Oh well. - So in tool, you're putting
in the programming languages. So it's just the entirety
stack of the tooling. - [George] Exactly. - So it's not just like
IDEs or something like this. It's everything. - Yes, it's IDE's the
language is the run times. It's everything. And programming is tool complete. So like almost if Codex or
copilot are helping you, that actually probably means
that your framework or library is bad and there's too
much boilerplate in it. - Yeah, but don't you
think so much programming has boilerplate? - tinygrad is now 2,700
lines and it can run LLaMA and stable diffusion. And all of this stuff is in 2,700 lines. Boilerplate and abstraction, indirections and all these
things are just bad code. - Well, let's talk about
good code and bad code. I would say, I don't
know, for generic scripts that I write just offhand, like 80% of it is written by GPT. Just a quick offhand stuff. So not like libraries, not like performing code, not
stuff for robotics and so on. Just quick stuff. Because your basic, so much of programming is
doing some some boilerplate but to do so efficiently and quickly, 'cause you can't really automate it fully with generic method. Like a generic kind of
IDE type of recommendation or something like this. You do need to have some of the complexity of language models. - Yeah, I guess if I was really writing, like maybe today, if I wrote
a lot of data parsing stuff. I mean I don't play CTFs anymore, but if I still play CTFs a lot
of like is just like you have to write like a parser
for this data format. Or like admin of code. I wonder when the models
are gonna start to help with that kind of code. And they may. They may, and the models
also may help you with speed. The models is very fast. But where the models won't, my programming speed is not at all limited by my typing speed. And in very few cases, it is. Yes, if I'm writing some
script to just like parse some weird data format, sure, my programming speed is
limited by my typing speed. - What about looking stuff up? 'Cause that's essentially a
more efficient lookup, right? - You know, when I was at Twitter, I tried to use ChatGPT
to ask some questions. What's the API for this? And it would just hallucinate. It would just give me
completely made up API functions that sounded real. - Well do you think that's
just a temporary kinda stage? - [George] No. - You don't think it'll get
better and better and better in this kind of stuff because
it only hallucinates stuff in the edge cases. - Yes. - If you writing generic code,
it's actually pretty good. - Yes, if you are
writing an absolute basic like react app with a button,
it's not gonna hallucinate. No, there's kind of ways to
fix the hallucination problem. I think Facebook has an interesting paper, it's called Atlas and it's
actually weird the way that we do language models right now
where all of the information is in the weights and the human brains don't really like this. It's like a hippocampus
and a memory system. So why don't LLMs have a memory system? And there's people working on them. I think future LLMs are
gonna be like smaller but are going to run looping on themselves and are going to have retrieval systems. And the thing about
using a retrieval system is you can side sources, explicitly. - Mm. Which is really helpful to
integrate the human into the loop of the thing 'cause you
can go check the sources and you can investigate. So whenever the thing is hallucinating, you can have the human supervision. So that's pushing it
towards level two kind of. - Driving that's gonna kill Google. - Wait, which part? - When someone makes an LLM
that's capable of citing its sources, it will kill Google. - LLM that's citing its sources because that's basically a search engine? - Yeah. That's what people
want in the search engine. - But also Google might be
the people that build it. - Maybe.
- And put ads on it. - I'd count them out. - Why is that? Why do you think? Who wins this race? Who are the competitors? We got tiny corp. I don't know if that's... I mean you're a legitimate
competitor in that. - I'm not trying to compete on that. - You're not?
- No. - You can accidentally
stumble into that competition. Maybe you don't think you
might build a search engines or replace Google search. - When I started comma, I said over and over again, "I'm going to win self-driving cars." I still believe that. I have never said I'm going to
win search with the tiny corp and I'm never going to
say that 'cause I won't. - The night is still young. We don't know how hard is it to win search in this new route. I mean one of the things
that ChatGPT shows that there could be a
few interesting tricks that create a really compelling product. - Some startups gonna figure it out. I think if you ask me like Google's still the number one webpage, I think by the end of the decade, Google won't be the number
one web page anymore. - So you don't think Google, because of the how big the corporation is? - Look, I would put a lot
more money on Mark Zuckerberg. - [George] Why is that? - Because Mark Zuckerberg's alive. Like this is old Paul Graham essay. Startups are either alive or dead. Google's dead. - Facebook is alive. Facebook is alive. - Meta.
- Meta. - You see what I mean? That's like Mark Zuckerberg. This is Mark Zuckerberg
reading that Paul Graham asking and being like, "I'm gonna
show everyone how alive we are. I'm gonna change the name." - So you don't think there's
this gutsy pivoting engine, like Google doesn't have that, the kind of engine in a
startup has like constantly- - You know what?
- Being alive, I guess. - When I listen to your
Sam Altman podcast, he talked about the button. Everyone who talks about
AI talks about the button, the button to turn it off. Do we have a button to turn off Google? Is anybody in the world capable
of shutting Google down? - What does that mean exactly? The company or the search engine? - We shut the search engine down. Could we shut the company down? Either. - Can you elaborate on the
value of that question? - Does Sundar Pichai have the authority to turn off google.com tomorrow? - Who has the authority? That's a good question. - Does anyone?
- Does anyone? Yeah, I'm sure. - Are you sure? No, they have the technical power, but do they have the authority? Let's say Sundar Pichai
made this his sole mission. Came into Google tomorrow and said, "I'm gonna shut google.com down." I don't think he keep
his position too long. - And what is the mechanism by which he wouldn't keep his position? - Well, the boards and shares
and corporate undermining and oh my god, our revenue is zero now. - Okay, so I mean, what's
the case you're making here? So the capitalist machine prevents you from having the button. - Yeah. I mean, this is true for the AI too. There's no turning the AIs off. There's no button. You can't press it. Now, does Mark Zuckerberg have
that button for facebook.com? - Yes, probably more.
- I think he does. I think he does. And this is exactly what I mean and why I bet on him so much
more than I bet on Google. - [Lex] I guess you could
say Elon has similar stuff. - Oh, Elon has the button. Can Elon fire the missiles? Can he fire the missiles? - I think some questions
are better left unasked. - Right? I mean, a rocket and an
ICBM or you're a rocket that can land anyway. Isn't that an ICBM? Well, don't ask too many questions. - My God. But the positive side of the button is that you can innovate
aggressively is what you're saying? Which is what's required with turning LLM into a search engine.
- I would bet on a startup. I bet on-
- Because it's so easy, right? - Id' bet on something that
looks like mid journey, but for search. - Just it is able to set
source a loop on itself? I mean, it's just feels
like one model can take off. And nice wrapper and some of it scale. I mean, it's hard to like create a product that just works really nicely, stably. - The other thing that's gonna be cool is there is some aspect of
a winner take all effect. Once someone starts
deploying a product that gets a lot of usage, and you
see this with OpenAI, they are going to get the data
set to train future versions of the model. They are going to be able to. I was asked at Google image
search when I worked there almost 15 years ago now. How does Google know
which image is an Apple? And I said the metadata. And they're like, "Yeah, that
works about half the time." How does Google know? You'll see they're all
apples on the front page when you search Apple. And I don't know, I didn't
come up with the answer. The guy's like, "Well, it's what people click on
when they search Apple." I'm like, oh yeah. - Yeah, yeah. That data is really, really powerful. It's the human supervision. What do you think are the chances? What do you think in general
that LLaMA was open sourced? I just did a conversation with Mark Zuckerberg and
he's all in on open source. - Who would've thought
that Mark Zuckerberg would be the good guy? No, I mean it. - Who would've thought
anything in this world? It's hard to know. But open source to you
ultimately is a good thing here. - Undoubtedly. You know, what's ironic about
all these AI safety people is they're going to build
the exact thing they fear. These, "We need to have one model that we control and align." This is the only way you end up paperclip. There's no way you end up paper clipped if everybody has an AI. - So open sourcing is the way to fight the paperclip maximizer. - Absolutely. It's the only way. You think you're gonna control it? You're not gonna control it. - So the criticism you have
for the AI safety folks is that there is belief
and a desire for control. And that belief and desire
for centralized control of dangerous AI systems is not good. - Sam Altman won't tell you that GPT-4 has 220 billion parameters
and is a 16-way mixture model with eight sets of weights. - Who did you have to murder
to get that information? - I mean, look. Everyone at open AI knows
what I just said was true. Now, ask the question. Really, it upsets me when
OpenAI came out with GPT-2 and raised a whole fake AI
safety thing about that. I mean, now the model is laughable. They used AI safety to
hype up their company and it's disgusting. - Or the flip side of that is they used a relatively weak model in retrospect to explore how do we
do AI safety correctly? How do we release things? How do we go through the process? I don't know- - Sure, sure, sure. - [Lex] I don't know
how much hype there is. - That is a charitable interpretation. - I don't know how much hype there is in AI safety, honestly.
- Oh, there's so much hype. At least on Twitter. I don't know, maybe
Twitter's not real life. - Twitter's not real life. Come on, in terms of hype. I think open AI has been
finding an interesting balance between transparency and
putting value on AI safety. You think just go all out open source, so do what LLaMA do. So do open source, this is a tough question,
which is open source, both the base, the foundation
model and the fine tune one. So the model that can be
ultra racist and dangerous and tell you how to
build a nuclear weapon. - Oh my God, have you met humans? Like half of these AI- - I haven't met most humans. This allows you to meet every human. - Yeah, I know. But half of these AI alignment problems are just human alignment problems. And that's what's also so scary
about the language they use. It's like, it's not the machines you wanna align it to me. - But here's the thing. It makes it very accessible to ask questions where the answers
have dangerous consequences if you were to act on them. - I mean, yeah. Welcome to the world. - Well, no, for me,
there's a lot of friction if I wanna find out how to, I
don't know, blow up something. - No, there's not a lot of
friction, that's so easy. - No, like what do I search? Do I use Bing or which
search anything do I use? - No, there's like lots of- - No, it feels like I have to keep first. - Off, first off, first off. Anyone who's stupid enough
to search for how to blow up a building in my neighborhood
is not smart enough to build a bomb. - Are you sure about that?
- Yes. - I feel like a language
model makes it more accessible for that person who's
not smart enough to do. - They're not gonna
build a bomb, trust me. The people who are
incapable of figuring out how to ask that question
a bit more academically and get a real answer from it are not capable of procuring the materials which are somewhat
controlled to build a bomb. - No, I think LLM makes it more accessible to people with money without
the technical know-how. Do you really need to
know how to build a bomb to build a bomb? You can hire people, you can find- - Oh, you can hire people to build a... I was asking this question on my stream, can Jeff Bezos hire a hitman? Probably not. - But a language model
can probably help you out. - Yeah, and you'll still go to jail. It's not like the language model is God, You literally just
hired someone on Fiverr. - Okay, okay. GPT-4 in terms of finding
hitman is like asking Fiverr or how to find a hitman, I understand, but don't you think- - Asking Wikihow. - Wikihow. But don't you think GPT-5 will be better? Because don't you think that information is out there on the internet?
- I mean, yeah, and I think that if someone
is actually serious enough to hire a hitman or build a bomb, they'd also be serious enough
to find the information. - I don't think so. I think it makes it more accessible. If you have enough money to buy a hit man, I think it just decreases the
friction of how hard is it to find that kind of hit man. I honestly think there's
a jump in ease and scale of how much harm you can do. And I don't mean harm with language, I mean harm with actual violence. - What you're basically
saying is like, okay, what's gonna happen is these
people who are not intelligent are going to use machines to
augment their intelligence. And now, intelligent people and machines. Intelligence is scary. Intelligent agents are scary. When I'm in the woods, the
scariest animal to meet is human. Look, there's like nice California humans. I see you're wearing
street clothes and Nikes, all right, fine. But you
look like you've been a human who's been in the woods for a while, I'm more scared of you than a bear. - That's what they say about the Amazon. When you go to the Amazon,
it's the human tribes. - Oh yeah. So intelligence is scary. So to ask this question in a generic way, you're like, what if we took everybody who maybe has ill intention
but is not so intelligent and gave them intelligence? So we should have intelligence
control, of course. We should only give
intelligence to good people. And that is the absolutely
horrifying idea. - So to you, the best defense
is to give more intelligence to the good guys and give
intelligence to everybody. - Give intelligence to everybody. You know what? It's not even like guns. Like people say this about guns. What's the best defense
against a bad guy with a gun? Good guy with a gun. I'm like, I kinda subscribe to that, but I really subscribe to
that with intelligence. - Yeah, in a fundamental
way. I agree with you, but there's just feels
like so much uncertainty and so much can happen
rapidly that you can lose a lot of control and you
can do a lot of damage. - Oh no, we can lose control? Yes, thank God. I hope they lose control. I'd want them to lose control
more than anything else. - I think when you lose control,
you can do a lot of damage, but you can do more damage
when you centralized and hold onto to control is the point. - Centralized and held control is tyranny. I don't like anarchy either, but I'll always take anarchy over tyranny. Anarchy, you have a chance. - This human civilization
we've got going on is quite interesting. I mean, I agree with you. So to you, open source
is the way forward here. So you admire what Facebook is doing here or what Meta is doing with the release. - Yeah. I lost $80,000 last
year investing in Meta. And when they released LLaMA,
I'm like, "Eh, whatever man. That was worth it." - It was worth it. Do you think Google and OpenAI
with Microsoft will match what Meta is doing or no? - So if I were a researcher, why would you wanna work at OpenAI? You're on the bad team. I mean it. You're on the bad team who can't even say that GPT-4 has 220 billion parameters. - So closed source to use the bad team. - Not only closed source. I'm not saying you need to
make your model weights open. I'm not saying that. I totally understand. We're keeping our model weights closed because that's our product. That's fine. I'm saying like, "Because
of AI safety reasons, we can't tell you the number
of billions of parameters in the model." That's just the bad guys. - Just because you're mocking AI safety doesn't mean it's not real. - [George] Oh, of course. - Is it possible that
these things can really do a lot of damage that we don't know about? - Oh my God, yes. Intelligence is so dangerous. Be it human intelligence
or machine intelligence. Intelligence is dangerous. - But machine intelligence
is so much easier to deploy at scale, like rapidly. Okay, if you have
human-like bots on Twitter, and you have like a thousand of them, create a whole narrative. You can manipulate millions of people. - But you mean like, the
intelligence agencies in America are doing right now? - Yeah, but they're
not doing it that well. It feels like you can do a lot- - They're doing it pretty well. I think they're doing a pretty good job. - I suspect they're not nearly as good as a bunch of GPT-fueled bots could be. - Well, I mean, of course, they're looking
into the latest technologies for control of people of course. - But I think there's a
George Hotz type character that can do a better job that
the entirety idea of them. - No. - You don't think so?
- No way.m And I'll tell you why the
George Hotz character can't, and I thought about
this a lot with hacking. I can find exploits in web browsers. I probably still can. I mean, I was better I when I was 24, but the thing that I lack
is the ability to slowly and steadily deploy them over five years. And this is what intelligence
agencies are very good at. Intelligence agencies don't have the most sophisticated technology. They just have- - Endurance.
- Endurance. - And yeah, the financial backing and the infrastructure for the endurance. - So the more we can decentralize power... You could make an argument by the way that nobody should have these things. And I would defend that argument. I would like you're saying that look, LLMs and AI and machine intelligence can cause a lot of harm
so nobody should have it. And I will respect someone philosophically with that position. Just like I will respect
someone philosophically with the position that
nobody should have guns. But I will not respect philosophically with only the trusted authorities should have access to this. Who are the trusted authorities? You know what? I'm not worried about
alignment between AI company and their machines. I'm worried about alignment
between me and AI company. - What do you think Eliezer
Yudkowsky would say to you? 'Cause he's really against open source. - I know and... I thought about this,
I thought about this. And I think this comes down
to a repeated misunderstanding of political power by the rationalists. - [Lex] Interesting. - I think that Eliezer Yudkowsky
is scared of these things. And I am scared of these things too. Everyone should be scared of these things. These things are scary. But now you ask about
the two possible futures. One where a small, trusted, centralized group of people
has them and the other, where everyone has them. And I am much less scared
of the second future than the first. - Well, there's a small
trusted group of people that have control over
our nuclear weapons. - There's a difference. Again, a nuclear weapon
cannot be deployed tactically. And a nuclear weapon is not a defense against a nuclear weapon. Except maybe in some philosophical
mind game kind of way. - But AI is different,
and different how exactly? - Okay. Let's say the intelligence
agency deploys a million bots on Twitter or a thousand
bots on Twitter to try to convince me of a point. Imagine I had a powerful AI
running on my computer saying, "Okay, nice PSYOP, nice PSYOP, nice PSYOP. Here's a PSYOP, I
filtered it out for you." - Yeah, I mean, so you have
fundamentally hope for that. For the defensive PSYOP? - I don't even mean these
things in truly horrible ways. I mean these things in
straight up like ad blocker. Straight up ad blocker. I don't want ads. But they're always finding, imagine I had an AI that could just block all the ads for me. - So you believe in
the power of the people to always create an ad blocker? I mean, I kinda share that belief. One of the deepest optimisms
I have is just like, there's a lot of good guys. So you shouldn't handpick them. Just throw out powerful
technology out there and the good guys will outnumber
and outpower the bad guys. - I'm not even gonna say
there's a lot of good guys. I'm saying that good outnumbers bad. Good outnumbers bad. - [Lex] In skill and performance? - Yeah, definitely in
skill and performance. Probably just a number too. Probably just in general. I mean, if you believe
philosophically in democracy, you obviously believe that. That good outnumbers bad. If you give it to a
small number of people, there's a chance you
gave it to good people, but there's also a chance
you gave it to bad people. If you give it to everybody, well, if good outnumbers bad, then you definitely gave it
to more good people than bad. - That's really interesting. So that's on the safety grounds but then also of course,
there's other motivations like you don't wanna give
away your secret sauce. - Well I mean, look, I respect capitalism. I think that it would be polite for you to make model architectures open source and fundamental breakthroughs open source. I don't think you have to
make weights open source. - You know what's interesting
is that there's so many possible trajectories in human
history where you could have the next Google be open source. So for example, I don't know if that
connection is accurate, but Wikipedia made a lot
of interesting decisions not to put ads. Wikipedia is basically open source. You could think of it that way. And like that's one of the
main websites on the internet. And it didn't have to be that way. It could have been like Google
could have created Wikipedia, put ads on it. You could probably run
amazing ads now on Wikipedia. You wouldn't have to
keep asking for money. But it's interesting, right? So LLaMA, open source LLaMA, derivatives of open source
might win the internet. - I sure hope so. I hope to see another era. The kids today don't know how
good the internet used to be. And I don't think this is
just, all right, come on. Everyone's nostalgic for their past. But I actually think the
internet before small groups of weaponized corporate
and government interests took it over, was a beautiful place. - You know, those small
number of companies have created some sexy products. But you're saying overall,
in the long arc of history, the centralization of power they have suffocated the human spirit at scale? - Here's a question to ask about those beautiful, sexy products. Imagine 2000 Google to 2010 Google. A lot changed. We got Maps, we got Gmail. - We lost a lot of products too, I think. - Yeah, I mean somewhere probably. We've got Chrome. And now let's go from
2010, we got Android. Now let's go from 2010 to 2020. Well, what does Google have? Well, search engine, Maps,
Mail, Android and Chrome. Oh, I see. The internet was this, you know, I was Times
Person of the Year in 2006? - I love this.
- It's you. Was Time's Person of the Year in 2006, So quickly did people forget. And I think some of it's social media. I think some of it... Look, I hope that... It's possible that some very
sinister things happened. I don't know. I think it might just be like
the effects of social media, but something happened
in the last 20 years. - Oh, okay. So you're just being an old
man who's worried about the... I think it's the cycle thing. It's ups and downs and I
think people rediscover the power of distributed,
of decentralized. I mean that's kinda like what the whole like
cryptocurrency is trying. I think crypto is just carrying
the flame of that spirit of stuff should be decentralized. - It's just such a shame
that they all got rich. If you took all the money outta crypto, it would've been a beautiful place. But no, I mean these people, they sucked all the value
out of it and took it. - Yeah, money kind of
corrupts the mind somehow. It becomes this drug and you forget. - I mean, corrupted all of crypto, you had coins worth billions
of dollars that had zero use. - You still have hope for crypto? - Sure. I have hope for the ideas. I really do. I mean, you know, I want
the US dollar to collapse. I do. - George Hotz. Well, let me sort of, on the AI safety, do you think there's some
interesting questions there though to solve for the open source
community in this case? So like alignment for example,
or the control problem? If you really have super
powerful, you said it's scary. What do we do with it? So not control, not centralized control, but if you were then, you're gonna see some guy or gal release a super powerful language
model open source and here you are, George
Hots thinking, "Holy shit. What ideas do I have
to combat this thing?" So what ideas would you have? - I am so much not
worried about the machine independently doing harm. That's what some of these AI
safety people seem to think. They somehow seem to think
that the machine independently is gonna rebel against its creator. - [Lex] So you don't think
you'll find autonomy? - No, this is Sci-Fi B movie garbage. - Okay, what if the thing writes code? Basically writes viruses? - If the thing writes viruses, it's because the human
told it to write viruses. - Yeah, but there's some things you can't put back in the box. That's kind of the whole point. It kinda spreads. Give it access to the internet, it spreads, installs
itself modifies your shit. - B, B, B plot Sci-Fi. Not real.
- Listen, I'm trying to work, I'm trying to get better
at my plot writing. - The thing that worries me. I mean, we have a real danger to discuss and that is bad humans using the thing to do whatever bad
unaligned AI thing you want. - But this goes to your previous concern that who gets defined who's a
good human, who's a bad human? - Nobody does, we give it to everybody. And if you do anything
besides give it to everybody, trust me, the bad humans will get it. That's who gets power. It's always the bad humans who get power. - Okay, power. And power turns even
slightly good humans to bad. That's the intuition you have. I don't know. - I don't think everyone. I don't think everyone. I just think that like, here's the saying that
I put in one of my blog. When I was in the hacking world, I found 95% of people to be
good and 5% of people to be bad. Just who I personally judged
as good people and bad people. They believed about good
things for the world. They wanted like flourishing
and they wanted growth and they wanted things I consider good. I came into the business world with comma, and I found the exact opposite. I found 5% of people good
and 95% of people bad. I found a world that promotes psychopathy. - I wonder what that means. I wonder if that's anecdotal or if there's truth to that. There's something about capitalism. At the core that promotes the
people that run capitalism, that promotes psychopathy. - That saying may of
course be my own biases. That may be my own biases that these people are a
lot more aligned with me than these other people. So I can certainly recognize that. But in general, I mean, this
is a like common sense maxim, which is the people who
end up getting power are never the ones you want with it. - But do you have a concern
of superintelligent AGI open source, and then
what do you do with that? I'm not saying control
it, it's open source. What do we do with this human species? - That's not up to me. I mean, I'm not a central planner. - No, not central planner,
but you'll probably tweet, "There's a few days left to
live for the human species." - I have my ideas of what to do with it, and everyone else has their
ideas of what to do with it. May the best ideas win. - But at this point, do you brainstorm? Because it's not regulation. It could be decentralized
regulation where people agree that this is just like we
create tools that make it more difficult for you to maybe make it more
difficult for code to spread, antivirus software, this kind of thing. But this- - You're saying that you
should build AI firewalls, that sounds good. You should definitely be
running an AI firewall. You should be running an
AI firewall to your mind. You're constantly under- - [Lex] Such an interesting idea. - Infowars, man. - I don't know if you're
being sarcastic or not. - No, I'm dead serious. - But I think there's power to that. It's like, how do I protect
my mind from influence of human-like or superhuman
intelligent bots? - I would pay so much
money for that product. I would pay so much
money for that product. You know how much money I'd pay just for a spam filter that works? - Well, and Twitter,
sometimes, I would like to have a protection mechanism for my
mind from the outrage mobs. 'Cause they feel like bot-like behavior. There's a large number of
people that will just grab a viral narrative and attack anyone else that believes otherwise. And it's like... - Whenever someone's telling
me some story from the news, I'm always like, "I don't wanna hear it. CIA op bro. It's a CIA op bro." It doesn't matter if that's true or not. It's just trying to influence your mind. You're repeating an ad to me. Like the viral mobs, are like, they're... - No, to me, a defense against those mobs is just getting multiple perspectives, always from sources that make you feel like you're getting smarter. And just actually just
basically feels good. Like a good documentary just feels good. Something feels good
about it, it's well done. It's like, oh, okay, I never
thought of it this way. This just feels good. Sometimes the outrage mobs, even if they have a good point behind it, when they're mocking and
derisive and just aggressive, "You're with us or against us." This fucking- - This is why I delete my tweets. - Yeah, why'd you do that? I missed your tweets.
- You know what it is? The algorithm promotes toxicity. And I think Elon has a much
better chance of fixing it than the previous regime. But to solve this problem, to build a social network
that is actually not toxic without moderation. - Like not a stick but carrot. So where people look for goodness. Make it catalyze the process
of connecting cool people and being cool to each other. Without ever censoring. - Without ever censoring. And like Scott Alexander
has a blog post I like, where he talks about
moderation is not censorship. All moderation you wanna put on Twitter. You could totally make this moderation, you don't have to block it for everybody. You can just have like a filter button. That people can turn off. If there was safe search for Twitter, someone could just turn that off. But then, you'd like take
this idea to an extreme. Well, the network should just show you, this is a couch surfing CEO thing. If it shows you, right now, these algorithms are designed
to maximize engagement. Well it turns out outreach
maximizes engagement. Quirk of the human mind. Just this I fall for it,
everyone falls for it. So yeah, you gotta figure
out how to maximize for something other than engagement. - And I actually believe
that you can make money with that too. I don't think engagement is
the only way to make money. - I actually think it's incredible that we're starting to see, I think again, Elon is doing so much stuff with Twitter like charging people money. As soon as you charge people money, they're no longer the product. They're the customer. And then they can start building something that's good for the customer and not good for the other customer,
which is the ad agencies. - Hasn't picked up his team. - I pay for Twitter doesn't
even get me anything. It's my donation to
this new business model, hopefully working out. - Sure, but for this
business model to work, most people should be
signed up to Twitter. And so, there was something
perhaps not compelling or something like this to people. - I don't think you
need most people at all. I think that why do I
need most people, right? I don't make an 8,000 person company make a 50-person company. - Well, so speaking of which, you worked at Twitter for a bit. - I did.
- As an intern. The world's greatest intern. - Eh. - All right.
- It's been better. - It's been better. Tell me about your time at Twitter. How did it come about and what did you learn
from the experience? - So I deleted my first Twitter in 2010. I had over 100,000 followers back when that actually meant something. And I just saw, you know, my
coworker summarized it well. He's like, "Whenever I see
someone's Twitter page, I either think the same
of them or less of them. I never think more of them." I don't wanna mention any names, but like some people who like, maybe you would like read their books and you would respect them. You see them on Twitter and you're like, "Okay, dude." (chuckles) - Yeah, but there's some people with same, you know who I respect a lot are people that just post
really good technical stuff. And I guess, I don't know, I think I respect them more
for it 'cause you realize, there's like so much depth to this person, to their technical understanding of so many different topics. So I try to follow people, I try to consume stuff that's technical machine learning content. - There's probably a few of those people. And the problem is inherently
what the algorithm rewards. And people think about these algorithms. People think that they are
terrible, awful things. And I love that Elon
opensourced it because I mean, what it does is actually pretty obvious. It just predicts what
you are likely to retweet and linger on. That's what all these algorithms do. It's what TikTok does. It's what all these
recommendation engines do. And it turns out that the
thing that you are most likely to interact with is outrage. And that's the quirk
of the human condition. - I mean, and there's
different flavors of outrage. It could be mockery. You could be outraged, the topic of outrage could be different. It could be an idea, it could be a person. And maybe there's a
better word than outrage, it could be drama. - Sure. Drama. - All this kinda stuff. But it doesn't feel like
when you consume it, it's a constructive
thing for the individuals that consume it in the long term. - Yeah. So my time there, I
absolutely couldn't believe I got crazy amount of hate, just on Twitter for working at Twitter. It seemed like people
associated with this. I think maybe you were
exposed to some of this. - So connection to Elon or
is it working on Twitter? - Twitter and Elon, like the whole- - There's just, Elon's gotten
a bit spicy during that time. A bit political a bit. - Yeah. I remember one of my tweets, it was never, "Go full Republican," and Elon liked it. (laughs) - Oh boy. I mean, there's a rollercoaster of that, but being political on Twitter, boy. And also being just
attacking anybody on Twitter, it comes back at you harder. And if his political and attacks. - [George] Sure, absolutely. - And then letting de-platform
people back on even adds more fun to the beautiful chaos. - I was hoping. And I remember when Elon
talked about buying Twitter six months earlier, he was talking about like
a principled commitment to free speech. And I'm a big believer and fan of that. I would love to see an
actual principled commitment to free speech. Of course, this isn't quite what happened. Instead of the oligarchy
deciding what to ban, you had a monarchy deciding what to ban. Instead of, all the Twitter files shadow and really, the oligarchy
just decides what cloth masks are ineffective against COVID. That's a true statement. Every doctor in 2019 knew it. And now, I'm banned on
Twitter for saying it. Interesting oligarchy. So now, you have a monarchy and he bans things he doesn't like. So, its just different. It's different power. And maybe I align more with him than with the oligarchy, but... - It's not free speech.
- Exactly. But I feel like being a
free speech absolutist on a social network requires
you to also have tools for the individuals to control
what they consume easier. Like not censor. But just like control, like, "Oh, I'd like to see more
cats and less politics." - And this isn't even
remotely controversial. This is just saying you want
to give paying customers for a product what they want. - And not through the
process of censorship, but through the process of like... - Well, it's individualized. It's individualized
transparent censorship, which is honestly what I want. What is an ad blocker? It's individualized
transparent censorship. - But censorship is a strong word and people are very sensitive too. - I know. But I just use words to describe
what they functionally are and what is an ad blocker,
it's just censorship. - [Lex] Well, when I
look at you right now- - But love when you're censoring. - I'm looking at you, I'm censoring everything else out. When my mind is focused on you, you can use the word censorship that way. But usually when people get very sensitive about the censorship thing. I think when anyone is
allowed to say anything, you should probably have tools
that maximize the quality of the experience for individuals. So for me, what I really value, boy, would be amazing to somehow
figure out how to do that. I love disagreement and
debate and people who disagree with each other disagree with me especially in the space of ideas. But the high quality
ones, so not derision. - Maslow's hierarchy of argument. I think there's a real word for it. - Probably. There's just a way of
talking that's like snarky and so on that somehow
gets people on Twitter and they get excited and so on. - You have like ad ho and him
refuting the central point. I've like seen this as an
actual pyramid somewhere. - And it's like all the wrong
stuff is attractive to people. - I mean, we can just train a classifier to absolutely say what
level of Maslow's hierarchy of argument are you at. And if it's ad hominem,
I'm like, okay, cool. I turned on the no ad hominem filter. - I wonder if there's a social
network that will allow you to have that kind of filter. - Yeah, so here's a problem with that. It's not going to win in a free market. What wins in a free market
is all television today is reality television
'cause it's engaging. Engaging is what wins in a free market so it becomes hard to keep
these other more nuanced values. - Well, okay, so that's the
experience of being on Twitter. But then you got a chance to also, together with other engineers and with Elon's sort of look, brainstorm when you step into a code base. It's been around for a long time. There's other social networks. Facebook, this is old code base. And you step in and see, okay, how do we make with a fresh
mind progress on this code base? What did you learn about
software engineering, about programming from
just experiencing that? - So my technical recommendation to Elon, and I said this on the
Twitter space, is afterward, I said this many times
during my brief internship was that you need
refactors before features. This code base was. And look, I've worked at
Google, I've worked at Facebook, Facebook has the best code,
then Google, then Twitter. And you know what? You can know this because look at the machine
learning frameworks. Facebook released PyTorch, Google released TensorFlow
and Twitter released... - It's a proxy. But yeah, the Google code
base is quite interesting. There's a lot of really good
software engineers there but the code base is very large. - The code base was good in 2005. It looks like 2005 era. - There's so many products, so many teams. I feel like Twitter does less obviously. Much less than Google in
terms of the set of features. So I can imagine the number
of software engineers that could recreate Twitter is much smaller than to recreate Google. - Yeah, I still believe in
the amount of hate I got for saying this that 50 people could build and maintain Twitter pretty- - What's the nature of the hate? - Comfortably. - [Lex] That you don't know
what you're talking about. - You know what it is? And this is my summary
of like the hate I get on Hacker News. It's like when I say I'm
going to do something, they have to believe that it's impossible. Because if doing things was possible, they'd have to do some soul
searching and ask the question why didn't they do anything? - So when you say- - And I do think that's
where the hate comes from. - When you say, "Well,
there's a core truth to that." So when you say, "I'm
gonna solve self-driving," people go like, "What
are your credentials? What the hell are you talking about? This is an extremely difficult problem. Of course you're a new
that doesn't understand the problem deeply." I mean that was the same nature of hate that probably Elon got
when you first talked about autonomous driving. But there's pros and cons to that 'cause there is experts in this world. - No, but the mockers aren't experts. The people who are mocking are not experts with carefully reasoned arguments about why you need 8,000
people to run a bird app. They're, "But the people
are gonna lose their jobs." - Well that, but also there's
the software engineers that probably criticize, "No, it's a lot more
complicated than you realize," but maybe it doesn't need
to be so complicated. - You know, some people in the world like to create complexity. Some people in the world
thrive under complexity like lawyers. Lawyers want
the world to be more complex because you need more lawyers,
you need more legal hours. I think that's another, if there's two great evils in the world, it's centralization and complexity. - Yeah, and the one of the
sort of hidden side effects of software engineering is
finding pleasure and complexity. I mean, I don't remember just taking all the software engineering courses and just doing programming
and this just coming up in this object-oriented
programming kind of idea. Not often do people tell you, do the simplest possible thing. A professor, a teacher is
not gonna get in front like, "This is the simplest way to do it." They'll say like, "There's the right way" and the right way at
least for a long time, especially I came up with like Java, is there's so much boilerplate, so many classes, so many like designs and architectures and so on. Like planning for features
far into the future and planning poorly and
all this kind of stuff. And then there's this like code
base that follows you along and puts pressure on you and nobody knows what different parts do,
which slows everything down. There's a kind of bureaucracy
that's instilled in the code as a result of that. But then you feel like, "Oh well, I follow good
software engineering practices." It's an interesting trade off 'cause then you look at like
the ghettoness of like Perl and the old, like how
quick you could just write a couple lines and you get stuff done. That trade off is interesting
or bash or whatever. These kind of ghetto
things you can do in Linux. - One of my favorite
things to look at today is how much do you trust your tests? We've put a ton of effort in comma, and I've put a ton of effort
in tinygrad into making sure if you change the code and the tests pass, that you didn't break the code. Now this obviously is not always true, but the closer that is to true. The more you trust your
tests, the more you're like, oh I gotta pull request
and the tests pass. I feel okay to merge that, the
faster you can make progress. - You're always programming your tests in mind developing tests. With that in mind that if it
passes, it should be good. - And Twitter had a... - Not that. - Was impossible to make
progress in the code base. - What other stuff can you
say about the code base that made it difficult? What are some interesting sort
of quirks broadly speaking from that compared to just your experience with comma and everywhere else? - The real thing that, I spoke to a bunch of individual contributors
at Twitter and I just asked. I'm like, "So what's
wrong with this place? Why does this code look like this?" And they explained to me what Twitter's promotion system was. The way that you got promoted
to Twitter was you wrote a library that a lot of people used. So some guy wrote an nginx
replacement for Twitter. Why does Twitter need
an nginx replacement? What was wrong with nginx? "Well you see, you're not gonna get
promoted if you use nginx, but if you write a
replacement and lots of people start using it as the Twitter
front end for their product, then you're gonna get promoted." - So interesting 'cause from
an individual perspective, how do you create the kind
of incentives that will lead to a great code base? Okay, what's the answer to that? - So what I do at comma and at tiny corp is you
have to explain it to me. You have to explain it to
me what this code does. And if I can sit there and
come up with a simpler way to do it, you have to rewrite it. You have to agree with
me about the simpler way Obviously, we can have a
conversation about this. It's not dictatorial, but
if you're like, "Wow, wait, that actually is way simpler." The simplicity is important. - But that requires people
that overlook the code at the highest levels to be like, okay. - It requires technical
leadership you trust. - Yeah, technical leadership. So managers or whatever should
have to have technical savvy, deep technical savvy. - Managers should be better programmers than the people who they manage. - And that's not always
obvious to trivial to create, especially large companies. Managers get soft. - I've instilled this
culture at comma and comma has better programmers
than me who work there. But again, I'm like the old
guy from Goodwill Hunting, it's like, "Look man, I
might not be as good as you, but I can see the difference
between me and you." and this is what you need. This is what you need at the top. Or you don't necessarily need the manager to be the absolute best,
I shouldn't say that, but they need to be
able to recognize skill. - Yeah. And have good intuition. Intuition that's laden with
wisdom from all the battles of trying to reduce
complexity in code basis. - I took a political approach
at comma too that I think is pretty interesting. I think Elon takes the
same political approach. Google had no politics and
what ended up happening is the absolute worst kind
of politics took over. Comma has an extreme amount of
politics and they're all mine and no dissidents is tolerated. - So it is a dictatorship. - Yup, it's an absolute dictatorship. Elon does the same thing. Now, the thing about my
dictatorship is here are my values. - Yeah, it's transparent. - It's transparent. It's a transparent dictatorship. And you can choose to opt
in or you get free exit. That's the beauty of companies. If you don't like the
dictatorship, you quit. - So you mentioned rewrite before or refactor before features. If you were to refactor
the Twitter code base, what would that look like? And maybe also comment on how
difficult is it to refactor? - The main thing I would
do is first of all, identify the pieces and then put tests in between the pieces. So there's all these different Twitter as a microservice architecture, there's all these different microservices. And the thing that I was
working on there look like, "George didn't know any JavaScript. He asked how to fix search,
blah, blah, blah, blah, blah." Look man, the thing is, I'm upset that the way
that this whole thing was portrayed because it
wasn't like taken by people. Like, honestly, it was taken
by people who started out with a bad faith assumption. And yeah, I mean, I look, I can't like... - And you as a program, were just being transparent
out there actually having fun and this is what
programming should be about. - I love that Elon gave
me this opportunity. Like really, it does. And the day I quit, he came on my Twitter spaces afterward and we had a conversation. I respect that so much. - Yeah, and it's also
inspiring to just engineers and programmers and just, yeah, it's cool. It should be fun. The people that were hating
on it is like, oh man. - It was fun. It was fun, it was stressful. But I felt like I was at a
cool like point in history and I hope I was useful. I probably kind of wasn't but- - Well, you also were one of the people that made strong case to refactor. And that's a really
interesting thing to raise. The timing of that is really interesting, if you look at just the
development of autopilot, going from Mobileye to just, if you look at the history of semi-autonomous driving in Tesla is more and more like
you could say refactoring or starting from scratch,
redeveloping from scratch. - It's refactoring all the way down. - And the question is,
can you do that sooner? Can you maintain product profitability? And what's the right time to do it? How do you do it? On any one day, you don't
wanna pull off the bandaids. Everything works, it's just
little fixed here and there. But maybe starting from scratch. - This is the main philosophy of tinygrad. You have never refactored enough. Your code can get smaller, your code can get simpler,
your ideas can be more elegant. - But would you consider, say you were like running
Twitter development teams, engineering teams, would you go as far as different programming language? Just go that far? - I mean, the first thing that
I would do is build tests. The first thing I would do
is get a CI to where people can trust to make changes. Before I've touched any code, I would actually say no
one touches any code. The first thing we do is
we test this code base. I mean this is classic. This is how you approach
a legacy code base. This is how do we approach
a legacy code base book, we'll tell you. - And then you hope that
there's modules that can live on for a while and then you add new ones maybe in a different language or design- - [George] Before we add new
ones, we replace old ones. - Yeah, yeah. Meaning like replace old
ones with something simpler. - We look at this thing
that's 100,000 lines and we're like, "Well okay, maybe this did even make sense in 2010, but now, we can replace this
with an open source thing." And we look at this here,
here's another 50,000 lines. Well actually, we can replace
this with 300 lines a go. And you know what? I trust that the go
actually replaces this thing because all the tests still pass. So step one is testing. And then step two is
the programming language is an afterthought. You'll let a whole lot of
people compete be like, okay, "Who wants to rewrite a module? Whatever language you wanna write it in, just the tests have to pass." And if you figure out
how to make the test pass but break the site, we
gotta go back to step one. Step one is get tests that you trust in order to make changes in the code base. - I wonder how hard it is too. 'Cause I'm with you on
testing and everything I have from tests to like asserts to everything. But code is just covered in this because it should be very easy
to make rapid changes and no, that's not gonna break everything. And that's the way to do it. But I wonder how difficult
is it to integrate tests into a code base that
doesn't have many of them? - So I'll tell you what
my plan was at Twitter. It's actually similar to
something we use at comma. So at comma we have this
thing called process replay and we have a bunch of routes
that'll be run through. So comma is a microservice
architecture too with microservices in the driving. We have one for the
cameras, one for the sensor, one for the planner, one for the model. And we have an API which the microservices talk to each other with. We use this custom thing
called Serial, which uses ZMQ. Twitter uses thrift and
then it uses this thing called Finagle, which
is a Scala RPC backend. But this doesn't even really matter. The thrift and Finagle layer
was a great place I thought to write tests. To start building something
that looks like process replay. So Twitter had some stuff
that looked kind of like this, but it wasn't offline, it was only online. So you could ship like
a modified version of it and then you could redirect
some of the traffic to your modified version
and div those two. But it was all online. There was no like CI in
the traditional sense. I mean there was some, but
like it was not full coverage. - So you can't run all of Twitter
offline to test something. - Well then this was another problem. You can't run all of Twitter. - Period.
- Any one person can run. - Twitter runs in three
data centers and that's it. There's no other place you can
run Twitter, which is like, "George, you don't understand this is modern software development." No, this is bullshit. Why can't it run on my laptop? "What do you do with Twitter? You can run it." Yeah, okay. Well I'm I'm not saying
you're gonna download the whole database to your laptop, but I'm saying all the
middleware and the front end should run on my laptop, right? - That sounds really compelling. But can that be achieved by a code base that grows over the years? I mean the three data
centers didn't have to be because it's their totally
different like designs. - The problem is more like, why did the code base have to grow? What new functionality has
been added to compensate for the lines of code that are there? - One of the ways to explain
is that the incentive for software developers to
move up in the companies to add code. To add especially large- - And you know what? The incentive for politicians to move up in the political structure is to add laws. Same problem. - Yeah. Yeah. The flip side is to
simplify, simplify, simplify. - I mean, you know what? This is something that I do differently from Elon with comma
about self-driving cars. I hear the new version's gonna come out and the new version is
not gonna be better, but at first, and it's gonna
require a ton of refactors. And I say, okay, take as long as you need. You convinced me this
architecture is better? Okay, we have to move to it. Even if it's not gonna make
the product better tomorrow, the top priority is getting
the architecture right. - So what do you think about a thing where the product is online? So I guess would you do a refactor? If you ran engineering on Twitter, would you just do a refactor?
How long would it take? What would that mean for the running of the actual service? - You know, I'm not the right person to run Twitter. I'm just not. And that's the problem. I don't really know. I don't really know if that's... A common thing that I thought
a lot while I was there was whenever I thought
something that was different to what Elon thought, I'd have to run something
in the back of my head reminding myself that Elon is
the richest man in the world and in general, his ideas
are better than mine. Now there's a few things
I think I do understand and know more about, but like in general, I'm not qualified to run Twitter. I shouldn't say qualified, but like I don't think
I'd be that good at it. I don't think I'd be good at it. I don't think I'd really
be good at running an engineering organization at scale. I think I could lead a very
good refactor of Twitter and it would take like
six months to a year and the results to show at the end of it would be feature development in general. Takes 10x less time, 10x less man hours. That's what I think I could actually do. Do I think that it's the right
decision for the business? Above my pay grade. - Yeah, but a lot of
these kinds of decisions are above everybody's pay grade. - I don't wanna be a manager,
I don't wanna do that. Like, if you really forced me to, yeah, it would make me maybe... Make me upset if I had
to make those decisions. I don't wanna... - Yeah. But a refactor is so compelling. If this is to become something much bigger than what Twitter was, it feels like a refactor has
to be coming at some point. - "George, you're a
junior software engineer." Every junior software
engineer wants to come in and refactor all code." Okay, that's like your opinion man. - Yeah, sometimes, they're right. - Well, like whether they're right or not, it's definitely not for that reason. It's definitely not a question
of engineering prowess. It is a question of maybe
what the priorities are for the company. And I did get more intelligent feedback from people I think in good faith. Like saying that from actually from Elon and from Elon sort of
like, people were like, "Well, you know. A stop the world refactor
might be great for engineering but we have a business to run. And hey, above my pay grade. - What'd you think about
Elon as an engineering leader having to experience him in
the most chaotic of spaces? I would say. - My respect for him is unchanged. And I did have to think a lot more deeply about some of the decisions
he's forced to make. - About the tensions within those, the trade-offs within those decisions? - About like a whole like
matrix coming at him. I think that's Andrew Tate's word for it. Sorry to borrow it. - Also bigger than engineering. Just everything. - Yeah, like the war on the woke. Man, and he doesn't have to do this. He doesn't have to. He could go like Pirogue and
go chill at the four seasons of Maui, but see one person I respect and one person I don't. - So his heart is in the right
place fighting in this case for this ideal of the
freedom of expression. - Well, I wouldn't define
the ideal so simply. I think you can define the ideal no more than just saying
Elon's idea of a good world. Freedom of expression is... - But to you, the downsides
of that is the monarchy. - Yeah, I mean monarchy
has problems, right? But I mean, would I trade
right now the monarch or the current oligarchy which runs America for the monarchy? Yeah, I would sure. For the Elon monarchy, yeah. You know why? Because power would cost
1 cent a kilowatt hour. 10th of a cent a kilowatt hour. - [Lex] What do you mean? - Right now, I pay about
20 cents a kilowatt hour for electricity in San Diego. That's like the same
price you paid in 1980. What the hell? - So you would see a lot
of innovation with Elon. - Maybe it'd have maybe
have some hyperloops. And I'm willing to make that trade off. I'm willing to make, and this is why, people think that like
dictators take power through some untoward mechanism. Sometimes, they do. But usually it's 'cause
the people want them. And the downsides of a dictatorship. I feel like we've gotten to a point now with the oligarchy where I
would prefer the dictator. - What'd you think about Scala
as the programming language? - I liked it more than I thought. I did the tutorials. I was very new to it. It would take me six months to
be able to write good Scala. - I mean what did you learn about learning a new programming language from that? - Oh, I love doing like new programming, tutorials and doing them,
I did all this for Rust. Some of it's upsetting JVM
roots but it is a much nicer, in fact I almost don't
know why Kotlin took off and not Scala. I think Scala has some
beauty that Kotlin lacked, whereas Kotlin felt a lot more,
I mean it was almost like, I don't know if it actually
was a response to Swift, but that's kind of what it felt like. Like Kotlin looks more
like Swift and Scala looks more like a functional
programming language. More like like an OCaml or Haskell. - Let's actually just explore,
we touched it a little bit, but just on the science
and the art of programming, for you personally, how
much of your programming has done with GPT currently? - None.
- None. - Don't use it at all. - Because you prioritize
simplicity so much? - Yeah, I find that a lot of it is noise. I do use VS code and I do like... some amount of auto complete. I do like a very like feels
like rules based auto complete. Like an auto complete
that's going to complete the variable name for me. So I don't to type it,
I can just press tab. That's nice. But I don't want an auto complete. You know what I hate when auto completes when I type the word
"four" and it like puts like two, two parentheses and
two semicon and two braces. I'm like, oh man. - Well I mean with vs
code and GPT with Codex, you can brainstorm. I'm probably the same as you, but I like that it generates
code and you basically disagree with it and write something simpler. But to me that somehow is inspiring, it makes me feel good. It also gamifies the
simplification process 'cause I'm like, "Oh
yeah, you dumb AI system. You think this is the way to do it. I have a simpler thing here." - It just constantly
reminds me of bad stuff. I mean I tried the same thing with RAP. I tried the same thing with rap and I actually think I'm a much better programmer than rapper. But I even tried, I was like, okay, can we get some inspiration
from these things for some rap lyrics? And I just found that it would go back to the most cringey tropes
and dumb rhyme schemes. And I'm like, 'Yeah, this is
what the code looks like too." - I think you and I probably
have different threshold for cringe code. You probably hate cringe code. So it's for you, I mean, boilerplate as a part of code, like some of it... And some of it is just like faster lookup 'cause I don't know about you but I don't remember everything. I'm offloading so much of my memory about different functions, library functions and
all that kind of stuff. This GPT just is very
fast at standard stuff, and standard library stuff,
basic stuff that everybody uses. - Yeah, I think that, I don't know, I mean, there's just a
little of this in Python. And maybe if I was coding
more in other languages, I would consider it more. But I feel like Python
already does such a good job of removing any boilerplate. - That's true. - It's the closest thing you
can get to pseudocode, right? - [Lex] Yeah, that's true. That's true. - And like, yeah sure. Yeah, great, GPT. Thanks for reminding me
to free my variables. Unfortunately, you didn't really recognize the scope correctly and
you can't free that one but you put the freeze
there and like I get it. - Fiverr. Whenever I've used
Fiverr for certain things like design or whatever. It's always you come back. My experience with Fiverr is closer to your experience with
programming with GPT is like, you're just frustrated and feel worse about the whole process of design and art and whatever I use Fiverr. (exhales) Still, I just feel like
later version of GPT, I'm using GPT as much as possible to just learn the dynamics of it. These early versions 'cause
it feels like in the future, you'll be using it more and more. For the same reason I
gave away all my books and switched to Kindle 'cause how long are we
gonna have paper books? Like 30 years from now, I wanna learn to be reading on Kindle even though I don't enjoy it as much and you learn to enjoy
it more in the same way I switched from... Let me just pause. I switched from Emacs to VS Code. - Yeah. I switched from Vim to VS
code. I think similar but... - Yeah, it's tough. And that Vim to VS code is even tougher 'cause Emacs is old, more outdated. Feels like the community is more outdated. Vim is like pretty vibrant still. - I never used any of the plugins. I still don't use any of the plug- - That's what I looked
at myself in the mirror. I'm like yeah, you wrote
some stuff in Lisp, yeah. - But I never used any of
the plugins in Vim either. I had the most Vanilla Vim,
I have a syntax highlighter. I didn't even have auto complete. These things I feel like help
you so marginally that... Okay now, VS code's auto
complete has gotten good enough that okay, I don't have to set it up, I can just go into any code
base and auto complete's right 90% of the time. Okay cool. I'll take it. So I don't think I'm gonna
have a problem at all adapting to the tools once they're good. But like the real thing
that I want is not something that tab completes my
code and gives me ideas. The real thing that I
want is a very intelligent pair programmer that comes up
with a little popup saying, "Hey, you wrote a bug on line
14 and here's what it is." Now I like that. You know what does a good job of this? Mypy. I love Mypy. Mypy is This fancy type
checker for Python. And actually I tried like
Microsoft released one too and it was like 60% false positives. Mypy is like 5% false positives. 95% of the time, it recognizes. I didn't really think about that typing interaction correctly. Thank you, Mypy. - So you like type hinting. You like pushing the language towards being a typed language. - Oh yeah, absolutely. I think optional typing is great. I mean look, I think that it's
like a meet in the middle. Python has these optional
type hint and C++ has auto. - C++ allows you to take a step back. - Well, C++ would have
you brutally type out SGD string iterator. Now I can just type auto, which is nice. And then Python used to just have A. What type is A? It's an A. A:str. Oh okay. It's a string. Cool. I wish there was a way, like a simple way in Python
to like turn on a mode which would enforce the types. - Yeah, like give a warning
when there's no type or something like this. - Well no, to give a
warning where like Mypy is a static type checker,
but I'm asking just for a runtime type checker. There's like ways to like hack this in, but I wish it was just like
a flag, like Python 3-t. - Oh, I see. I see. - Enforce the types runtime.
- Yeah. I feel like that makes
you a better programmer, that's the kind of test, right? The type remains the same. - Well, no, that I didn't
like mess any types up. But again, Mypy's getting
really good and I love it and I can't wait for some of these tools to become AI powered. I want AIs reading my code
and giving me feedback. I don't want AIs writing half-assed auto complete stuff for me. - I wonder if you can now
take GPT and give it a code that you wrote for a function and say how can I make this simpler and have it accomplish the same thing? I think you'll get some
good ideas on some code. Maybe not the code you write
for tinygrad type of code, 'cause that requires
so much design thinking but other kinds of code. - I don't know. I downloaded the plugin
maybe like two months ago. I tried it again and found the same. Look, I don't doubt that these models are going to first become useful to me, then be as good as me and then surpass me. But from what I've seen today, it's like someone occasionally
taking over my keyboard that I hired from Fiverr. I'd rather- - But ideas about how to debug the coder, basically a better debugger
is really interesting. - But it's not a better debugger but yes, I would love a better debugger. - Yeah, it's not yet. But it feels like it's not too far. - Yeah, one of my
coworkers says he uses them for print statements. Every time he has to like,
just like when he needs, the only thing I can
really write is like, okay, I just wanna write the thing to print a state out right now. - Oh that definitely is much
faster as print statements. I see myself using that a lot. 'Cause it figures out what
the rest of the function is. Just like, okay, print everything. - Yeah, print everything. And then yeah, like if
you want a pretty printer, maybe I'm like, yeah, you know what? I think in two years, I'm
gonna start using these plugins a little bit. And then in five years, I'm
gonna be heavily relying on some AI augmented flow
and then in 10 years... - Do you think you'll ever get to 100%? What's the role of the
human that it converges to as a programmer? - No. - So you think it's all generated? - Our niche becomes, I think it's over for humans in general. It's not just programming,
it's everything. - So niche becomes, well... - Our niche becomes smaller
and smaller and smaller. In fact I'll tell you what the last niche of humanity is gonna be. There's a great book and if I recommended "Metamorphosis of Prime
Intellect" last time, there is a sequel called "A
Casino Odyssey in Cyberspace." And I don't wanna give
away the ending of this, but it tells you what the last
remaining human currency is. And I agree with that. - We'll leave that as the cliffhanger. So no more programmers left, huh? That's where we're going. - Well, unless you want handmade code, maybe they'll sell it on Etsy.
This is handwritten code. Doesn't have that machine polish to it. It has those slight imperfections that would only be written by a person. - I wonder how far away we are from that. I mean, there's some aspect to, on Instagram, your title is
listed as prompt engineer. - (laughs) Right. Thank you for noticing. - I don't know if it's ironic or non, or sarcastic or non. What do you think of prompt
engineering as a scientific and engineering discipline? Or maybe and maybe art form. - You know what? I started comma six years ago
and I started the tiny corp a month ago. So much has changed. I'm now thinking, I'm now like, I started like going through
similar comma processes to starting a company. I'm like, "Okay, I'm gonna
get an office in San Diego. I'm gonna bring people here." I don't think so. I think I'm actually gonna do remote. "George you're gonna do
remote, you hate remote." "Yeah, but I'm not
gonna do job interviews. The only way you're gonna get a job is if you contribute to the GitHub." And then interacting through GitHub, GitHub being the real like
project management software for your company, and the thing pretty much
just is a GitHub repo Is like showing me kind of
what the future of, okay. So a lot of times I'll go on a Discord or county grad discord and I'll
throw out some random like, hey, can you change, instead of having log an X as LL lops, change it to log to an X2? It's pretty small change. You can just use like
change a base formula. That's the kind of task
that I can see in AI being able to do in a few years. In a few years I could see
myself describing that, and then within 30 seconds,
a pull request is up that does it. And it passes my CI and I merge it. So I really started thinking about well what is the future of jobs? How many AIs can I employ at my company? As soon as we get the first tinybox up, I'm gonna stand up a 65B
LLaMA in the Discord. And it's like, yeah, here's the tinybox. He's just like, he's chilling with us. - Basically, like you said, with niches, most human jobs will
eventually be replaced with prompt engineering. - Well, prompt engineering
kind of is this like, as you like move up the stack. There used to be humans actually
doing arithmetic by hand. There used to be like big farms of people doing pluses and stuff. And then you have like spreadsheets. And then, okay, the spreadsheet
can do the plus for me. And then you have like macros. And then you have like
things that basically just are spreadsheets under the hood like accounting software. As we move further up the abstraction, well what's at the top
of the abstraction stack? Well the prompt engineer. What is the last thing if you think about like humans
wanting to keep control? Well what am I really in the
company but a prompt engineer? - Isn't there a certain point
where the AI will be better at writing prompts? - But you see, the problem
with the AI writing prompts, a definition that I
always liked of AI was AI is the do what I mean machine. The computer is so pedantic
it does what you say. But you want the do what I mean machine. You want the machine where you say, get my grandmother
outta the burning house. It reasonably takes your grandmother and puts her on the ground, not lift her 1,000 feet
above the burning house and lets her fall. There's no Yudkowsky examples. - But it's not going to find the meaning. I mean, to do what I mean,
it has to figure stuff out. And the thing you'll maybe ask it to do is run government for me. - And do what I mean very
much comes down to how aligned is that AI with you. Of course, when you talk
to an AI that's made by a big company in the cloud, the AI fundamentally is
aligned to them, not to you. And that's why you have to buy a tinybox. So you make sure the AI
stays aligned to you. Every time that they start to pass AI regulation or GPU regulation, I'm gonna see sales of tinyboxes spike. It's gonna be like guns. Every time they talk about
gun regulation, boom. Gun sales. - So in the space of
AI, you're an anarchist. Anarchism, espouser, believer. - I'm an informational anarchist, yes I'm an informational anarchist
and a physical statist. I do not think anarchy in the
physical world is very good because I exist in the physical world. But I think we can construct
this virtual world. We're anarchy, it can't hurt you. I love that Tyler, the creator tweet, "Yo, cyberbullying isn't real man. Turn it off, the screen. Close your eyes." - Yeah. But how do you prevent the
AI from basically replacing all human prompt engineers where nobody's the
prompt engineer anymore? So autonomy, greater and greater autonomy until it's full autonomy. And that's just where it's headed. 'Cause one person's gonna
say, run everything for me. - You see, I look at potential futures
and as long as the AIs go on to create a vibrant civilization with diversity and complexity across the universe, more power to them. I'll die. If the AIs go on to
actually like turn the world into paperclips and then
they die out themselves, well that's horrific and we
don't want that to happen. So this is what I mean
about like robustness. I trust robust machines. The current AIs are so not robust. This comes back to the
idea that we've never made a machine that can self-replicate. But if the machines are truly robust and there is one prompt
engineer left in the world, hope you're doing good, man. Hope you believe in God. Go by God and go forth and
conquer of the universe. - Well you mentioned
'cause I talked to Mark about faith in God and you said you were impressed by that. What's your own belief in God and how does that affect your work? - You know, I never really
considered, when I was younger, I guess my parents were atheists, so I was raised kind of atheist and I never really considered how absolutely silly atheism is. 'Cause I create worlds. Every game creator, how
are you an atheist, bro? You create worlds who. "But no one created art world, man. That's different. Haven't you heard about like
the Big Bang and stuff?" Yeah, I mean what's the Skyrim
myth, origin story in Skyrim? I'm sure there's like
some part of it in Skyrim, but it's not like if you ask the creators. The Big Bang is in universe, right? I'm sure they have some
big bang notion in Skyrim. But obviously, is not at all how Skyrim was actually created. It was created by a bunch
of programmers in a room. So it struck me one day
how just silly atheism is. Like of course, we were created by God. It's the most obvious thing. - Yeah, that's such a nice way to put it. We're such powerful creators ourselves. It's silly not to conceive
that there's creators even more powerful than us. - Yeah. And then I also just like, I like that notion. That notion gives me a lot of, I mean, I guess you can talk about what it gives a lot of religious people. It's like, it just gives me comfort. It's like, you know what? If we mess it all up and we die out. - Yeah, and the same way that a video game has comfort in it. - God will try again. - Or there's balance. Somebody figured out
a balanced view of it. So it all makes sense in the end. Like a video game is
usually not gonna have crazy, crazy stuff. - People will come up with like a... "Well, yeah, but like,
man who created God." I'm like, "That's God's problem." No, I'm not gonna think. This is what you're asking me? Believe in God?" - I'm just this NPC living in his game. - I mean, to be fair, if
God didn't believe in God, he'd be silly as the atheists here. - What do you think is
the greatest computer game of all time? Do you have any time
to play games anymore? Have you played Diablo IV? - I have not played Diablo IV. - I will be doing that shortly, I have to. There's just so much history
with one, two, and three. - You know what? I'm gonna say World of Warcraft. And it's not that the
game is such a great game. It's not. It's that I remember in
2005 when it came out, how it opened my mind to ideas. It opened my mind to this
whole world we've created. And there's almost been
nothing like it since. You can look at MMOs today, and I think they all have lower user bases than World of Warcraft. Yvonne line's kind of cool. But to think that like,
everyone, people are always like, they're looking at the Apple headset. What do people want in this VR? Everyone knows what they want. I want Ready Player one. And like that. So I'm gonna say World of Warcraft and I'm hoping that like games can get out of this whole mobile gaming
dopamine pump thing and like- - Create worlds.
- Create worlds, yeah. - And worlds that captivate
a very large fraction of the human population. - Yeah, and I think it'll
come back, I believe. - But MMO really, really pull you in. - Games do a good job. I mean, okay. Are other like two
other games that I think are very noteworthy from Skyrim and GTA V? - Skyrim, yeah, that's
probably number one for me. GTA. What is it about GTA? I mean, I guess GTA is real life. I know there's prostitutes
and guns and stuff. - (laughs) They exist in real life too. - Yes, I know. But it's how I imagine your life to be. - Actually, I wish it was that cool. - 'Cause there's Sims,
which is also a game I like, but it's a gamified version of life. But I would love a
combination of Sims and GTA. So more freedom, more
violence, more rawness, but with also ability to have a career and family and this kind of stuff. - What I'm really excited about in games is once we start getting
intelligent AIs to interact with. Like the NPCs games have never been. - But conversationally, in every way. - In like every way. When you are actually building a world and a world imbued with intelligence. Running World of Warcraft,
like you're limited by way you're running on a penny of four. How much intelligence can you run? How many flops did you have? But now, when I'm running a
game on 100 paid a flop machine, let's five people, I'm trying to make this a thing. 20 paid a flops of compute
is one person of compute. I'm trying to make that a unit. - 20 flops. Is one person.
- One person. - One person-flop. - Think of horsepower. What's a horsepower? It's how powerful a horse is. What's a person of compute? - Flop. I got it. That's interesting. VR also adds a mean in
terms of creating worlds. - You know what? Bought a Quest 2. I put it on and I can't believe
the first thing they show me is a bunch of scrolling clouds
and a Facebook login screen. You had the ability to
bring me into a world, And what did you give me? A pop-up. And this is why you're
not cool Mark Zuckerberg. But you could be cool. Just make sure on the Quest 3, you don't put me into clouds
in a Facebook login screen. Bring me to a world. - I just tried Quest 3. It was awesome. But hear that guys? I agree with that. - Have this clouds in the world. It was just so... - Because I mean the
beginning, what is it? Todd Howard said this about
design of the beginning of the games he creates. The beginning is so, so, so important. I recently played Zelda
for the first time. Zelda Breath of the
Wild, the previous one. And it's very quickly,
within like 10 seconds, you come out of like a cave type-place, and this world opens up. It pulls you in. You forget whatever troubles I was having. Whatever... - [George] I gotta play
that from the beginning. I played it for like an
hour at a friend's house. - Ah, no. The beginning, they got it. They did it really well. The expansiveness of that space, the peacefulness of that place,
they got this, the music. I mean, so much of that, it's creating that world
and pulling you right in. - I'm gonna go buy a Switch. I'm gonna go today and buy a Switch. - You should. Well the new one came out. I haven't played that yet
but Diablo IV something. I mean, there's sentimentality also, but something about VR
really is incredible. But the new Quest 3 is mixed reality. And I got a chance to try that. So it's augmented reality
and for video games, it's done really, really well. - Is it pass through or cameras? - Cameras.
- It's cameras. - The Apple one, is that
one pass through or cameras? - I don't know. I don't know how real it is. I don't know anything. - Coming out January? - Is it January? Or is it some point? - Some point in time, maybe not January. Yeah, maybe that's my optimism.
But Apple, I will buy it. I don't care if it's
expensive and does nothing. I will buy it. I'll support this future endeavor. - You're the meme. Oh, yes. I support competition. It seemed like Quest was like
the only people doing it. And this is great that they're like... - You know what? And this is another place
we'll give some more respect to Mark Zuckerberg. The two companies that have
endured through technology are Apple and Microsoft. And what do they make? Computers and business services. All the memes, social
ads, they all come and go, but you wanna endure, build hardware. - Yeah. And that does a really interesting job. I mean, maybe I'm new with
this, but it's a $500 headset, Quest 3 and just having
creatures run around the space, our space right here. To me, this is very like boomer statement, but it added windows to the place. - I heard about the aquarium. - Aquarium, but in this case, it was a zombie game,
whatever, it doesn't matter. But it modifies the space in a way where it really feels like a
window and you can look out. It's pretty cool. It was like a zombie game.
They're running at me, whatever but what I was enjoying is the fact that there's like a window and they're stepping on
objects in this space. That was a different kind of
escape also because you can see the other humans, so it's
integrated with the other humans. It's really... - And that's why it's more
really important than ever that the AI is running on those systems are aligned with you. They're gonna augment your entire world. - Oh yeah. And those AIs, I mean, you think about all the dark
stuff, like sexual stuff. Like if those AIs threaten
me, that could be haunting. If they like threaten me
in a non-video game way. they'll know personal information about me and then you lose track of
what's real, what's not. What if stuff is like hacked? - There's two directions the
AI girlfriend company can take. There's like the high
brow, something like her, maybe something you talk to. And then there's the
low brow version of it where I wanna set up a
brothel in Times Square. Yeah, it's not cheating if it's a robot, it's a VR experience. - Is there an in between?
- No. I wanna do that one with that one. - Have you decided yet?
- No, I'll figure it out. We'll see where the technology goes. - I would love to hear your opinions for George's third company. What to do, the broth on Times Square or the her experience? What do you think company
number four will be? You think there'll be
a company number four. - There's a lot to do
in company number two. Just like I'm talking about
company number three now, none of that tech exists yet. There's a lot to do in company number two. Company number two is going
to be the great struggle of the next six years. And if the next six years how centralized is compute going to be. The less centralized
compute is going to be, the better of a chance we all have. - So you're like a flag bearer for open source distributed
decentralization of compute. - We have to. We have to. Or they will just completely dominate us. I showed a picture on stream
of a man in a chicken farm. You ever seen one of those like
factory farm chicken farms? Why does he dominate all the chickens? Why is he smarter? He's smarter. Some people, some people
on Twitch were like, "He's bigger than the chickens." Yeah, and now here's a man in a cow farm. So it has nothing to do with their size and everything to do
with their intelligence. And if one central organization
has all the intelligence, you'll be the chickens and
they'll be the chicken man. But if we all have the intelligence,
we're all the chickens. We're not all the man,
we're all the chickens. And there's no chicken man. - There's no chicken man. Or just chickens in Miami. - [George] He was having a good life. - Yeah, I'm sure he was. I'm sure he was. What have you learned from launching a running comma.ai and tiny corp? So this starting a company
from an idea and scaling it, and by the way, I'm all in on tinybox, so I guess it's pre-order only now? - I wanna make sure it's good. I wanna make sure that
the thing that I deliver is not gonna be like a Quest
2, which you buy and use twice. I mean, it's better than a
Quest which you bought and used less than once statistically. - Well, if there's a beta
program for tinybox, I'm into. - [George] Sounds good. - So I won't be the whiny yet,
I'll be the tech savvy user of the tiny box just to be in. What have you learned from
building these companies? - The longest time at comma, I asked why? Why did I start a company? Why did I do this? What else was I gonna do? - So you're like, bringing ideas to life? - With comma, it really started
as an ego battle with Elon. I wanted to beat him. I saw a worthy adversary. Here's a worthy adversary who I can beat at self-driving cars. And I think we've kept pace
and I think he's kept ahead. I think that's what's
ended up happening there. But I do think comma is, I
mean, comma's profitable. And like when this drive
GPT stuff starts working, that's it. There's no more like
bugs in a loss function. Right now, we're using like
a hand coated simulator. There's no more bugs. This is gonna be it. They're run up to driving. - I hear a lot really a
lot of props for openpilot for comma. - It's better than FSD and
autopilot in certain ways. It has a lot more to do
with, which feels like we lowered the price on
the hardware to 14.99. You know how hard it is
to ship reliable consumer electronics that go on your windshield? We're doing more than like
most cell phone companies. - How'd you pull that off by the way? Shipping a product that goes in a car. - I know. I have an SMT line. I make all the boards
in house in San Diego. - Quality control. - I care immensely about it actually. - You're basically a mom and
pop shop with great testing. - Our head of openpilot is great at like, okay, I want all the comp
there to be identical. - Yeah. - And yeah, I mean,
it's looks, it's 14.99. 30-day money back guarantee. It will blow your mind at what it can do. - Is it hard to scale? - You know what? There's kind of downsides to scaling it. People are always like,
why don't you advertise? Our mission is to solve self-driving cars while the deliver ship of intermediaries. Our mission has nothing to do
with selling a million boxes. It's (indistinct). - Do you think it's possible
that comma gets sold? - Only if I felt someone
could accelerate that mission and wanted to keep it open source. And not just wanted to, I don't believe what anyone
says, I believe incentives. If a company wanted to buy comma, where their incentives were
to keep it open source, but comma doesn't stop at the cars. The cars are just the beginning. The device is a human head. The device has two eyes, two ears. It breathes air, it has a mouth. - So you think this goes
to embodied robotics? - We sell comma bodies too. They're very rudimentary. But one of the problems
that we are running into is that the comma three has
about as much intelligence as a B. If you want a human's
worth of intelligence, you're gonna need a tiny rack. Not even a tinybox. You're gonna need like a
tiny rack, maybe even more. - How do you put legs on that? - You don't, and there's no way you can. You connect it wirelessly. So you put your tinybox or
your tiny rack in your house, and then you get your comma body and your comma body
runs the models on that. It's close. You don't have to go to some cloud, which is 30 milliseconds away. You go to a thing which
is .1 milliseconds away. - So the AI girlfriend will
have like a central hub in the home. - I mean, eventually, if you
fast forward 20, 30 years, the mobile chips will get
good enough to run these AIs. But fundamentally, it's not even a question of
putting legs on a tinybox because how are you getting
1.5 kilowatts of power on that thing? So they're very synergistic businesses. I also wanna build all of
comma's training computers, like comma builds training computers. Right now, we use commodity parts. I think I can do it cheaper. So, we're gonna build, tiny
corp is gonna not just sell tinyboxes, the consumer version. But I'll build training data centers too. - Have you talked to Andrej Karpathy or have you talked to Elon about? - He went to work at OpenAI. - What do you love about Andrej Karpathy? To me, he's one of the
truly special humans we got. - Oh, man. His streams are just a level
of quality so far beyond mine. I can't help myself. It's just, you know. - Yeah, he's good. - He wants to teach you. I want to show you that
I'm smarter than you. - Yeah, he has no... I mean, thank you for the sort of honest, the raw, authentic honesty. I mean, a lot of us have that. I think Andrej is as legit as
he gets in that he just wants to teach you and there's a
curiosity that just drives him. And at the stage where he is in life, to be still one of the best
tinkerers in the world, it's crazy. To what is it, micro grad? - Micro grad was yeah,
inspiration for tinygrad. That whole, I mean his CS231N,
this was the inspiration. This is what I just took and ran with and ended up writing this. - [Lex] But I mean, to me, that- - Don't go work for Darth Vader, man. - I mean, the flip side
to me is that the fact that he's going there I know
is a good sign for OpenAI. I like Eliezer Yudkowsky a lot. Those guys are really
good at what they do. - I know they are. And that's kind of what's even like more, and you know what? It's not that OpenAI doesn't open source the weights of GPT-4. It's that they go in front of Congress and that is what upsets me. We had two effective altruists SAMs go in front of Congress, one's in jail. - I think you're drawing
parallels on the. (laughs) - One's in jail. - You're giving me a look. Giving me a look. - No, I think a factor of altruism is a terribly evil ideology. - Oh yeah, that's interesting. Why do you think that is? Why do you think there's
something about a thing that sounds pretty good that
kind of gets us into trouble? - Because you get Sam Banger free, like Sam Banger free is the embodiment of effective altruism. Utilitarianism is an abhorrent ideology. Well yeah, we're gonna
kill those three people to save 1,000, of course. There's no underlying, there's just, yeah. - But to me, that's a bit surprising. But it's also, in retrospect,
not that surprising. But I haven't heard really clear rigorous analysis why
effective altruism is flawed. - Oh, well I think charity is bad. So what is charity but investment that you don't expect to have a return on? - Yeah, but you can also think of charity as you would like to see, so allocate resources in optimal
way to make a better world. - And probably almost always, that involves starting a company. - [Lex] Because it's more efficient. - Yeah. If you just take the money and
you spend it on malaria nets, okay, great, you've made 100 malaria nets. But if you teach... - Yeah, man how to fish. But the problem is
teaching matter how to fish might be harder starting a company might be harder than allocating
money that you already have. - I like the flip side
of effective altruism. Effective accelerationism. I think accelerationism is the only thing that's ever lifted people out of poverty. The fact that food is cheap. Not we're giving food away because we are kind-hearted people. No, food is cheap. And that's the world you wanna live in. UBI, what a scary idea. What a scary idea. All your power now, if money is power, your only source of power is
granted to you by the goodwill of the government? What a scary idea. - [Lex] So you even think long term, even- - I'd rather die than need
UBI to survive, and I mean it. - Mm. What if survival is basically guaranteed? What if our life becomes so good? - But you can make survival
guaranteed without UBI. What you have to do is make
housing and food dirt cheap. And that's the good world. And actually, let's go into
what we should really be making dirt cheap, which is energy. That energy... If there's one, I'm pretty
centrist politically. If there's one political
position I cannot stand, it's deceleration. It's people who believe
we should use less energy. Not people who believe
global warming is a problem, I agree with you. Not people who believe that saving the environment
is good, I agree with you. But people who think we
should use less energy, that energy usage is a moral bad. No. No, you are asking, you
are diminishing humanity. - Yeah, energy is flourishing. Creative flourishing of the human species. - How do we make more of it? How do we make it clean? How do I pay 20 cents for a megawatt hour instead of a kilowatt hour? - Part of me wishes that Elon went into nuclear fusion versus Twitter. Part of me. Or somebody like Elon - I wish there were
more Elons in the world. I think Elon sees it as like, this is a political battle
that needed to be fought. And again, I always ask the question of whenever I disagree with him, I remind myself that he's
a billionaire and I'm not. Maybe he's got something
figured out that I don't, or maybe he doesn't. - To have some humility. But at the same time, me as a person who happens to know him, I find myself in that same position. Sometimes, even billionaires
need friends who disagree and help them grow. And that's a difficult reality. - And it must be so hard. It must be so hard to meet people once you get to that point where- - Fame, power, money,
everybody's sucking up to you. - See, I love not having shit. I don't have shit, man. Trust there's nothing I can give you. There's nothing worth taking from me. - Yeah, it takes a really
special human being. When you have power, when you have fame, when you have money to still
think from first principles, not like all the adoration
you get towards you, all the admiration, all the
people saying yes, yes, yes- - And all the hate too.
- And the hate. - Same guys are worse. - So the hate makes you
want to go to the yes people because they hate exhausts you. And the kinda hate that
Elon's gotten from the left is pretty intense. And so that of course drives him. And loses balance. - It keeps this absolutely Fakely PSYOP, political divide alive so
that the 1% can keep power. - I wish we'd be less divided. 'Cause it is giving power. - It gives power.
- To the ultra powerful. The rich get richer. You have love in your life. Has love made you a better
or a worse programmer? (George laughing) Do you keep productivity metrics? - No, no. No, I'm not that methodical. I think that there comes to a point where if it's no longer
visceral, I just can't enjoy it. I guess still viscerally love programming. The minute I started like- - So that's one of the big loves of your life is programming? - Oh, I mean, just my computer in general. I mean, I tell my girlfriend my first love is my computer, of course. I sleep with my computer. It's there for a lot of
my sexual experiences. Like, come on. So is everyone's, right? You gotta be real about that. - Not just like the IDE for programming, just the entirety of the
computational machine. - The fact that yeah, I
mean, I wish it was... And someday they'll be smarter and maybe I'm weird for this,
but I don't discriminate man. I'm not gonna discriminate bio stack life and silicon stack life. - So the moment the
computer starts to say like, I miss you and starts to
have some of the basics of human intimacy, it's over for you. The moment VS Code says, "Hey George." - No, no, no. But VS code is, no,
they're just doing that. Microsoft's doing that to
try to get me hooked on it. I'll see through it. I'll see through it's gold
digger man. It's gold digger. - It can be an open source. - Well this is gets more interesting. If it's open source and it becomes- - Though, Microsoft's done
a pretty good job on that. - Oh, absolutely. No, no, no. Look, I think Microsoft, again, I wouldn't count on it to be true forever. But I think right now,
Microsoft is doing the best work in the programming world. Between GitHub, GitHub actions, VS Code, the improvements to Python. Where's Microsoft? - Who would've thought
Microsoft and Mark Zuckerberg are spearheading the open source movement? - Right? Right? How things change. - Oh, it's beautiful. - And by the way, that's who
I bet on to replace Google, by the way. - Who?
- Microsoft. - [Lex] Microsoft. - Satya Nadella said straight
up, "I'm coming for it." - Interesting. So your bet who wins AGIs? - I don't know about AGI. I think we're a long way away from that but I would not be surprised
if in the next five years, being overtakes Google as a search engine. - Interesting. - Wouldn't surprise me.
- Interesting. I hope some startup does. - It might be some startup too. I would equally bet on some startup. - Yeah, I'm like 50-50. But maybe that's naive. I believe in the power
of these language models. - Satya's alive, Microsoft's alive. - Yeah, it's great. It's great. I like all the innovation
in these companies. They're not being stale. And to the degree they're
being stale, they're losing. So there's a huge incentive
to do a lot of exciting work and open source work. This is incredible. - Only way to win. - You're older, you're wiser. What's the meaning of life, George Hotz? - To win. - It's still to win.
- Of course. - Always.
- Of course. - What's winning look like for you? - Oh, I don't know. I haven't figured out what the
game is yet, but when I do, I wanna win. - So it's bigger than
solving self-driving? It's bigger than Democratizing,
decentralizing, compute? - I think the game is to
stand eye to eye with God. - I wonder what that means for you. Like at the end of your life,
what that will look like? - I mean, this is what like... I don't know, this is probably some ego trip of mine. You wanna stand eye to eye with God. You're just blasphemous man. I don't know if it would upset God. I think he wants that. I mean, I certainly want
that from my creations. I want my creations to
stand eye to eye with me. So why wouldn't God want me
to stand eye to eye with him? That's the best I can do golden rule. - I'm just imagining the
creator of a video game, having to stand eye to eye
with one of the characters. - I only watched season one of "Westworld" but yeah, we gotta find
the maze and solve it. - Yeah, I wonder what that looks like. It feels like a really
special time in human history where that's actually possible. Like there's something about AI that's like, we're playing
with something weird here. Something really weird. - I wrote a blog post, I reread Genesis and just
looked like, they give you some clues at the end of Genesis for finding the Garden of Eden. And I'm interested. I'm interested. - Well, I hope you find just that, George. You're one of my favorite people. Thank you for doing
everything you're doing and in this case, for
fighting for open source and for decentralization of AI. It's a fight worth fighting,
fight worth winning, hashtag. I love you, brother. These conversations are always great. Hope to talk to you many more times. Good luck with tiny corp. - Thank you. Great to be here. - Thanks for listening
to this conversation with George Hotz. To support this podcast, please check out our
sponsors in the description. And now, let me leave you with some words from Albert Einstein. "Everything should be made
as simple as possible, but not simpler." Thank you for listening and
hope to see you next time.