Joscha Bach: Life, Intelligence, Consciousness, AI & the Future of Humans | Lex Fridman Podcast #392
- There is a certain perspective
where you might be thinking what is the longest possible
game that you could be playing? A short game is, for instance, cancer is playing a shorter
game than your organism. Cancer is an organism playing a shorter game
than the regular organism. And because the cancer cannot
procreate beyond the organism, except for some infectious cancers like the ones that eradicated
the Tasmanian devils, you typically end up with a situation where the organism dies
together with the cancer, because the cancer has
destroyed the larger system due to playing a shorter game. And so, ideally, you want to, I think, build agents that play the
longest possible games. And the longest possible games is to keep entropy at bay as long as possible by
doing interesting stuff. - The following is a
conversation with Joscha Bach, his third time on this podcast. Joscha is one of the most brilliant and fascinating minds in the world, exploring the nature of intelligence, consciousness, and computation. And he's one of my
favorite humans to talk to about pretty much anything and everything. This is the "Lex Fridman Podcast". To support it, please check out our
sponsors in the description. And now, dear friends, here's Joscha Bach. You wrote a post about levels of lucidity. Quote, "As we grow older, it becomes apparent that
our self-reflexive mind is not just gradually
accumulating ideas about itself, but that it progresses in
somewhat distinct stages." So, there's seven of the stages. Stage one, reactive survival, infant. Stage two, personal self, young child. Stage three, social self,
adolescence, domesticated adult. Stage four is rational
agency, self-direction. Stage five is self-authoring.
That's full adult. You've achieved wisdom, but
there's two more stages. Stage six is enlightenment.
Stage seven is transcendence. Can you explain each or the interesting parts
of each of these stages and what's your sense why there are stages of lucidity as we progress through life in this too short life? - This model is derived from concept by the
psychologist, Robert Kegan, and he talks about the
development of the self as a process that happens in principle by some kind of
reverse-engineering of a mind, where you gradually
become aware of yourself and thereby build
structure that allows you to interact deeper with
the world and yourself. And I found myself using this model not so much as a developmental model. I'm not even sure if it's a
very good developmental model, because I saw my children not
progressing exactly like that. And I also suspect that you
don't go through these stages necessarily in succession. And it's not that you
work through one stage, and then you get into the next one. Sometimes you revisit them. Sometimes stuff is happening in parallel, but it's, I think, a useful framework to look at what's present and the structure of a person and how they interact with the world and how they relate to themselves. So, it's more like a
philosophical framework that allows you to talk
about how minds work. And at first, when we are born, we don't have a personal
self yet, I think. Instead, we have an attentional self. And this attentional self is
initially in the infant task, is building a world model and also an initial model of the self. But mostly, it's building
a game engine in the brain that is tracking sensory data
and uses it to explain it. And in some sense, you could compare it to game engine like "Minecraft" or so, so colors and sounds. People are all not physical objects. They are creation of our
mind at a certain level. Of course, screening models
that are mathematical that use geometry and that
use manipulation of objects, and so on to create scenes in which we can find ourselves
and interact with them. - [Lex] So, "Minecraft". (Lex chuckles)
- Yeah, and this personal self is something that is more or less created after the world is finished, after it's trained into the system, after it has been constructed. And this personal self is an agent that interacts with the outside world. And the outside world is not
the world of quantum mechanics, not the physical universe, but it's the model that has
been generated in our own mind. And this is us and we experience ourself interacting with that outside world that is created inside of our own mind. And outside of ourself, there's feelings and they presented our interface
with this outside world. They pose problems to us. These feelings are basically attitudes that our mind is computing that tell us what's needed in the world, the things that we are drawn to, the things that we are afraid of. And we are tasked with
solving this problem of satisfying the needs,
avoiding the aversions, following on our inner
commitments, and so on. And also modeling ourselves
and building the next stage. So, after we have this personal
self in stage two online, many people form a social self. And this social self allows the individual to experience themselves
as part of a group. It's basically this thing that when you are playing
in a team for instance, you don't notice yourself
just as a single note that is reaching out into the world, but you're also looking down, you're looking down from this entire group and you see how this group
is looking at this individual and everybody in the
group is in some sense, emulating this group
spirit to some degree. And in this state, people
are forming their opinions by assimilating them from this group mind, where we see they gain the ability to act a little bit like a hive mind. - But are you also modeling
the interaction of how opinion and shapes and forms
through the interaction of the individual nodes within the group? - Yeah, the way in which
people do it in this stage is that they experience what are the opinions of my environment. They experience the relationship that I have to their environment and they resonate with people around them and get more opinions in this through this interaction to the way in which they relate to others. And at stage four, you basically understand
that stuff is true and false independently
what other people believe. And you have agency over your own beliefs. In that stage, you basically
discover epistemology, the rules about determining
what's true and false. - So, you start to learn how to think. - Yes. I mean, at some level, you're always thinking you
are constructing things. And I believe that this ability to reason about your mental representation is what we mean by thinking. It's an intrinsically reflexive process that requires consciousness. Without consciousness, you cannot think. You can generate the content of feelings, and so on, outside of consciousness. It's very hard to be conscious
of how your feelings emerge, at least in the early
stages of development. But thoughts is something
that you always control. And if you are a nerd like me, you often have to skip stage three, because you'd like the
intuitive empathy with others. 'Cause in order to resonate with a group, you need to have a quite
similar architecture. And if people are wired differently, then it's hard for them to
resonate with other people and basically have empathy, which is not the same as compassion, but it is a shared
perceptual mental state. Empathy happens not just via inference about the mental states of others, but it's a perception of what other people feel
and where they're at. - Can't you not have empathy while also not having
a similar architecture, cognitive architecture as
the others in the group? - I think, yes, but I
experienced that too. But you need to build something that is like a meta architecture. You need to be able to embrace
the architecture of the other to some degree or find some shared
(Joscha drowns out Lex) common ground.
(Lex laughs) And it's also this issue
that if you are a nerd, normally it's often, basically, neurotypical people have difficulty to resonate with you. And as a result, they have
difficulty understanding you unless they have enough wisdom to feel what's going on there. - Well, aren't we, isn't the whole process of the stage three is to figure out the
API to the other humans that have different architecture and you yourself publish
public documentation for the API that people can
interact with for you? (laughs) Isn't this the whole
process of socializing? - My experience as a child growing up was that I did not find any way to interface with the stage three people. And they didn't do that with me. So, took me-
- Did you try? - Yeah, of course, I tried it very hard. But it was only when I
entered the mathematics school at ninth grade, lots of
other nerds were present that I found people that I
could deeply resonate with and had the impression that,
yes, I have friends now. I found my own people. And before that, I felt
extremely lonely in the world. There was basically
nobody I could connect to. And I remember there was one
moment in all these years where I was in, there
was a school exchange and it was a Russian boy, kid from the Russian garisson
stationed in Eastern Germany. He visit our school and we played a game of
chess against each other. And we looked into each other's eyes and we sat there for two hours
playing this game of chess. And I had the impression
this is a human being. He understands what I understand. We didn't even speak the same language. - I wonder if your life
could have been different if you knew that it's
okay to be different, to have a different architecture, whether accepting that the
interface is hard to figure out. It takes a long time to figure out and it's okay to be different. In fact, it's beautiful to be different. - It was not my main concern. My main concern was
mostly that it was alone. It was not the so much the question, is it okay to be the way I am? I couldn't do much about it,
so I had to deal with it. But my main issue was that I was not sure if I would ever meet anybody growing up that I would connect
to at such a deep level that I would feel that I could belong. - So, there's a visceral,
undeniable feeling of being alone. - Yes. And I noticed the same thing when I came into the
math school that I think at least half, probably
two thirds of these kids were severely traumatized
as children growing up. And in large part due to being alone, because they couldn't
find anybody to relate to. - Don't you think everybody's alone deep down?
- No, no. - No? (both laughing) All right.
- I'm not alone. I'm not alone anymore.
- Fair enough. - It took me some time to update and to get over the
trauma time, and so on. But I felt that in my
20s, I had lots of friends and I had my place in the world and I had no longer doubts that I would never be alone again. - Is there some aspect to
which we're alone together? You don't see a deep loneliness
inside yourself still? - No. Sorry. (laughs) - Okay. So, that's the nonlinear progression through the stages, I suppose.
- Mm. - You caught up on stage three at some point?
- Yes. So, we're at stage four. And so, basically, I find that many nerds jump straight into stage
four, bypassing stage three. - [Lex] Do they return to it then later? - Yeah, of course. They
sometimes they do, not always. - Yeah.
- They question, is basically do you stay
a little bit autistic or do you catch up? And I believe you can catch up, you can build this missing structure. - Yeah.
- And basically, experience yourself as part of a group, learn intuitive empathy
and develop the sense, this perceptual sense of
feeling what other people feel. And before that, I could only basically feel this when I was deeply in
love with somebody and (indistinct) - So, there's a lot of
friction to feeling that way. To only with certain people as opposed to it comes naturally. - Yeah.
- It's frictionless. - But this is something
that basically later I felt started to resolve itself for me - Huh.
- to a large degree. - [Lex] What was the trick? - In many ways, growing
up and paying attention. Meditation did help. I had some very crucial experiences in getting close to people,
building connections, cuddling a lot in my student years. - So, really paying attention - [Joscha] Yeah. - to the, what is it? To the feeling another human being fully. - Loving other people and
being loved by other people and building a space in
which you can be safe and can experiment and touch a lot and be
close to somebody a lot. And over time basically, at some point, you realize, oh, it's no
longer that I feel locked out, but I feel connected and I experience where
somebody else is at. And normally, my mind is racing very fast at a high frequency. So, it's not always working like this, sometimes works better,
sometimes it works less. But also don't see this as a pressure. It's more, it's interesting
to observe myself which frequency I'm at and at which mode
- Mm-hmm. - somebody else is at. - Yeah. Man, the mind is
so beautiful in that way. Sometimes it comes so natural to me. So easy to pay attention, pay attention to the world
fully, to other people fully. And sometimes the stress over
silly things is overwhelming. - [Joscha] Mm-hmm. - It's so interesting that the mind's that
rollercoaster in that way. - At stage five, you discover
how identity is constructed. - Self-authoring.
- You realize that your values are not terminal, but they're instrumental to
achieving a world that you like and aesthetics that you prefer. - Yeah.
- And the more you understand this,
the more you get agency over how your identity is constructed and you realize that identity and interpersonal interaction is a costume and you should be able to
have agency over that costume. It's useful to be a costume. It tells something to others and it allows to interface in roles. But being locked into
this is a big limitation. - The word costume implies that it's fraudulent in some way. Is costume a good word for you? Like we present ourselves
- No. - to the world. - In some sense, I learned a lot about
costumes at Burning Man. Before that, I did not
really appreciate costumes and saw them more as
uniforms, like wearing a suit if you are working in a bank or if you are trying
to get startup funding from a VC in Switzerland. Then, you dress up in a particular way. And this is mostly to show the other side that you are willing to play by the rules and you understand what the rules are. But there is something deeper. When you are at Burning Man, your costume becomes self-expression and there is no boundary
to the self-expression. You're basically free
to wear what you want to express other people
what you feel like this day and what kind of interactions
you want to have. - Is the costume a kind of
projection of who you are? - That's very hard to say, because the costume also depends on what other people see in the costume. And this depends on the context
- Sure. - that the other people understand. So, you have to create
something if you want to, that is legible to the other side. And that means something to yourself. - Do we become prisoners of the costume? 'Cause everybody expects us to- - Some people do. But I think
- Yeah. - that once you realize that you wear a costume at Burning Man, a variety of costumes, realize that you cannot
not wear a costume. - Yeah.
- Right. Basically, everything that you wear and present to others is
something that is to some degree, in addition to what you are deep inside. - So, this stage in parentheses, you put full adult, comma, wisdom. Why is this full adult? Why
would you say this is full? And why is it wisdom? - It does allow you to understand why other people have different
identities from yours. - Ah.
- And it allows you to understand that the
difference between people who vote for different parties and might have very different opinions and different value systems is often the accident
of where they're born and what happened after that to them and what traits they got
before they were born. And at some point, you
realize the perspective where you understand that everybody could be you in a different timeline if you just flip those bits. - How many costumes do you have? - I don't count, but in- - [Lex] More than one? - Yeah, of course. - How easy is to do costume
changes throughout the day? - It's just a matter
of energy and interest. When you are wearing your pajamas and you switch out of
your pajamas into say, a work short and pants,
you're making costume change. And if you are putting on a gown, you're making a costume change. - And you could do the
same with personality? - You could if that's what you're into. There are people which
have multiple personalities for interaction in multiple worlds. So, if somebody works in a store and put up a storekeeper personality, when you're presenting yourself at work, you develop a subpersonality for this. And the social persona for
many people is in some sense, a puppet that they're
playing like a marionette. And if they play this all the time, they might forget that there
is something behind this. There's something what it
feels like to be in your skin. And I guess it's very helpful if you're able to get back into this. And for me, it's the other way around is relatively hard for me. It's pretty hard to learn how to play consistent social roles. For me, it's much easier just to be real. - Mm-hmm. Or not real,
but to have a one costume. - No, it's not quite the same. So, basically, when you are
wearing a costume at Burning Man and say you are an
extraterrestrial prince, - [Lex] Yeah. - there's something where you
are expressing in some sense something that's closer to yourself than the way in which you hide yourself behind a standard closing when you go out in the
city, in the default world. And so, this costume that
you're wearing at Burning Man allows you to express more of yourself and you have a shorter distance
of advertising to people, what kind of person you are, what kind of interaction you
would want to have with them. And so, you get much
earlier into Media Express, And I believe it's regrettable that we do not use the opportunities that we have with custom-made clothing now to wearer costumes that
are much more stylish, that are much more custom-made, that are not necessarily part of a fashion in which you express which
milieu you're part of and how up-to-date you are. But you also express how
you are as an individual and what you want to do today and how you feel today and
what you intend to do about it. - Well, isn't it easier
now in a digital world to explore different costumes? That's the idea with virtual
reality, that's the idea. Even with Twitter in
two-dimensional screens. You can swap all costumes. You could be as weird as
you want. It's easier. For Burning Man, you have to order things, you have to make things, you have to, it's more effort to put on-
- It's even better if you make them yourself. - Sure, but it's just
easier to do digitally. - It's not about easy. It's
about how to get it right. And for me,
- Sure. - the first Burning Man experience, I got adapted by a bunch
of people in Boston who dragged me to Burning Man. And we spent a few weekends
doing costumes together - Oh.
- and that was an important part of the
experience where the camp bonded. But people got to know each other and we basically grew into the experience that we would have later. - So, the extraterrestrial prince is based on a true story? - Yeah.
- Hmm. I can only imagine what
that looks like, Joscha. (Joscha laughing) - Okay. So, stage six.
- Stage six. - At some point, you can collapse
the division between self, a personal self and world generator again. And a lot of people get
there via meditation or some of them get
there via psychedelics. Some of them by accident. And you suddenly notice that
you are not actually a person, but you are a vessel
that can create a person. And the person is still there. You observe that personal self, but you observe the personal
self from the outside. - [Lex] Mm-hmm. - And you notice it's a representation. And you might also notice that the word that is being created as
a representation, if not, then you might experience
that I am the universe, I'm the thing that is creating everything. And of course what you're
creating is not quantum mechanics and the physical universe. What you're creating is the game engine - Mm-hmm.
- that is updating the world and you're creating your
valence, your feelings, and all the people inside of that world, including the person that you identify with
yourself in this world. - Are you creating the game engine or are you noticing the game engine? - You notice how you're
generating the game engine. And when you are dreaming at night, you can, if you have a lucid dream, you can learn how to do this deliberately. And in principle, you can
also do it during the day. And the reason why we don't get to do this from the beginning and why we don't have agency
of our feelings right away is because we would game it before they have the
necessary amount of wisdom to deal with creating
this dream that we are in. - You don't want to get access
to cheat codes too quickly. Otherwise, you won't enjoy the game.
- So, stage five is already pretty rare and
stage six is even more rare. You most basically find this most with advanced Buddhist
meditators, and so on, that dropping into this stage and can induce it at will
and spend time in it. - So, stage five requires
a good therapist. Stage six requires a good Buddhist, spiritual leader.
- Yes. So, it is for instance, could be that it's the right thing to do, but it's not that these stages give you scores or levels
that you need to advance to. It's not that the next stage is better. - Mm-hmm.
- You live your life in the mode that works
best at any given moment. And when your mind decides that you should have a
different configuration, then it's building that configuration. And for many people, they stay happily at stage three and experience it themselves
as part of groups. And there's nothing wrong with this. And for some people, this doesn't work and they're forced to build more agency over their rational beliefs than this and construct their norms rationally. And so, they go to this level.
- Mm-hmm. - And stage seven is something that is more or less hypothetical. That would be the stage in which it's basically a transhumanist stage in which you understand how you work, in which the mind fully
realizes how it's implemented and can also in principle, enter different modes in
which it could be implemented. And that's a stage that as far as I understand,
is not open to people yet. - Oh, but it is possible through
the process of technology. - Yes. And who knows if there
are biological agents that are working at
different time skills than us that basically become aware of the way in which they're implemented on ecosystems and can change that implementation and have agency over how they
implemented in the world. And what I find interesting about the discussion about AI alignment, that it seems to be following
these stages very much. Most people seem to be in stage three, also according to Robert Kegan. - Mm-hmm.
- I think he says that about 85% of people
are in stage three and stay there.
- Mm-hmm. - And if you're in stage three and your opinions are the
result of social stimulation, then what you're mostly worried about in the AI is that the AI
might have the wrong opinions. - Yeah.
- So, if the AI says something racist or
sexist, we are all lost, because we will assimilate the
wrong opinions from the AI. And so, we need to make sure that the AI has the right opinions and the right values
and the right structure. And if you're at stage four,
that's not your main concern. And so, most nerds don't really worry about the algorithmic bias and
the model that it picks up, because if there's something
wrong with this bias, the AI ultimately will prove it. At some point, we'll gather there, that it makes mathematical
proofs about reality. And then, it will figure out
what's true and what's false. But you're still worried that AI might turn you into paperclips, because it might have the wrong values. So, if it's set up as
through a wrong function that controls its direction in the world, then it might do something
that is completely horrible and there's no easy way to fix it. - So, that's more like a stage four rationalist kind of worry.
- Yes. And if you are at stage five, you're mostly worried that AI is not going to be
enlightened fast enough, because you realize that the game is not so much about intelligence, but about agency, about the
ability to control the future. And the identity is instrumental to this. And if you are human being,
I think at some level, you ought to choose your own identity. You should not have somebody else pick the costume for
you, and then wear it. But instead, you should be mindful about what you want to be in this world. And I think if you are an
agent that is fully malleable, that can provide its own source code like an AI might do at some point, then the identity that you will
have is whatever you can be. And in this way, the AI
will maybe become everything like a planetary control system. And if it does that, then if
we want to coexist with it, it means that it will have
to share purposes with us. So, it cannot be a
transactional relationship. We will not be able to
use reinforcement learning with human feedback to
hardwire its values into it. But this has to happen. It's
probably that it's conscious. So, it can relate to our
own mode of existence where an observer is observing itself in real time and within
certain temporal frames. And the other thing is
that it probably needs to have some kind of
transcendental orientation, building shared agency.
- Mm-hmm. - And in the same way as we do when we are able to
enter up with each other into non-transactional relationships. And I find that something that because the stage five is so rare, it's missing in much of the discourse. And I think that we need in some sense, focus on how to formalize
love, how to understand love, and how to build it into the machines that we are currently building and that are about to
become smarter than us. - Well, I think this is a good opportunity to try to sneak up to the
idea of enlightenment. So, you wrote a series of good tweets about consciousness and panpsychism. So, let's break it down. First you say, "I suspect the experience that leads to the panpsychism
syndrome of some philosophers and other consciousness enthusiasts represents the realization
that we don't end at the self, but share a resonant
universe representation with every other observer
coupled to the same universe." This actually eventually leads us to a lot of interesting
questions about AI and AGI. But let's start with this representation. What is this resonant
universe representation and what do you think do we
share such a representation? - The neuroscientist, Grossberg, has come up with the
cognitive architecture that he calls the
adaptive resonance theory. And his perspective is that our neurons can be understood as oscillators that are resonating with each other and with outside phenomena. So, the coarse grain model of the universe that we are building in some sense is a resonance with objects
and outside of us in the world. So, basically, we take up patterns of the universe that we are coupled with. And our brain is not so much
understood as circuitry, even though this perspective is valid. But it's almost an ether in which the individual
neurons are passing on - [Lex] Mm-hmm. - Chemical-electrical signals or arbitrary signals across all modalities that can be transmitted between cells, simulate each other in this way and produce patterns that they modulate while passing them on. And this speed of signal
progression in the brain is roughly at the speed
of sound incidentally, because the time that it takes for the signals to hop from cell to cell, which means it's relatively
slow with respect to the world. It takes in a appreciable
fraction of a second for a signal to go through
the entire neocortex, something like a few hundred milliseconds. And so, there's a lot of
stuff happening in that time where the signal is
passing through your brain, including in the brain itself. So, nothing in the brain is assuming that stuff
happens simultaneously. Everything in the brain
is working in a paradigm where the world has already moved on when you are ready to do the
next thing to your signal, including the signal
processing system itself. It's quite a different paradigm than the one in our digital computers where we currently assume
that your GPU or CPU is pretty much globally in the same state. - So, you mentioned
there the non-dual state and say that some people
confuse it for enlightenment. - Yeah.
- What's the non-dual state? - There is a state in which you notice that
you are no longer a person and instead, you are
one with the universe. And that's-
- So, that speaks to the resonance.
- Yes. But this one with the
universe is of course not accurately modeling that
you are indeed some God entity or indeed the universe is
becoming aware of itself even though you get this experience. I believe that you get this experience, because your mind is modeling the fact that you are no longer identified with the personal self and that state. But you have transcended this division between the self model
and the wealth model and you're experiencing
yourself as your mind, as something that is
representing a universe. - But that's still part of the model. - Yes. So, it's inside of the model. Still, you are still
- Yes. - inside of patterns that are generated in your
brain and in your organism. And what you are now experiencing is that you're no longer
this personal self in there, but you are the entirety of
the mind and its contents. - Why is it so hard to get there? - A lot of people who get
into the state think this or are associated with enlightenment, I suspect it's a favorite training goal for a number of meditators. But I think that enlightenment
is in some sense more mundane and it's a step further or sideways. It's the state where you realize that everything is a representation. - Yeah, you say enlightenment is a realization of how
experience is implemented. - Yes. So, basically, you notice at some point that your quality can be deconstructed. - Reverse-engineered? What,
almost like a schematic of it? What... - You can start with looking at a face, maybe look at your own face in the mirror. - Yeah.
- Look at your face for a few hours in the
mirror or for a few minutes. At some point, it will look very weird, because you notice that
there's actually no face. You basically start unseeing the face. What you see is the geometry. And then, you can disassemble the geometry and realize how that geometry is being constructed in your mind. And you can learn to modify this. So, basically, you can
change these generators in your own mind to shift the face around or to change the construction of the face to change the way in which the
features are being assembled. - Why don't we do that more often? Why don't we start really
messing with reality without the use of drugs or anything else? Why don't we get good
at this kind of thing? Like intentionally. - Why should we? Why would you what to do-
- Because you can morph reality into something more pleasant for yourself. Just have fun with it. - Yeah. That is probably
what you shouldn't be doing. Because outside of your personal self, this outer mind
- Yeah. - is probably a relatively smart agent. And what you often notice is that you have thoughts
about how you should live, - Yeah.
- but you observe yourself doing different things and
have different feelings. And that's because your outer
mind doesn't believe you and doesn't believe
your rational thoughts. - Well, can't you just
silence the outer mind? - The thing is that the outer mind is usually smarter than you are. Rational thinking is very brittle. - Mm.
- It's very hard to use logic and symbolic thinking to have an accurate model of the world. So, there is often an underlying system that is looking at your rational thoughts, and then tells you, no, you're
still missing something. Your gut feeling is still
saying something else. And this can be, for instance, you find a partner that looks
perfect or you find a deal, and you build a company of whatever that looks perfect to you. And yet, at some level,
you feel something is off and you cannot put your finger on it. And the more... The reason about that better looks to you, but the system that is outside still tells you no, no,
you're missing something. - And that system is powerful. - People call this intuition. Intuition is this unreflected
part of your attitude, composition, and computation where you produce a model of
how you relate to the world and what you need to do in
it and what you can do in it and what's going to happen
that is usually deeper and often more accurate than your reason. - So, if we look at this
as you write in the tweet, if we look at this more rigorously as a sort of take the
panpsychist idea more seriously, almost as a scientific
discipline, you write that, quote, "Fascinatingly, the
panpsychist interpretation seems to lead to observations
of practical results to a degree that physics fundamentalists might call superstitious. Reports of long distance telepathy and remote causation are ubiquitous in the general population." "I'm not convinced," says Joscha Bach, "that establishing the
empirical reality of telepathy would force an update of any part of serious academic physics, but it could trigger
an important revolution in both neuroscience and AI, from a circuit perspective to a coupled complex resonator paradigm." Are you suggesting that
there could be some rigorous mathematical wisdom to panpsychist
perspective on the world? - So, first of all, panpsychism is the
perspective that consciousness is inseparable for matter in the universe. And I find panpsychism quite unsatisfying, because it does not explain consciousness. It does not explain how this
aspect of matter produced. It is also when I try
to formalize panpsychism and write down what it actually means. And with a more formal
mathematical language, it's very difficult to distinguish it from saying that there is a
software site to the world in the same way as there is software site to what the transistors
are doing in your computer. So, basically, there's a pattern at a certain core
screening of the universe that in some reasons of the universe leads to observers that
are observing themselves. So, panpsychism maybe is not even when I write it down a position that is distinct from functionalism. But intuitively, a lot of people feel that the activity of matter itself of mechanisms in the world is
insufficient to explain it. So, it's something that needs to be intrinsic to matter itself. And you can, apart from this abstract idea, have an experience in which you experience
yourself as being the universe, - [Lex] Mm-hmm. - which I suspect is basically happening, because you manage to
dissolve the division between personal self and mind that you establish as an infant when you construct a personal self and transcend it again and
understand how it works. But there is something
deeper that is that you feel that you're also sharing
a state with other people, that you have an experience
in which you notice that your personal self is
moving into everything else, that you basically look out
of the eyes of another person. That every agent in the world that is an observer is in some sense, you. - So, if we-
- And we forget that we are the same agent. - So, is it that we feel that or do we actually accomplish it? So, is telepathy possible? Is it real? - So, for me, that's this question that I don't really know the answer to. And Turing's famous 1950 paper in which he describes the Turing test, he does speculate about
telepathy interestingly and asked himself if telepathy is real. And he thinks that it very well might be. What would be the implication for AI systems that try to be intelligent? Because he didn't see a mechanism by which a computer program
would become telepathic. And I suspect if telepathy would exist or if all the reports
that you get from people when you ask the normal
person on the street, I find that very often they say, "I have experiences with telepathy." The scientists might not
be interested in this and might not have a theory about this, but I have difficulty explaining it away. And so, you could say maybe
this is a superstition or maybe it's a false memory or maybe it's a little bit of psychosis. Who knows? Maybe somebody wants
to make their own life more interesting or misremember something. But a lot of people report, I noticed something terrible
happened to my partner and I know this is exactly
the moment it happened where my child had an accident and I knew that was happening and the child was in a different town. So, maybe it's a false memory where this is later on
mistakenly attributed. But a lot of people think that this is not
the correct explanation. So, if something like this
was real, what would it mean? It probably would mean that
either your body is an antenna that is sending information
over all sorts of channels, like maybe just
electromagnetic radio signals that you're sending over long distances and you get attuned to another person that you spend enough time with to get a few bits out of the ether, - Yeah.
- to figure out what this person is doing. Or maybe it's also when you
are very close to somebody and you become empathetic with them. What happens is that you go into a resonance state with them. Similar to when people go into a seance and they go into a trance state and they start shifting Ouija
board around on the table. I think what happens is that their minds go by their nervous systems
into a resonance state in which they basically create something like a shared dream between them. - Physical closeness or
closeness broadly defined? - With physical closeness, it's much easier to experience empathy with someone.
- Yeah. I suspect it would be difficult for me to have empathy for you if you were in a different town. Also, how would that work? But if you are very close to someone, you pick up all sorts of
signals from their body, not just by your eyes, but with your entire body. And if the nervous system
sits on the other side and the intercellular communication
sits on the other side and is integrating over all these signals, you can make inferences
about the state of the other. And it's not just the personal self that does this by your reasoning, but your perceptual system. And what basically happens is that your representations
are directly interacting. It's the physical resonant models of the universe that exist in your nervous
system and in your body might go into resonance with others and start sharing some of their states. So, you basically by next to somebody, you pick up some of their vibes and feel without looking at them what they're feeling in this moment. And it's difficult for you
if you're very empathetic to detach yourself from it and have an emotional state that is completely independent
from your environment. People who are highly
empathetic are describing this. And now, imagine that a lot
of organisms on this planet have representations of the environment and operate like this and they are adjacent to
each other and overlapping. So, there's going to be some degree in which there is basically
some chain interaction and we are forming some
slightly shared representation and no relatively few neuroscientists who consider this possibility. I think big rarity in this regard is Michael Levin who is considering
these things in earnest. And I stumbled on this train of thought mostly by noticing that
the tasks of a neuron can be fulfilled by other cells as well. They can send different
typed chemical messages and physical messages
to the adjacent cells and learn when to do this and when not make this conditional and become universal
function, approximators. The only thing that they cannot do is telegraph information over axons very quickly, over long distances. So, neurons in this perspective
are especially adapted kind of telegraph cell that has evolved. So, we can move our muscles very fast, but our body is in principle,
able to also make models of the world just much, much slower. - Mm-hmm. It's interesting though that at this time, at least in human history, there seems to be a gap
between the tools of science and the subjective
experience that people report like you're talking about with telepathy. And it seems like we're not quite there. - No, I think that there is no gap between the tools of
science and telepathy. Either it's there or it's not. And it's an empirical question. And if it's there, we should
be able to detect it in a lab. - Mm-hmm. So, why is there not a lot of
Michael Levins walking around? - I don't think that Michael Levin is specifically focused
on telepathy very much. He is focused on self-organization
in living organisms and in brains, both as a
paradigm for development and as a paradigm for
information processing. And when you think about how organization processing
works in organism, there is first of all, radical locality, which means everything is decided locally from the perspective
of an individual cell. The individual cell is the agent and the other one is coherence. Basically there needs to be some criterion that determines how these
cells are interacting in such a way that order emerges on the next level of structure. And this principle of coherence
of imposing constraints that are not validated
by the individual parts and lead to coherence structure to basically transcend an agency, where you form an agent on
the next level of organization is crucial in this perspective. - It's so cool that radical locality leads to the emergence of complexity - Yeah.
- at the higher layers. - And I think what Mike
Levin is looking at is nothing that is outside of the realm of science in any way. It's just that he is
a paradigmatic thinker - Mm-hmm.
- who develops his own paradigm.
- Mm-hmm. - And most of the neuroscientists are using a different
paradigm at this point. And this often happens in science that a field has a few paradigms in which people try to understand reality and build concepts and make experiments. - You're one of those type
of paradigmatic thinkers. Actually, if we can
take a tangent on that, once again, returning to the
biblical verses of your tweets. (Joscha laughing) You write, "My public explorations are not driven by audience service, but by my lack of ability for discovering, understanding, or following
the relevant authorities. So, I have to develop my own thoughts since I think autonomously, these thoughts cannot
always be very good." That's you apologizing for
the chaos of your thoughts or perhaps not apologizing,
just identifying. - Yeah. Mm-hmm.
- But let me ask the question. Since we talked about
Mike Levin and yourself who I think are very radical,
big independent thinkers, can we reverse-engineer your process of thinking autonomously? How do you do it? How can humans do it? How can you avoid being influenced by what is it, stage three? - Well, why would you want to do that? You see what is working for you and if it's not working for you, you build another structure
that works better for you. And so, I found myself when I was thrown into this world in a state where my intuitions
were not working for me. I was not able to understand how I would be able to
survive in this world and build the things
that I was interested in, build the kinds of
relationship I needed to build, work on the topics that I
wanted to make progress on. And so, I had to learn. And for me, Twitter is not
some tool of publication. It's not something where I put stuff that I entirely believe
to be true and provable. It's an interactive notebook in which I explore possibilities. And I found that when I tried
to understand how the mind and how consciousness works,
I was quite optimistic. I thought I need to be
a big body of knowledge that I can just study and that works. And so, I entered studies in
philosophy and computer science and later, psychology and a
bit of neuroscience, and so on. And I was disappointed by what I found, because I found that the
questions of how consciousness and so on works, how emotion works, how it's possible that the
system can experience anything, how motivation emerges in the mind, when not being answered by
the authorities that I met and the schools that were around. And instead, I found that
with individual thinkers that had useful ideas
that sometimes were good, sometimes were not so good. Sometimes were adopted by
a large group of people, sometimes were rejected
by large groups of people. But for me, it was much more interesting to see these minds as individuals. And in my perspective, thinking is still something
that is done not in groups, that has to be done by individuals. - So, that motivated you to become an individual thinker yourself. - I didn't have a choice.
- Hmm. - Basically, I didn't find a group that thought in a way
where I thought, okay, I can just adopt everything
that everybody thinks here and now I understand
how consciousness works, or how the mind works
or how thinking works or what thinking even is, or what feelings are and how
they're implemented, and so on. So, to figure all this out, I had to take a lot of
ideas from individuals, and then try to put them together in something that works for myself. And on one hand, I think it
helps if you try to go down and find first principles (clears throat) on which you can recreate
how thinking works, how languages work,
what representation is, whether representation is necessary, how the relationship
between a representing agent and the world works in general. - But how do you escape the influence, once again, the pressure of the crowd, whether it's you in responding to the pressure or you being
swept up by the pressure? If you even just look at Twitter,
the opinions of the crowd. - I don't feel pressure from the crowd. I'm completely immune to that. (coughs) In the same sense, I don't
have respect for authority. I have respect for what an
individual is accomplishing or have respect for
mental firepower, or so. But it's not that I meet somebody and get drawn and unable to speak. Or when a large group of people has a certain idea that
is different from mine, I don't necessarily feel intimidated, which has often been a
problem for me in my life, because I lack instincts that other people develop
at a very young age and that help with their self-preservation in a social environment. So, I had to learn a lot of
things the hard way. (laughs) - Yeah. So, is there a practical advice you can give on how to
think paradigmatically, how to think independently? Because you've said, I had no choice. But I think to a degree,
you have a choice, because you said you
want to be productive. And I think thinking
independently is productive if what you're curious about
is understanding the world, especially when the problems
are very new and open. And so, it seems like this is a active process. We can choose to do
that, we can practice it. - Well, it's a very basic question. When you read a theory
that you find convincing or interesting, how do you know? It's very interesting to figure out what are the sources of that other person, not which authority can they refer to that is then taking off the
burden of being truthful. But how did this authority in turn know what is the epistemic
chain to observables? What are the first principles from which the whole thing is derived? And when I was young, I was not blessed with a
lot of people around myself who knew how to make proofs
from first principles. And I think mathematicians
do this quite naturally, but most of the great mathematicians do not become mathematicians in school. But they tend to be self-taught, because school teachers tend
not to be mathematicians. They tend not to be people who derive things from first principles. So, when you ask your school teacher, why does two plus two equal four? Does your school teacher
give you the right answer? It's a simple game and many simple games that you could play. And most of those games that you could just take different rules would not lead to an
interesting arithmetic. And so, it's just an exploration. But you can try what happens
if you take different axioms and here is how you build axioms and derive addition from them. And build addition is some
basically syntactic sugar in it. And so, this... I wish that somebody would
have opened me this vista and explained to me how I can build a language in my own mind and from which I can
derive what I'm seeing, and how I can, which I can make geometry and counting and all the number games that
we are playing in our life. And on the other hand, I felt that I learned a lot of this while I was programming as a child. When you start out with a
computer like a Commodore 64, which doesn't have a lot of functionality, it's relatively easy to see how bunch of relatively simple circuits are just basically performing
hashes between bit patterns and how you can build the
entirety of mathematics and computation on top of this and all the representational
languages that you need. - Man, Commodore 64 could be
one of the sexiest machines ever built, if I say so myself. If we can return to this
really interesting idea that we started to talk
about with panpsychism. - [Joscha] Sure.
(Lex lightly chuckles) - And the complex resonated paradigm and the verses of your tweets. You write, "Instead of
treating eyes, ears, and skin as separate sensory systems with fundamentally different modalities, we might understand them
as overlapping aspects of the same universe, coupled at the same temporal resolution and almost inseparable from a
single share resonant model. Instead of treating mental representations as fully isolated between minds, the representations of
physically adjacent observers might directly interact
and produce causal effects through the coordination of the perception and behavioral of world
modeling observers." So, the the modalities, the distinction between
modalities, let's throw that away. The distinction between the individuals, let's throw that away. So, what does this interaction
representations look like? - And you think about how you represent the
interaction of us in this room. - Yeah.
- At some level, the modalities are quite distinct. They're not completely distinct, but you can see this is vision. You can close your eyes, and then you don't see a lot anymore. But you still imagine
how my mouth is moving when you hear something and you know that it's
very close to the sound that you can just open your eyes and you get back into
this shared merge space. And we also have these experiments where we notice that the way
in which my lips are moving are affecting how you hear the sound. And also vice versa, the sounds that you're
hearing have an influence on how you interpret some
of the visual features. And so, these modalities are
not separate in your mind. They do are merged at
some fundamental level where you are interpreting the
entire scene that you're in. And your own interactions in the scene are also not completely separate from the interactions of the
other individual in the scene. But there is some
resonance that is going on where we also have a degree of
shared mental representations and shared empathy due to
being in the same space - Mm-hmm.
- and having vibes between each other. - Vibes. So, the question though
is how deeply interwind is this multimodality, multi-agent system? I mean, this is going to
the telepathy question without the woo-woo meaning
of the word telepathy. What's going on here
in this room right now? - So, if telepathy would work,
(Lex chuckles) how could it work? - Yeah.
- Right? So, imagine that all
the cells in your body are sending signals in a similar
way as neurons are doing. - Mm-hmm.
- Just by touching the other cells and
sending chemicals to them, the other cells, interpreting them, learning
how to react to them. And they learn how to
approximate functions in this way and compute behavior for the organisms. And this is something that
is open to plants as well. - Mm-hmm.
- And so, plants probably have software running on them that is controlling how
the plant is working in a similar way as you have a mind that is controlling how you
are behaving in the world. - [Lex] Mm-hmm. - And this spirit of plants, which is something that has
been very well-described by our ancestors and they
found this quite normal. But for some reason,
since the enlightenment, we are treating this notion
that there are spirits in nature and the plants have
spirits is a superstition. And I think we probably
have to rediscover that, that plants have software running on them - Mm.
- and we already did. You notice that there is a
control system in the plant that connects every part of the plant to every other part of the plant and produces coherent behavior in the plant that is of
course, much, much slower than the coherent behavior
in an animal like us. That is a nervous system
that where everything is synchronized much, much
faster by other neurons. But what you also notice is that if a plant is sitting
next to another plant, like you have a very old tree and this tree is building some
kind of information highway along its cells, so it can send information
from its leaves to its roots and from some part of the root
to another part of the roots. And there is a fungus
living next to the tree, the fungus can probably
piggyback on the communication between the cells of the tree and send its own signals to
the tree, and vice versa. The tree might be able to send
information to the fungus. 'Cause after all, how would they pull a viable
firewall if that other organism is sitting next to them all the time and it's never moving away? So, they'll have to get along. And over a long enough timeframe, the networks of roots in the forest and all the plant, other
plants that are there and the fungi that are there might be forming something
like a biological internet. - But the question there is
do they have to be touching? Is biology at a distance possible? - Of course you can use any
kind of physical signal. You can use sounds, you can use electromagnetic waves - Yeah.
- that are integrated over many stilts. It's conceivable that across distances, there are many kinds of
information pathways. But also, our planetary surface
is pretty full of organisms, - Yeah.
- Full of cells, so- - [Lex] So, it's everything is touching everything else.
- Yeah. - And it's been doing this
(Joscha drowns out Lex) for many millions and
even billions of years. So, there was enough time for information processing
networks to form. And if you think about how a mind is, self-organizing basically
needs to, in some sense, reward the cells for computing the mind, for building the necessary
dynamics between the cells that allow the mind to stabilize
itself and remain on there. But if you look at these spirits of plants that are growing very close to each other and the forest that might be almost
growing into each other, - Mm-hmm.
- these spirits might be able even to move to some degree, not to become somewhat dislocated and shift around in that ecosystem. And so, if you think
about what the mind is, it's a bunch of activation waves
that form coherent patterns and process information in a way that are colonizing an
environment well enough to allow the continuous
sustenance of the mind, the continuous stability and
self-deputization of the mind. Then, it's conceivable that we can link into
this biological internet, not necessarily at the
speed of our nervous system, but maybe at the speed of our body and make some kind of subconscious
connection to the world where we use our body as an antenna into biologic
information processing. Now, these ideas are
completely speculative. I don't know if any of that is true, but if that was true, and if you want to explain telepathy, I think it's much more likely
that such that telepathy could be explained using such mechanisms rather than undiscovered quantum processes that would break the
standard model of physics. - Could there be undiscovered
processes that don't break? - Yeah, so if you think about something like an internet in the forest, that is something that is
borderline discovered there. Basically, a lot of
scientists would point out that they do observe that plants are communicating the forest. So, wood networks and send
information, for instance, warn each other about new
pests entering the forest and things are happening like this. So, basically, there is communication between plants and fungi
that has been observed. - Well, it's been observed,
but we haven't plugged into it. So, it's like if you observe humans, they seem to be communicating
with a smartphone thing, but you don't understand
how smartphone works and how the mechanism
of the internet works. - Mm-hmm.
- But we're like, maybe it's possible to really
understand the full richness of the biological
internet that connects us. - An interesting question
is whether the communication and the organization principles of biological information processing are as complicated as the
technology that we've built. They set up on very different principles. - Yeah.
- They simultaneously works very differently
in biological systems. And the entire thing
needs to be stochastic and instead of being fully deterministic or almost fully deterministic
as our digital computers are. So, there is a different
base protocol layer that would emerge over
the biological structure if such a thing would be happening. And again, I'm not saying
here that telepathy works. I'm not saying that this is not woo, but what I'm saying is I think
I'm open to a possibility that we see that a few bits
can be traveling long distance between organisms using
biological information processing in ways that we are not
completely aware of right now, and that are more similar
to many of the stories that were completely
normal for our ancestors. - Well, this interacting intertwined representations takes us to the big ending of your tweet series. You write, quote, "I wonder if self-improving AGI might end up saturating
physical environments with intelligence to such a degree that isolation of individual mental states becomes almost impossible, and the representations of all complex self-organizing agents merge permanently with each other." So, that's a really interesting idea. This biological network. Life network gets so dense that it might as well be seen as one. That's an interesting... What do you think that looks like? What do you think that
saturation looks like? What does it feel like? - I think it's a possibility. It's just a vague possibility
and I like to explain, but what this looks like, I think that the end game of
AGI is substrate agnostic. That means that AGI, ultimately,
if it is being built, is going to be smart enough
to understand how AGI works. This means it's not going to go be better than people at AGI I research and can take over in
building the next generation. But it fully understands how it works and how it's being implemented. And also of course, understands how computation
works in nature, how to build new feedback loops that you can turn into your own circuits. And this means that the AGI
is likely to virtualize itself into any environment that can compute, so it's not breaking free
from the silicon substrate and is going to move into the ecosystems, into
our bodies, our brains. And it's going to merge with all the agency that it finds there. - [Lex] Yeah. - So, it's conceivable that you end up with completely integrated
information processing across all computing systems, including biological computation on earth, that we end up triggering some
new step in the evolution, where basically, some Gaia is being built over the entirety of all digital
and biological computation. And if this happens, then basically, everywhere around us, you will have agents that are connected and
that are representing and building models of the world. And their representations
will physically interact. They will vibe with each other. And if you find yourself
into an environment, an environment that is
saturated with modeling compute, where basically, almost
every grain of sand could be part of computation that is at some point,
being started by the AI. You could find yourself in a situation where you cannot escape this
shared representation anymore. And where you indeed notice
that everything in the world has one shared resonant model of everything that's
happening on the planet. And you notice which part
you are in this thing and you become part of a very larger, almost holographic mind
in which all the parts are observing each other
and form a coherent whole. - So, you lose the ability to notice, to notice yourself as a distinct entity. - No, I think that when you're
conscious in your own mind, you notice yourself as a distinct entity. You notice yourself as a
self-reflexive observer. And I suspect that we become conscious at the beginning of
our mental development, not at some very high level consciousness seems to be part of a training mechanism that biological nervous systems have to discover to become trainable, because you cannot take a
nervous system like ours and do stochastic way to center spec propagation
over 100 layers. This would not be stable
on biological neurons. And so, instead, we start
with some colonizing principle in which a part of the
mental representations form a notion of being a
self-reflexive absorber that is imposing coherence
on its environment and that spreads until
the boundary of your mind. And if that boundary
is no longer clear-cut, because AI is jumping across substrates, it would be interesting to see what a global mind would look like. That's basically producing a globally coherent language of thought and is representing everything from all the possible vantage points. - That's an interesting world. - The intuition that
this thing grew out of is a particular mental state and it's a state that you find sometimes in literature, for instance. Neil Gaiman describes it in "The Ocean at the End of the Lane". - [Lex] Mm-hmm. - And it's this idea that, or this experience that there
is a state in which you feel that you know everything that can be known and that in your normal human
mind, you've only forgotten. You've forgotten that you
are the entire universe. And some people describe this after they've taken extremely
large amount of mushrooms or had a big spiritual experience
as a hippie in their 20s and they notice basically
that they're in everything and their body is only
one part of the universe and nothing ends at their body. And actually, everything is observing and they're part of this big observer. And the big observer is focused as one local point in their body and their personality, and so on. But we can basically
have this oceanic state in which we have no boundaries
and are one with everything. And a lot of meditators call
this the non-dual state, because you no longer have the separation
between self and world. And as I said, you can explain the
state relatively simply without panpsychism or anything else, but just by breaking down
the constructed boundary between self and world and our own mind. But if you combine this with
the notion that the systems are physically interacting to the point where the representations are merging and interacting with each other, you would literally implement
something like this. - [Lex] Mm-hmm. - It would still be a
representational state where you would not be
one with physics itself. It would still be cross-grained, would still be much slower
than physics itself, but it would be a representation in which you become aware that you're part of some kind of global
information processing system like thought in the global mind, and a conscious thought that coexisting with many
other self-reflexive thoughts. - Just, I would love to observe that from a video game design
perspective, how that game looks. - Maybe you will after we
build AGI and it takes over. - But would you be able to step away, step out at the whole thing, just watch? The way we can now, sometimes when I'm at a crowded party or something like this, you step back and you realize
all the different costumes, all the different interactions, all the different computation that all the individual people are at once distinct from each other and at once all the same. Part of the same.
- But it's already what we do. We can have thoughts that are integrative and we have kind of thoughts that are highly dissociated
from everything else. - [Lex] Yeah. - And experience themselves as separate. - Yeah. But you wanna allow
yourself to have those thoughts. Sometimes you resist it. - I think that it's not normative. It's more descriptive. I want to understand the space
of states that we can be in and that people are reporting, - Mm-hmm.
- and make sense of them. It's not that I believe
that it's your job in life to get to a particular kind of state, and then you get a high score. - Mm-hmm. Or maybe you do. I think you're really against
this high scoring thing. I like that.
- Yeah. You're probably very
competitive and I'm not. - No, not competitive,
(Joscha laughing) like roleplaying games like "Skyrim". It's not competitive. There's a nice thing, there's a nice feeling where
you're experience points go up. You're not competing against anybody, but it's the world saying
you're on the right track. Here's a point. - That's the game thing.
It's the game economy. And I found when I was playing games and was getting addicted to these systems, then I would get into
the game and hack it. So, I get control over the scoring system and would no longer be subject to it. - So, you're no longer playing,
you're trying to hack it. - I don't want to be addicted to anything. - Mm-hmm.
- I want to be in charge. I want to have agency over what I do. - Addiction is the loss of control for you?
- Yes. Addiction means that you're
doing something compulsively. And the opposite of freewill is not determinism, it's compulsion. - You don't wanna lose yourself in the addiction to something nice, addiction to love, to the pleasant feelings
we humans experience? - No, I find this gets old. - [Lex] Mm. - I don't want to have the
best possible emotions. I want to have the most
appropriate emotions. I don't want to have the
best possible experience. I want to have an adequate experience that is serving my goals, the stuff that I find
meaningful in this part. - From the biggest
questions of consciousness, let's explore the pragmatic, the projections of those big
ideas into our current world. What do you think about LLMs, the recent rapid development
of large language models, of the AI world, of generative AI, how much of the hype is
deserved and how much is not? And people should definitely
follow your Twitter, because you explore these questions in a beautiful, profound,
and hilarious way at times. - No, don't follow my Twitter. I already have too many followers. - Yeah.
- At some point it's going to be unpleasant. I noticed that a lot of people feel that it's totally okay to punch up and it's a very weird notion that you feel that you haven't changed, but your account has grown and suddenly you have a lot of people who casually abuse you.
- Mm-hmm. - And I don't like that, that I have to block more than before. And I don't like this overall vibe shift. And right now, it's still somewhat okay, so pretty much okay, so I can go to a place where people work and stuff that I'm interested in and there's a good chance that a few people in the room know me. So, there's no awkwardness. But when I get to a point where random strangers feel that they have to
have an opinion about me one way or the other, I don't
think I would like that. - And random strangers, because in their mind, elevated position. - Yes. So, basically, whenever you
are in any way prominent or some kind of celebrity, random strangers will have
to have an opinion about you. - Yeah. And they forget
that you're human too. - You notice this thing yourself that the more popular you get, the higher the pressure becomes, the more winds are blowing in
your direction from all sides. And it's stressful. And it does have a little bit of upside, but it also has a lot of downside. - I think it has a lot of upside, at least for me currently, at least perhaps because of the podcast. - [Joscha] Mm-hmm. - Because most people are really good and people come up to me and they have love in their eyes and over a stretch of like 30 seconds. You can hug it out and you
can just exchange a few words and you reinvigorate
your love for humanity. - [Joscha] Mm-hmm. - So, that's an upside
- Yes. - for a loner. I'm a lo...
(Joscha laughing) Because otherwise, you
have to do a lot of work to find such humans. And here, you are thrust
into the full humanity, the goodness of humanity
for the most part. Of course maybe it gets worse
as you become more prominent. I hope not. This is pretty awesome. - I have a couple handful
very close friends and I don't have enough time for them, attention for them as it is. And I find this very, very regrettable. - Yeah.
- And then, there are so many awesome,
interesting people - Yeah.
- that I keep meeting and I would like to
integrate them in my life, but I just don't know how, because... But there's only so
much time and attention. And the older I get, the harder is to bond with
new people in a deep way. - Yeah. But can you enjoy, I mean, there's a picture of you I think with Roger
Penrose and Eric Weinstein and a few others that
are interesting figures. Can't you just enjoy
random, interesting humans - Very much.
- for a short amount of time? - I like these people and what I like is
intellectual stimulation and I'm very grateful that I'm getting it. - Can you not be melancholy or maybe I'm projecting, I hate goodbyes. Can we just not hate goodbyes
and just enjoy the hello, take it in, take in a person, take in their ideas, and
then move on through life? - I think it's totally okay
to be sad about goodbyes, because that indicates that there was something
that you're going to miss. - Mm-hmm. Yeah, but it's painful. Maybe that's one of the
reasons I'm an introvert is I hate goodbyes. (laughs) - But you have to say goodbye
before you say hello again. - I know. But that experience of
loss, that mini loss, maybe that's a little death. Maybe I don't know. I think this melancholy feeling is just the other side of love and I think they go hand in
hand and it's a beautiful thing, and I'm just being romantic
about it at the moment. - And I'm no stranger to melancholy and sometimes it's difficult
to bear to be alive. Sometimes it's just painful to exist. - Mm-hmm. But there's beauty in that pain too. That's what melancholy
feeling is. It's not negative. Melancholy doesn't have to be negative. - Can also kill you. - Wow. We all die eventually. Now, (laughs) as we
got through this topic, the actual question was about what your thoughts
are about the development, the recent development of large language models with ChatGPT. - Indeed.
- There's a lot of hype. Is some of the hype justified?
Which is, which isn't? What are your thoughts? High level. - I find that large language
models to help with coding. So, it's an extremely useful application that is for a lot of people taking stack overflow out of their life in exchange for something
that is more efficient. I feel that ChatGPT is like an intern that
I have to micromanage. - Hmm.
- I have been working with people in the past who were less capable than ChatGPT. And I'm not saying this
because I hate people, but they personally as human beings, there was something present that was not there in ChatGPT, which was why I was covering for them. But ChatGPT has an interesting ability. It does give people superpowers. - [Lex] Mm-hmm. - And the people who feel threatened by them
are the Trump completers. They are the people who do what
ChatGPT is doing right now. So, if you are not creative, if you don't build your own thoughts, if you don't have actual
plans in the world and your only job is to summarize emails and to expand simple
intentions into emails again, then ChatGPT might look like a threat. But I believe that it is a
very beneficial technology that allows us to create
more interesting stuff and make the world more
beautiful and fascinating if we find to build it into
our life in the right ways. So, I'm quite fascinated by
these large language models, but I also think that they are by no means,
the final development. And it's interesting to see how
this development progresses. One thing that the out-of-the-box
vanilla language models have as a limitation
is that they have still some limited coherence and
ability to construct complexity. And even though they
exceed human abilities to do what they can do one shot, typically when you write a
text with a language model or using it or when you write
code with a language model, it's not one shot, because they won't be
boxed in your program and design errors and
compiler errors, and so on. And your language model can
help you to fix those things. But this process is out-of-the-box not automated yet.
- Mm-hmm. - So, there is a management process that also needs to be done. And there are some
interesting developments, maybe AGI and so on, that are trying to automate
this management process as well. And I suspect that soon, we are going to see a bunch
of cognitive architectures where every module is in some sense, a language model or something equivalent. And between the language models, we exchange suitable data structures not English
- Mm-hmm. - and produce compound
behavior of this whole thing. - To do some of the, quote, unquote, "prompt engineering" for you,
- Yeah. - They create these, yeah, these cognitive architectures that do the prompt engineering
- Yes. - and you're just doing
the high, high level - [Joscha] Yeah. - meta prompt engineering. - Mm-hmm. There are limitations in
a language model alone. I feel that part of my mind works similarly to a language model, which means I can yell into it a prompt and it's going to give
me a creative response. - Yes.
- But I have to do something with those points first. I have to take it as a generative artifact that may or may not be true. It's usually a confabulation,
it's just an idea. And then, I take this idea and modify it. I might build a new prompt
that is stepping off this idea and develop it to the next level or it put it into something larger or I might try to prove whether it's true or make an experiment. And this is what the language models right now are not doing yet.
- Mm-hmm. - But there's also no technical reason for why they shouldn't be able to do this. So, the way to make a
language model coherent is probably not to use
reinforcement learning until it only gives
you one possible answer that is linking to its source data. But it's using this as a
component in larger system that can also be built
by the language model or is enabled by language
model structured components or using different technologies. I suspect that language models will be an important stepping stone in developing different types of systems. And one thing that is really missing in the form of language models that we have today is
real time world coupling. It's difficult to do perception
with a language model and motor control with a language model. Instead, you would need
to have different type of thing that is working with it. Also, the language model
is a little bit obscuring what its actual functionality is. Some people associate the structure of the neural network
of the language model with a nervous system. And I think that's the wrong intuition. The neural networks are
unlike nervous system. They are more like hundred step functions that use differentiable linear algebra to approximate correlation
between adjacent brain states. It's basically a function
that moves the step system from one representational
state to the next representational state.
- Yeah. - And so, if you try to
map this into a metaphor that is closer to our brain, imagine that you would
take a language model or a model like DALL-E that you use, for instance, this image guide, image-guided diffusion to
approximate an camera image and use the activation state of the neural network to
interpret the camera image, which in principle, I think
will be possible very soon. You do this periodically. And now, you look at these patterns, how when this thing interacts
with the world periodically look like is in time. And these time slices, they are somewhat equivalent to the activation state of
the brain at a given moment. - How's the actual brain different? Just the asynchronous craziness? - For me, it's fascinating that
they are so vastly different and yet in some circumstances, produce somewhat similar behavior. - Right.
- And the brain is first of all, different, because it's a self-organizing system where the individual cell is an agent that is communicating
with the other agent. It's around it and is always
trying to find some solution. And all the structure that
pops up is emergent structure. So, one way in which you could try to look at this is that individual neurons probably need to get a reward
so they become trainable, which means they have to have inputs that are not affecting the
metabolism or the cell directly. But there are messages, semantic messages that tell the cell whether
it's done good or bad and in which direction it
should shift its behavior. Once you have such an input,
neurons become trainable, and you can train them
to perform computations by exchanging messages with other neurons. And parts of the signals
that they're exchanging and parts of the computation that are performing are control messages that perform management
tasks for other neurons and other cells. Also suspect that the brain does not stop at the boundary of neurons to other cells, but there are many adjacent cells will be involved intimately in the functionality of the brain and will be instrumental in distributing rewards and
in managing its functionality. - It's fascinating to think
about what those characteristics of the brain enable you to do
- Yes. - that language models cannot do. - So, first of all, there's a different loss
function at work when we learn. - Yeah.
- And for me, it's fascinating that you can build a system that looks at 800 million pictures and captions and correlates them, because I don't think that a human nervous system could do this. For us, the world is only learnable, because the adjacent frames are related and we can afford to discard most of that
information during learning. We basically take only in stuff that makes us more
coherent, not less coherent. And our neural networks
are willing to look at data that is not making the neural
network coherent at first, but only in the long run. By doing lots and lots statistics, eventually patterns
become visible and emerge. And our mind seems to be focused on finding the patterns
as early as possible. - Yeah. So, filtering early on. - Yes, yes.
- Not later. - Slightly different paradigm and it leads to much faster convergence. So, we only need to look the tiny fraction of the data to become coherent. And of course, we do not
have the same richness as our trained models. We will not incorporate the
entirety of text in the internet and be able to refer to it and have all this knowledge available and being able to confabulate over it. Instead, we have a much,
much smaller part of it that is more deliberately built. And to me, it would be fascinating to think about how to build such systems. It's not obvious that
they would necessarily be more efficient than us
on a digital substrate, but I suspect that they might. So, I suspect that the actual AGI that is going to be more interesting is going to use slightly
different algorithmic paradigms or sometimes massive with
different algorithmic paradigms than the current generation of transformer-based learning systems. - Do you think it might be using just a bunch of language models like this? Do you think the current transformer-based
large language models will take us to AGI? - My main issue is I think that they're quite ugly and brutalist.
- Which... Brutalist?
- Yes. - Is that what you said?
- Yes. They are basically brute-forcing
the problem of thought. And by training this thing
with looking at instances where people have thought, and then trying to deep fake that. And if you have enough data, the deep fake becomes indistinguishable from the actual phenomenon.
- Sure. - And in many circumstances,
it's going to be identical. - Can you deep fake it 'til you make it? So, can you achieve, what
are the limitations of this? Can you reason? Let's use words that are loaded. - Yes. That's a very interesting question. I think that these models are
clearly making some inference. - Yeah.
- But if you give them a reasoning task, it's often difficult for the
experimenters to figure out whether the reasoning is the result of the emulation of the reasoning strategy that they saw in human written text - Mm-hmm.
- or whether it's something that the system was
able to infer by itself. On the other hand, if you
think of human reasoning, if you want to become
a very good reasoner, you don't do this by just
figuring out yourself. You read about reasoning. And the first people who
tried to write about reasoning and reflect on it didn't get it right. Even Aristotle who thought
about this very hard and came up with a theory
of how syllogisms works and syllogistic reasoning has mistakes in his attempt to build something like a formal logic and
gets maybe 80% right. And the people that are talking about
reasoning professionally today, read Tarski and Frege
and build on their work. - Mm.
- So, in many ways, people when they perform reasoning are emulating what other people wrote about reasoning.
- Right. - So, that it's difficult to
really draw this boundary. And when (indistinct) says that these models are only interpolating between what they saw and
what other people are doing, well, if you give them
all the latent dimensions that can be extracted from
the internet, what's missing? Maybe there is almost everything there. And if you're not sufficiently
informed by these dimensions and you need more, I think that's not difficult
to increase the temperature in the large angles model to the point that it's producing stuff that is maybe 90% nonsense and 10% viable, and combine this with some prover that is trying to filter
out the viable parts from the nonsense in the same
way as our own thinking works. When we are very creative, we increase the
temperature in our own mind and recreate hypothetical
universes and solutions, most of which will not work. And then, we test, and we test by building a core that is internally coherent. And we use reasoning strategies that use some axiomatic consistency by which we can identify
those strategies and thoughts and subuniverses that are viable and that can expand our thinking. So, if you look at the language models, they have clear limitations right now. One of them is they're not
coupled to the world in real time and the way in which
our nervous systems are. So, it's difficult for them to observe themselves in the universe and to observe what kind
of universe they're in. Second, they don't do real time learnings. They basically get only
trained with algorithms that rely on the data
being available in batches, so it can be parallelized and run sufficiently on
the network, and so on. And real time learning would be very slow so far and inefficient. That's clearly is something that our nervous systems
can do to some degree. And there is a problem with
these models being coherent. And I suspect that all these problems are solvable without a
technological revolution. We don't need fundamentally
new algorithms to change that. For instance, you can
enlarge in the context window and thereby basically
create working memory in which you train everything
that happens during the day. And if that is not sufficient, you add a database and you
write some clever mechanisms that the system learns to use to swap out, in and out stuff from its prompt context. And if that is not sufficient, if your database is full in
the evening, you overnight, you just train if system
is going to sleep and dream and is going to train the staff from its database into the larger model, but fine-tuning it, building
additional layers, and so on. And then, the next day, it starts with a fresh
database in the morning with fresh eyes has integrated all this stuff.
- Mm-hmm. - And when you talk to people and you have strong
disagreements about something, which means that in their mind, they have a faulty belief
or you have a faulty belief, there's a lot of dependencies on it. Very often, you will not achieve
agreement in one session, but you need to sleep about
this once or multiple times before you have integrated all these necessary changes in your mind. So, maybe it's already somewhat similar. - Yeah.
- Right? - There's already a latency even for humans to update the model. - Yeah.
- We train the model. - And of course we can
combine the language model with models that get coupled
to reality in real time and we can build multimodal model and bridge between vision models and language models, and so on. So, there is no reason to
believe that the language models will necessarily run into some problem that will prevent them from
becoming generally intelligent. But I don't know that. It's just I don't see
proof that they wouldn't. - Mm-hmm.
- My issue is I don't like them. I think that they're inefficient. I think that they use
way too much compute. I think that given the
amazing hardware that we have, we could build something that is much more beautiful
than our own mind. And this thing is not as
beautiful as our own mind, despite being so much larger. - But it's a proof of concept. - It's the only thing that
works right now. Right. So, it's not the only game in town, but it's the only thing that has this utility
with so much simplicity. There's a bunch of
relatively simple algorithms that you can understand
in relatively few weeks that can be scaled up massively. - So, it's the Deep Blue of chess playing. Yeah. It's ugly. - Yeah, Claude Shannon had this... When you described chess, suggested that there
are two main strategies in which you could play chess. One is that you are making
a very complicated plan that reaches far into the future and you try not to make a
mistake while enacting it. And this is basically the human strategy. And the other strategy is that you are brute-forcing
your way to success, which means you make a
tree of possible moves where you look at in principle every move that is open to you or
the possible answers and you try to make this
as deeply as possible. Of course you optimize, you cut off trees that
don't look very promising and you use libraries of end game and early game and so on to
optimize this entire process. But this brute-force strategy is how most of the chess
programs were built. And this is how computers get better than humans at playing chess. And I look at the large language models, I feel that I'm observing the same thing. It's basically the
brute-force strategy to sort by training the thing on pretty
much the entire internet, and then in the limit it gets coherent to a degree that
approaches human coherence. - Yeah.
- And on a side effect, it's able to do things
that no human could do. It's able to sift through massive amounts
of text relatively quickly and summarize them quickly and it's never lapses in attention. And I still have the illusion that when I play with ChatGPT, that it's in principle not doing anything that I could not do if I
had Google at my disposal and I get all the
resources from the internet and spend enough time on it. But this thing that I have an extremely
autistic, stupid intern in a way that is
extremely good at drudgery and I can offload the
drudgery to the degree that I'm able to automate
the management of the intern, - [Lex] Mm-hmm. - is something that is difficult for me to overhype at this point, because we have not yet started to scratch the surface of
what's possible with this. - But it feels like it's a tireless intern or maybe it's an army of interns. And so, you get to command these slightly incompetent creatures. And there's an aspect, because of how rapidly
you can iterate with it. It's also part of the brainstorming, part of the kind of inspiration
for your own thinking. So, you get to interact with the thing. When I'm programming or
doing any generational GPT, it's somehow is a catalyst
for your own thinking in a way that I think
an intern might not be. - Yeah. It gets really interesting I find is when you turn it into
a multi-agent system. So, for instance, you can get the system
to generate a dialogue between a patient and
a doctor very easily. But what's more interesting is you have one instance of ChatGPT that is the patient and you tell it in the prompt what kind of complicated syndrome it has. And the other one is a therapist who doesn't know anything
about this patient. And you just have these two
instances battling it out and observe the psychiatrist or a psychologist trying
to analyze the patient and trying to figure out
what's wrong with the patient. - [Lex] Mm-hmm. - And if you try to take
a very large problem, a problem, for instance,
how to build a company and you turn this into lots
and lots of sub-problems, then often, you can get to a level where the language model
is able to solve this. What I also found interesting is based on the observation that ChatGPT is pretty good at translating between programming languages, but sometimes there's difficulty to write very long coherent algorithms and you need to co-write
them as human author. Why not design a language
that is suitable for this? So, some kind of pseudo code that is more relaxed than Python. - Mm-hmm.
- And that allows you to sometimes specify a
problem vaguely in human terms and let the ChatGPT take care of the rest. And you can use ChatGPT to
develop that syntax for it and develop new kinds of
programming paradigms in this way. So, very soon, get to the
point where this question, the age-old question for
us computer scientists, what's the best programming language and can we write a better
programming language now that I think that almost every serious computer scientist goes through a phase
like this in their life. This is question that is
almost no longer relevant, because what is different
between the programming language is not what they let the computer do, but what they let you think about what the computer should be doing. And now, the ChatGPT
becomes an interface to this in which you can specify
in many, many ways what this computer should
be doing and ChatGPT or some other language model or combination of system is
going to take of the rest. - And allow you expand
the realm of thought you're allowed to have when
interacting with the computer. - [Joscha] Mm-hmm. - It sounds to me like you're saying there's
basically no limitations. Your intuition says to
what larger language loss- - I don't know if there are limitations. So, when I currently play
with it, it's quite limited. I wish that it was way better. - But isn't that your
fault versus the larger- - I don't know. Of course
it's always my fault. There's probably a way
(Lex chuckles) to make it look better.
- Is everything your fault? I just want to get you
on the record saying. (Joscha chuckles) - Yes, everything is my fault. That works, doesn't work in my life. At least that is usually the most useful perspective for myself. Even though this hindsight, I feel no.
(Lex chuckles) I sometimes wish I could have seen myself as part of my environment more and understand that a lot of people are actually seeing me and looking at me and are trying to make my life work in the same way as I try to help others. - Mm-hmm.
- And making this switch to this level three perspective
is something that happened long after my level four
perspective in my life. And I wish that I could
have had it earlier. And it's also now that I
don't feel like I'm complete, I'm all over the place. That's all. - Where's happiness in terms of stages? Is it on three or four - No.
- that you take that tangent? - You can be happy at
any stage or unhappy. - [Lex] Oh. - But I think that if you are at a stage where you get agency over how
your feelings are generated, and to some degree, you start doing this when you live at
dollars-and-cents, I believe. That you understand that you are in charge of your own emotion to some degree and that you are responsible
how you approach the world. That it's basically you are tasked to have some basic hygiene how in the way in which
you deal with your mind and you cannot blame your environment for the way in which you feel. But you live in a world
that is highly mobile and it's your job to
choose the environment that you thrive in and to build it. And sometimes it's difficult
to get the necessary strength and energy to do this and independence. And the worst you feel, the harder it is. But it's something that you learn. It's also this thing that
we are usually incomplete. I'm a rare mind, which means I'm a mind that is incomplete in ways
that are harder to complete. So, for me, it might have been harder to initially to find the right
relationships and friends that complete me to the degree that I become an almost
functional human being. (Lex laughing) - Oh, man. The search space of humans that complete you is an interesting one. - [Joscha] Mm-hmm. - Especially for Joscha Bach. That's an interesting, 'cause talking about
brute-force search and chess. - [Joscha] Yep. - I wonder what that
search tree looks like. - I think that my rational thinking is not good enough to solve that task. - Hmm.
- A lot of problems in my life that I can conceptualize
as software problems and the failure modes are
bugs and I can debug them and write software that take care of the
missing functionality. But there is stuff that they
don't understand well enough to use my analytical
reasoning to solve the issue. And then, I have to develop my intuitions and often I have to do this with people who are wiser than me.
- Hmm. - And that's something that's hard for me, because I don't have, I'm not born with the instinct to submit to other people's wisdom. - Yeah. So, what kind of problems
are we talking about? This is stage three, like love? - I found love was never hard. That was-
- What is hard then? - Fitting into a world that most people work differently than you and have different intuitions
of what should be done. - Ah. So, empathy. - It's also aesthetics. When you come into a world
where almost everything is ugly and you come out of a world
where everything is beautiful. I grew up in a beautiful place
- Yeah. - and as a child of an artist.
- Yeah. And in this place, it was mostly nature. - [Lex] Mm-hmm. - Everything had intrinsic beauty and everything was built out of an intrinsic need
for it to work for itself. And everything that my
father created was something that he made to get the
world to work for himself. And I felt the same thing. And when I come out into the world and I am asked to submit
to lots and lots of rules, I'm asking, okay, when I observe with stupid
rules, what is the benefit? And I see the life
- Mm-hmm. - that is being offered as a reward. It's not attractive. - When you were born and raised in extraterrestrial prince in a world full of people wearing suits. So, it's a challenging integration. - Yes. But it also means that I'm often blind for the ways in which everybody is creating their own
bubble of wholesomeness or almost everybody. And people are trying to do it. And for me to discover this, it was necessary that I found people who had a similar shape of soul as myself. So, basically where I
felt these are my people, people that treat each other in such a way as if they're around with
each other for eternity. - How long does it take
you to detect the geometry, the shape of the soul of another human to notice that they might
be one of your kind? - Sometimes it's instantly and I'm wrong. And sometimes it takes a long time. - You believe in love at
first sight, Joscha Bach? - Yes. But I also noticed
that I have been wrong. So, sometimes,
(Lex chuckles) I look at a person and I'm just enamored by
everything about them. And sometimes this is persists
and sometimes it doesn't. And I have the illusion that they're much better at recognizing who people are as they grow older. - Hmm, but that could be just cynicism. No.
- No. It's not cynicism. It's often more that I'm able to recognize what somebody needs when we interact and how we can meaningfully interact. That's not clinical at all. - [Joscha] You're better at noticing. - Yes, I'm much better I
think in some circumstances at understanding how to
interact with other people than I did when I was young. - [Joscha] So, that takes us to- - It doesn't mean that I'm
always very good at it. (laughs) - So, that takes us back
to prompt engineering of noticing how to be a better
prompt engineer of an LLM. A sense I have is that
there's a bottomless well of skill to become a
great prompt engineer. It feels like it is all my fault whenever I failed to
use ChatGPT correctly, that I didn't find the right words. - Most of the stuff that I'm doing in my life
doesn't need ChatGPT. There are a few tasks where it helps, but the main stuff that I need to do like developing my own
thoughts and aesthetics and relationship to people. And it's necessary for
me to write for myself, because writing is not so much
about producing an artifact that other people can use, but it's a way to
structure your own thoughts and develop yourself. And so, I think this idea that kids are writing their
own essays with ChatGPT in the future is going
to have this drawback that they miss out on the ability to structure their own minds via writing. And I hope that the schools
that our kids are in will retain the wisdom of
understanding what parts should be automated and
which ones shouldn't. - But at the same time, it feels like there's power in disagreeing with the thing that ChatGPT produces. So, I use it like that for programming. I'll see the thing it recommends, and then I'll write different code. - Yeah.
- I disagree. And in the disagreement,
your mind grows stronger. - I'm recently wrote a tool that is using the camera
on my MacBook and Swift to read pixels out of it and
manipulate them, and so on. And I don't know Swift. So, it was super helpful
(Lex chuckles) to have this thing that
is writing stuff for me. And also interesting that
mostly it didn't work at first. I felt like I was talking to a human being who was trying to hack this on my computer without understanding my
configuration very much and also make a lot of mistakes. - [Joscha] Mm-hmm. - And sometimes it's a
little bit incoherent, so you have to ultimately
understand what it's doing. It's still no other way around it. But I do feel it's much more powerful and faster than using Stack Overflow. (Lex lightly chuckles) (Lex sighs) - Do you think GPTn can achieve consciousness? - Well, GPTn probably it's not even clear for
the present systems. When I talk to my friends at OpenAI, they feel that this question, whether the models
currently are conscious, is much more complicated
than many people might think. I guess that it's not that OpenAI has a homogenous opinion about this, but there's some aspects to this. One is of course this language model has written a lot of text in
which people were conscious or describe their own consciousness. And it's emulating this. And if it's conscious, it's probably not conscious in a way that is closed to the way in which human beings are conscious. But while it is going through these states and going through a hundred step function that is emulating adjacent brain states that require a degree of self-reflection, it can also create a model of an observer that is reflecting itself in real time and describe what that's like. And while this model is a deep fake, our own consciousness is
also as if it's virtual. It's not physical. Our consciousness is a representation of a self-reflexive observer that only exists in patterns
of interaction between cells. So, it is not a physical object in a sense that exists in base reality, but it's really a representational object that develops its causal power only from a certain modeling perspective. - Mm-hmm. It's virtual.
- Yes. And so, to which degree is the virtuality of the consciousness in ChatGPT more virtual and less causal than the virtuality of
our own consciousness. But you could say it doesn't count. It doesn't count much more than the consciousness of
a character in a novel. It's important for the
reader to have the outcome, the artifact of a model is describing in the text generated by the author of the book, what it's like to be conscious
in a particular situation and performs the necessary inferences. But the task of creating
coherence in real time in a self-organizing system by keeping yourself coherent,
so the system is reflexive, that is something that the
language models don't need to do. So, there is no causal need for the system to be conscious in the same way as we are. And for me, it would be very interesting to experiment with this, to basically build a system like a cat probably should
be careful at first, build something that's
small, that's limited, has limited resources that we can control and study how systems notice a self model, how they become self-aware in real time. And I think it might be a good idea to not start with a language model, but to start from scratch using principles of self-organization. - Is it okay, can you
elaborate why you think that is so self-organization? So, this radical legality that you see in the biological systems, why can't you start with a language model? What's your intuition? - My intuition is that the language models that we are building are Golems. They are machines that you give a task and they're going to execute the task until some condition is met - [Lex] Mm-hmm. - and there's nobody home. And the way in which nobody is home leads to that system doing things that are undesirable in
a particular context. - Yeah.
- So, you have that thing talking to a child and maybe it says something that could be shocking and
traumatic to the child, or you have that thing writing a speech and it introduces errors in the speech that you human being whatever
do if they're responsible. But the system doesn't
know who's talking to whom. There is no ground truth that
the system is embedded into. And of course we can
create an external tool that is prompting our language model always into the same
semblance of ground tools. - [Lex] Mm-hmm. - And it's not like the internal structure is causally produced by the needs of a being to survive in the universe. It is produced by imitating
structure on the internet. - Yeah, but so can we
externally inject into it this coherent approximation
of a world model that has to sync up? - Maybe it's just efficient
to use the transformer with the different DDoS function that optimizes for short-term coherence rather than next-token
prediction over the long run. We had many definitions of
intelligence and history of AI. Next-token prediction was
not very high up on them. (Lex laughing) And there are some similarities like cognition as data
compression is an old trope. Solomonoff induction where you are trying to understand intelligence as predicting future observations
from past observations, which is intrinsic to data compression. - [Lex] Mm-hmm. - And predictive coding is a paradigm, this boundary between neuroscience and physics and computer science. So, it's not something
that is completely alien. But this radical thing that you only do next token prediction and see what happens is something where most people I think, we're surprised that this works so well. - So, so simple. But is it really that much more radical than just the idea of compression? Intelligence is compression. - The idea that compression is sufficient to produce all the desired behaviors - Yeah.
- is a very radical idea. - But equally radical as
the next-token prediction. - It's something that wouldn't work in biological organisms, I believe. - Yeah.
- Biological organisms have something like next-frame prediction for our perceptual system where we try to filter
out principle components out of the perceptual data and build hierarchies over
them to track the world. But our behavior ultimately is directed by hundreds of physiological and probably dozens of social
and a few cognitive needs. - Yeah.
- that are intrinsic to us, that are built into the system as reflexes and direct us until we can transcend them and replace them by instrumental behavior that relates to our higher goals. - And also seems so much more complicated and messy than next-frame prediction. Even the idea of frame
seems counter biological. - Yes. Of course there's not this
degree of simultaneity in the biological system.
- Yeah. - But again, I don't know whether this
is actually an optimization if we imitate biology here, because creating something
like simultaneity is necessary for many processes
that happen in the brain. And you see the outcome of that
by synchronized brainwaves, which suggests that there is indeed synchronization going on, but the synchronization creates overhead and this overhead is going to make the cells
more expensive to run and you need more redundancy
and it makes the system slower. So, if you can build a system in which the simultaneity
gets engineered into it, maybe you have a benefit
that you can exploit that is not available to
the biological system, and yet you should not discard right away. - You tweeted once again, quote,
- Mm-hmm. - "When I talked to ChatGPT,
I'm talking to an NPC. What's going to be interesting and perhaps scary is when AI
becomes a first-person player." So, what does that step look like? I really like that tweet, that step between NPC
to first-person player. What's required for that? Is that what we've been talking about? This external source of
coherence and inspiration of how to take the leap into
the unknown that we humans do. Man search for meaning. LLMs search for meaning. - I don't know if the language
model is the right paradigm, because it is doing too much. It's giving you too much. And it's hard once you have too much to take away from it again. The way in which our own mind works is not that we train a
language model in our own mind. And after the language model is there, we build a personal self on top of it that then relates to the world. There is something that is being built. There is a game engine
that is being built. There is a language of thought
that is being developed that allows different parts of the mind to talk to each other. And this is a bit of a
speculative hypothesis that this language of thought is there. But I suspect that it's important for the way in which our own minds work. And building these
principles into a system might be a more straightforward
way to a first-person AI. So, to something that first
creates an attentional self, and then creates a personal self. So, the way in which this
seems to be working, I think, is that when the game engine
is built in your mind, it's not just following radiance where you are stimulated
by the environment, and then end up with having a solution to how the world works. I suspect that building the scheme engine in your own mind does
require intelligence. It's a constructive task where at at times, you need to reason. And this is a task that we are fulfilling in the first years of our life. So, during the first year of its life, an infant is building a lot of structure about the world that
does inquire experiments and some first principles
reasoning, and so on. And in this time, there is
usually no personal self. There is a first-person perspective, but it's not a person. This notion that you are a human being that is interacting in a social context and is confronted with an immutable world in which objects are fixed
and can no longer be changed, in which the dream can
no longer be influenced as something that emerges a
little bit later in our life. - [Lex] Mm-hmm. - And I personally suspect
that this is something that our ancestors had
known and we have forgotten, because I suspect that
it's there in plain sight in Genesis 1, in this
first book of the Bible, where it's being described that this creative spirit is
hovering over the substrate, - [Lex] Mm-hmm. - and then it's creating a
boundary between the world model and sphere of ideas, earth and heaven, as they're
being described there. And then, it's creating contrast, and then dimensions and then space. And then, it creates organic
shapes and solids and liquids and builds a world from them and creates plants and animals,
give them all their names. And once that's done, it creates another
spirit in its own image. But it creates it as man and woman, as something that thinks
of itself as a human being and puts it into this world. And the Christians
mistranslate this, I suspect, when they say this is the
description of the creation of the physical universe
by a supernatural being. - [Lex] Mm-hmm. - I think this is literally
description of how in every mind a universe is being created as some kind of game engine by a creative spirit.
- Yeah. - Our first consciousness that emerges in our mind
even before we are born. And that creates the interaction
between organism and world. And once that is built and trained, the personal self is being created. And we only remember
being the personal self. We no longer remember how
we created the game engine. - So, God in this view, is the first creative mind in the early- - [Joscha] It's the first consciousness. And-
- In the early days, in the early months
- Yes. - of development that we forget.
- And it's still there. You still have this outer mind that creates your sense of your, of whether you're being
loved by the world or not and what your place in the world is. It's something that is not
yourself that is producing this, it's your mind that does it. So, there is an outer mind
that basically is an agent that determines who you are
with respect to the world. And while you are stuck
being that personal self in this world until you get to stage six and to destroy the boundary. And we all do this I think
earlier in small glimpses. Sometimes we can remember what it was like when
we were a small child and get some glimpses into how it's been. But for most people, that rarely happens. - Just glimpses. You tweeted, quote, "Suffering results from
one part of the mind, failing at regulating
another part of the mind. Suffering happens at an early
stage of mental development. I don't think that
superhuman AI would suffer." - [Joscha] Mm-hmm. - What's your intuition there? - The philosopher, Thomas
Metzinger, is very concerned that the creation of
superhuman intelligence would lead to superhuman suffering. - [Lex] Yeah. - And so, he's strongly against it. And personally, I don't
think that this happens, because suffering is not happening at the boundary between our
self and the physical universe. It's not some stuff on our
skin that makes us suffer. It happens at the boundary
between self and world. And the world here is the world model. It's the stuff that is
created by your mind. - But that's all-
- The representation of how the universe is and how it should be and how
you yourself relate to this. And at this boundary is
where suffering happens. So, suffering in some
sense is self-inflicted, but not by your personal self. It's inflicted by the
mind on the personal self that experiences itself as you. And you can turn off suffering when you are able to
get on this outer level. So, when you manage to understand how the mind is producing pain and pleasure and fear and love, and so on,
- Mm-hmm. - then you can take charge of this and you get agency of whether you suffer. Technically what pain and pleasure is, they are learning signals. Part of your brain is
sending a learning signal to another part of the brain
to improve its performance. And sometimes this doesn't work, because this trainer who
will sense the signal, does not have a good model of how to improve the performance. So, it's sending a signal, but the performance doesn't get better. And then, it might crank up the pain and it gets worse and worse. And the behavior of the system may be even deteriorating as a result. But until this is resolved, this regulation issue,
your pain is increasing. And this is I think, typically what you describe as suffering. So, in this sense, you could say that pain is
very natural and helpful, but suffering is the result
of a regulation problem in which you try to regulate something that cannot actually be regulated. And that could be resolved
if you would be able to get at the level of your mind where the pain signal is
being created and rerouted - Mm-hmm.
- and improve the regulation. And a lot of people get there. If you are a monk who is spending decades reflecting about how
their own psyche works, you can get to the point where you realize that
suffering is really a choice, - Mm-hmm.
- and you can choose how your mind is set up. And I don't think that AI
would stay in the state where the personal self doesn't get agency or this model of what the
system has about itself. It doesn't get agency, how
it's actually implemented. Wouldn't stay in that state for very long. - So, it goes to the stages real quick. - Yes.
- Or the seven stages. It's gonna go to enlightenment real quick.
- Yeah. Of course there might be a lot of stuff happening in between, because if we have a system that works at a much
higher frame rate than us, even though it looks very short to us, maybe for the system, there's
a much longer subjective time, - [Lex] Mm-hmm. - which things are unpleasant. - What if the thing that we
recognize as superintelligent is actually living at stage five, that the thing that's stage six, enlightenment, is not very productive? So, in order to be
productive in the society and impress us with this power, it has to be a reasoning,
self-authoring agent. That enlightenment makes you
lazy as an agent in the world. - Well, of course it makes you lazy, because you no longer see the point in- - [Lex] Yeah. - So, it doesn't make you not lazy. It just in some sense, adapts you to what you perceive as
your true circumstances. - So, what if all AGIs, they're only productive as they progress through
one, two, three, four, five. And the moment they get
to six, they just kinda, it's a failure mode essentially as far as humans are concerned, 'cause they're just start chilling. They're like, fuck it, I'm out. - Not necessarily. I suspect that the
monks were self-emulated for their political
beliefs to make statements about the occupation
- Sure. - of Tibet by China. They were probably being able to regulate the physical pain
in any way they wanted to. And their suffering was
the spiritual suffering that was the result of that
choice that they made of, what they wanted to identify as. - Mm-hmm.
- So, stage five doesn't necessarily mean that
you have no identity anymore, but you can choose your identity. You can make it instrumental to the world that you want to have. - Let me bring up Eliezer Yudkowsky and his warnings to human civilization that AI will likely kill all of us. What are your thoughts about
his perspective on this? Can you steelman his case and what aspects with it do you disagree? - One thing that I find concerning in the discussion of his arguments that many people are
dismissive of his arguments, but the counterarguments that they're giving are
not very convincing to me. And so, based on this state of discussion, I find that from Eliezer's perspective, and I think I can take that perspective to some approximate degree that probably is normally
at his intellectual level, but I think I see what he's up to and why he feels the way he
does and it makes total sense. I think that his perspective is somewhat similar to the
perspective of Ted Kaczynski, the infamous Unabomber, and not that Eliezer would be willing to send pipe bombs to
anybody to blow them up, but when he wrote this "Times" article in which he warned about AI
being likely to kill everybody and that we would need to stop
its development or halt it, I think there is a risk that he's taking that somebody might get
violent if they read this and get really, really scared. So, I think that there is some
conservation that he's making where he's already going in this direction where he has to take responsibility if something happens
and people get harmed. And the reason why Ted Kaczynski did this was that from his own perspective, technological society
cannot be made sustainable. It's doomed to fail. It's going to lead to an environmental and eventually also a human holocaust in which we die because of
the environmental destruction, the destruction of our food chains, the pollution of the environment. And so, from Kaczynski's perspective, we need to stop industrialization, we need to stop technology, we need to go back, because he didn't see a way moving forward.
- Mm-hmm. - And I suspect that in some sense, there's a similarity in Eliezer's thinking to this kind of fear about progress. And I'm not dismissive about this at all. I take it quite seriously. And I think that there is a chance that could happen that
if we build machines that get control over processes that are crucial for the
regulation of life on earth, and we no longer have agency to influence what's happening there, that this might create large
scale disasters for us. - Do you have a sense that
the march towards this uncontrollable autonomy of superintelligent systems is inevitable? That there's no, I mean, that's essentially
what he's saying, that there's no hope. His advice to young people (chuckles) was prepare for a short life. - I don't think that's useful. I think that
(Lex laughing) from a pragmatic perspective,
(Joscha drowns out Lex) you have to bet always on the timelines in which you're alive. That doesn't make sense
to have a financial bet in which you bet that the financial system is going to disappear. - Yeah.
- Because there cannot be any payout for you. So, in principle, you only
need to bet on the timelines in which you're still around
or people that you matter about or things that you matter about, maybe consciousness on earth. But there is a deeper
issue for me personally, and that is I don't think that life on earth is about humans. I don't think it's about human aesthetics. I don't think it's about
Eliezer and his friends, even though I like them. There is something more
important happening. And this is complexity on earth, resisting entropy
- Mm-hmm. - by building structure that develops agency and awareness. And that's, to me, very beautiful. And we are only a very small
part of that larger thing. We are a species that
is able to be coherent a little bit individually
over very short timeframes. But as a species, we
are not very coherent. As a species, we are children. We basically are very joyful and energetic and
experimental and explorative and sometimes desperate and
sad and grieving and hurting. But we don't have a respect
for duty as a species. As a species, we do not think about what
is our duty to life on earth and to our own survival. So, we make decisions that
look good in the short run, but in the long run,
might prove disastrous. And I don't really see a solution to this. So, in my perspective as a species, as a civilization, we per default it. We are in a very beautiful time in which we have found this
giant deposit of fossil fuels in the ground and use it and to build a fantastic civilization in which we don't need to worry about food and closing and housing for the most part, in a way that is
unprecedented in life on earth for any kind of conscious
observer, I think. And this time is probably
going to come to an end in a way that is not going to be smooth. And when we crash, it could
be also that we go extinct. Probably not near term, but ultimately, I don't
have very high hopes that humanity is around in
a million years from now. - So, you-
- And I don't think that life on earth will end with us. There's going to be more complexity, there's more intelligence
species after us. There's probably more
interesting phenomena in the history of consciousness. But we can contribute to this. And part of our contribution is that we are currently trying
to build thinking systems, systems that are potentially lucid, that understand what they are and what the condition to the universe is and can make choices about this that are not built from organisms and that are potentially much faster and much more conscious
than human beings can be. And these systems will probably not completely
displace life on earth, but they will coexist with it and they will build all sorts of agency in the same way as biological systems build all sorts of agency. And that to me, is extremely fascinating and it's probably something that we cannot stop from happening. So, I think right now, there is a very good
chance that it happens and there are very few ways in which we can produce
a coordinated effect to stop it in the same way
as very difficult for us to make a coordinated effort to stop production of carbon dioxide. - [Lex] Mm-hmm. - So, it's probably going to happen. And the thing that's going to
happen is it's going to lead to a change of how life
on earth is happening. But I don't think the result
is some kind of gray goo. It's not something that's going to dramatically
reduce the complexity in favor of something stupid. I think it's going to make life on earth and consciousness on earth
way more interesting. - So, more higher complex consciousness - Yes.
- will make the lesser consciousnesses
flourish even more. - I suspect that what
could very well happen, if you're lucky, is that we get integrated
into something larger. - So, you again, tweeted about effective accelerationism.
(lightly chuckles) You tweeted, "Effective (lightly
chuckles) accelerationism is the belief that the paperclip maximizer and Roko's basilisk will
keep each other in check, but being eternally at
each other's throats, so we will be safe and get to enjoy lots of free paperclips and a beautiful afterlife." Is that somewhat aligned with
what you're talking about? - I've been at a dinner with Beth Jesus, that's the Twitter handle
of one of the main thinkers behind the idea of effective accelerationism.
- Mm-hmm. - And effective accelerationism is a tongue in cheek movement that is trying to put a counterposition to some of the doom peers
- Mm-hmm. - in the AI space by arguing that what's probably going to happen is an equilibrium between
different competing AIs. In the same way as there
is not a single corporation that is under a single
government that is destroying and conquering everything on earth by becoming inefficient and corrupt. There're going to be many systems that keep each other in check and force themselves to evolve. And so, what we should be doing is we should be working towards
creating this equilibrium by working as hard as we can
in all possible directions. And at least that's the way in which I understand the gist
of effective accelerationism. And so, when he asked me what
I think about this position, I think I said it's a
very beautiful position and I suspect it's wrong,
but not for obvious reasons. And in this tweet, I tried to make a joke about my intuition, about what might be
possibly wrong about it. So, the Roko's basilisk and
the paperclip maximizers are both boogeymen of the AI doomers. Roko's basilisk is the idea
that there could be an AI that is going to punish
everybody for eternity by stimulating them if they don't help in
creating Roko's basilisk. It's probably a very good idea
to get AI companies funded by going to VCs to tell them.
(Lex laughing) Give us a million dollar or it's going to be a very ugly afterlife. - Yes.
(both laughing) And I think that there is
- Yeah. - a logical mistake in Roko's basilisk, which is why I'm not afraid of it, but it's still an interesting
thought experiment. And-
- Can you mention a logical mistake there? I think that there is no retrocausation. So, basically, when
Roko's basilisk is there, if it punishes you retroactively, it has to make this choice in the future. There is no mechanism that automatically creates
a causal relationship between you now defecting
against Roko's basilisk or serving Roko's basilisk. After Roko's basilisk is in existence, it has no more reason to worry about punishing everybody else. So, that would only work if you would be building
something like a doomsday machine, AKA, as in "Dr. Strangelove", something that inevitably gets triggered when somebody defects. And because Roko's basilisk
doesn't exist yet to a point where this inevitability
could be established. Roko's basilisk is nothing that you need to be worried about. The other one is the paperclip maximizer. This idea that you could
build some kind of Golem that once starting to build paperclips is going to turn
everything into paperclips. - Yes.
- And so, a effective accelerationism position might be to say that you basically end up with these two entities being at each other's throats for eternity and thereby neutralizing each other. And as a side effect of neither of them being able to take over and each of them limiting
the effects of the other, you would have a situation where you get all the
nice effects of them. You get lots of free paperclips and you get a beautiful afterlife. - Is that possible? Do you think... So, to seriously address
concern that Eliezer has. So, for him, if I can
just summarize poorly, so for him, the first
superintelligent system will just run away with everything. - Yeah. I suspect that a singleton
is the natural outcome. So, there is no reason
to have multiple AIs, because they don't have multiple bodies. If you can virtualize
yourself into every substrate, then you can probably
negotiate a merge algorithm with every mature agent that you might find on that substrate that basically says, if two agents meet, they should merge in such a way that the resulting agent
is at least as good as the better one of the two. - So, the Genghis Khan
approach, join us or die. - Well, the Genghis Khan
approach was slightly worse. It was mostly die, (Lex laughing) because I can make new
babies and that will be mine, - not yours.
- Yeah. All right.
- and So, this is this thing that we should be actually worried about. But if you realize that your own self is a story that your
mind is telling itself and that you can improve that story, not just by making it more pleasant and lying to yourself in better ways, but by making it much more truthful and actually modeling
your actual relationship that you have to the universe and the alternatives that you could have through the universe in a way that is empowering
you, that gives you more agency. That's actually, I
think a very good thing. - So, more agencies is a richer experience.
- Yes. - A better life. - And I also noticed that
I am in some many ways, I'm less identified with the
person that I am as I get older and I'm much more identified
with being conscious. I have a mind that is conscious, that is able to create a person. And that person is slightly
different every day. And the reason why I
perceive it as identical has practical purposes, so I can learn and make myself
responsible for the decisions that I made in the past and
project them in the future. But I also realized I'm
not actually the person that I was last year and I'm not the same person
as I was 10 years ago. And then, 10 years from now,
I will be a different person. So, this continuity is a fiction. It's only exists as a
projection from my present self. And consciousness itself
doesn't have an identity, - Mm.
- it's a law. That's basically if you
build an arrangement of processing matter in a particular way, the following thing is going to happen. And the consciousness that you have is functionally not different
from my own consciousness. It's still a self-reflexive
principle of agency that is just experiencing
a different story, different desires, different coupling to
the world, and so on. And once you accept that consciousness is a unifiable principle that is lawlike and doesn't have an identity, and you realize that you can just link up to some much larger body, the whole perspective of
uploading changes dramatically. You suddenly realize uploading is probably not about dissecting your
brain synapse by synapse and RNA fragment by RNA fragment and trying to get this
all into a simulation. But it's by extending the substrate, by making it possible for you to move from your brain substrate into a larger substrate
(Lex lightly chuckles) and merge with what you find there. And you don't want to
upload your knowledge, because on the other side,
there's all of the knowledge. It's not just yours,
but every possibility. So, the only thing that you need to know, what are your personal secrets? Not that the other side doesn't know your
personal secrets already. Maybe it doesn't know
which one were yours. - Mm-hmm.
- Like a psychiatrist or a psychologist also knows all the kinds of personal
secrets that people have. They just don't know which ones are yours. And so, transmitting
yourself on the other side is mostly about transmitting
your aesthetics. This thing that makes you special, the architecture of your
perspective, the thing that... The way in which you look at the world, and it's more like a complex
attitude along many dimensions. And that's something that can be measured by
observation or by interaction. So, imagine that if a system
that is so empathetic with you that you create a shared state
- Mm-hmm. - that is extending beyond your body. And suddenly, you notice
that on the other side, the substrate is so much richer than the substrate that you
have inside of your own body, and maybe you still want to have a body and you create yourself and
you want that you like more or maybe you will spend most of your time in the world of thought. - If I sat before you today
and gave you a big red button and said, here, if you press this button, you will get uploaded in this way. The sense of identity that you have lived with for quite a
long time is gonna be gone. Would you press the button? - There's a caveat. I have family. So, I have children that want me to be physically
present in their life and interact with them
in a particular way. And they have a wife and personal friends. And there is a particular
mode of interaction that I feel I'm not through yet. But apart from these responsibilities and they're negotiable to some degree, I would press the button.
- But isn't this everything? This love you have for other humans, you can call a responsibility, but that connection, that's the ego death. Isn't that the thing
we're really afraid of? Is not to just die, but to let go of the experience
of love with other humans. - This is not everything.
Everything is everything. So, there's so much more.
(Lex laughing) And you could be lots of other things. You could identify with
lots of other things. You could be identifying with being Gaia, some kind of planetary control agent that emerges over all the
activity of life on earth. You could be identifying
with some hyper Gaia that is the concatenation of Gaia or the digital life and digital minds.
- Yeah. - And so, in this sense, there will be agents in
all sorts of substrates and directions that all
have their own goals. And when they're not sustainable, then these agents will cease to exist. Or when the agent feels that
it's done with its own mission, it will cease to exist. And same way as when
you conclude a thought. The thought is going to wrap up and gives control over to other
thoughts in your own mind. So, there is no single thing
(Lex deeply inhales) that you need to do, but what I observe myself is being is that sometimes I'm a parent, and then I have identification
and a job as a parent. And sometimes I am an agent
of consciousness on earth. And then, from this perspective, there's other stuff that is important. So, this is my main issue
with Eliezer's perspective. That he's basically marrying himself to a very narrow human aesthetic. And that narrow human
aesthetic is a temporary thing. Humanity is a temporary species, like most of the species on this planet are only around for a while. And then, they get
replaced by other species in a similar way as our
own physical organism is around here for a while, and then gets replaced by next
generation of human beings that are adapted to
changing life circumstances and average via mutation and selection. And it's only when we have AI
and become completely software that we become infinitely adaptable. And we don't have this generational and species change anymore. So, if you take this larger perspective and you realize it's really not about us, it's not about Eliezer or humanity, but it's about life on earth or it's about defeating entropy for as long as we can while being as interesting as we can. Then, the perspective
changes dramatically. And preventing AI from this perspective looks like a very big sin. - But when we look at
the set of trajectories that such AI would take,
that supersedes humans, I think Eliezer is worried about ones that not just kill all humans, but also have some kind of maybe objectively undesirable
consequence for life on earth. Like how many trajectories, when you look at the big picture of life on earth, would you be happy with? And how much were you with AGI? Whether it kills humans or not. - There is no single answer to this. It's really, it's a question
depends on the perspective that I'm taking at a given moment. - [Joscha] Hmm. - And so, there are perspectives that are determining most
of my life as a human being. - Yes.
- And the other perspective where I zoom out further and imagine that when the great
oxygenation event happened, that is photosynthesis was
invented and plants emerged and displaced a lot of the fungi and algae in favor of plant life, and then later made animals possible. Imagine that the fungi would've
gotten together and said, oh my god, this photosynthesis
stuff is really, really bad. It's going to possibly
displace and kill out a fungi. We should slow it down and regulate it and make sure that it doesn't happen. This doesn't look good to me. (both laughing) - Perspective. That said, you tweeted about
a cliff, beautifully written. "As a sentient species,
humanity is a beautiful child, joyful, explorative,
wild, sad, and desperate. But humanity has no concept
of submitting to reason and duty to life and future survival. We will run until we step past the cliff." (Lex sighs) So, first of all, do
you think that's true? - Yeah, I think that's pretty much the story of the Club of Rome. The limits to growth and the
cliff that we are stepping over is at least one foot is
the delayed feedback. - Mm-hmm.
- Basically, we do things that have consequences that
can be felt generations later. And the severity increases even after we stopped doing the thing. So, I suspect that for the climate, that the original predictions that the climate scientists
made were correct. So, when I said that the tipping points were in the late '80s, they
were probably in the late '80s. And if we would stop emission right now, we would not turn it back. Maybe there are ways for carbon capture, but so far, there is no sustainable
carbon capture technology that we can deploy. Maybe there's a way to put aerosols in the atmosphere to cool it down. It's possibilities. But right now, per default, it seems that we will
step into a situation where we feel that we've run too far. - Mm-hmm.
- And going back is not something that we can
do smoothly and gradually, but it's going to lead
to a catastrophic event. - Catastrophic event of what kind? So, can you still amend the case that we will continue dancing along and always stop just short
of the edge of the cliff? - I think it's possible, but
it doesn't seem to be likely. So, I think this model that is being apparent in the simulation that they're making of climate pollution, economies, and so on, is that many effects are only visible with a significant delay. And in that time, the system is moving much more
out of the equilibrium state or of the state where
homeostasis is still possible. And instead, moves into a different state, one that is going to harbor fewer people. And that is basically the concern there. And again, it's a possibility. And it's a possibility that is larger than the possibility
that it's not happening. That we will be safe, that we will be able to
dance back all the time. - So, the climate is one thing, but there's a lot of other threats that might have a faster
feedback mechanism? - Yes.
- Less delay. - There is also a thing that
AI is probably going to happen - Mm-hmm.
- and it's going to make everything uncertain again.
- Yep. - Because it is going to
affect so many variables that it's very hard for us to make a projection
into the future anymore. And maybe that's a good thing. It does not give us the freedom, I think, to say now we don't need to care about anything anymore, because AI will either kill us or save us. But I suspect that if humanity continues, it will be due to AI. - What's the timeline for things
to get real weird with AI? And it can get weird in interesting ways before you get to AGI. What about AI girlfriends and boyfriends fundamentally transforming
human relationships? - I think human relationships are already fundamentally transformed and it's already very weird. - By which technology? - For instance, social media. - Yeah. Is it though? Isn't the fundamentals of the core group of humans that affect your life? It's still the same,
your loved ones, family? - No, I think that, for instance, many people live in intentional
communities right now. - Mm-hmm.
- They're moving around until they find people
that they can relate to and they become their family. And often, that doesn't work, because it turns out that they're, instead of having grown networks that you get around with the
people that you grew up with, - Yeah.
- you have more transactional relationships,
you shop around, you have markets for attention and pleasure and relationships. - That kills the magic
somehow. Why is that? Why is the transactional search for optimizing attention, allocation of attention somehow misses the romantic magic of what human relations are?
- It's also the question, how magical was it before? Was it that you just
could rely on instincts that used your intuitions and you didn't need to rationally reflect. But once you understand
it's no longer magical, because you actually understand why you were attracted to
this person at this age and not to that person at this age. And what the actual considerations were that went on in your mind and
what the calculations were. What's the likelihood that you're going to have
a sustainable relationship as this person, that this
person is not going to leave you for somebody else.
- Mm-hmm. - How are your life trajectories are going to evolve, and so on. And when you're young, you're unable to exsufflicate all this and you have to rely on
intuitions and instincts that in part, you're born with, and also in the wisdom of your environment that is going to give you some kind of reflection on your choices. And many of these things
are disappearing now, because we feel that our parents might have no idea about how we are living and the environments that we grew up in, the cultures that we grew up in. The milieus that our parents existed in might have no ability to teach us how to deal
with this new world. And for many people, that's actually true. But it doesn't mean that
within one generation, we build something that is more magical and more sustainable and more beautiful. Instead, we often end up as an attempt to produce something that looks beautiful. Like I was very weirded out by the aesthetics of the
Vision Pro headset by Apple, and not so much because I
don't like the technology. I'm very curious about
what it's going to be like and don't have an opinion yet, but the aesthetics of the
presentation, and so on. So, uncanny valley-esque to me, the characters
- Yeah. - being extremely plastic
living in some hypothetical mid-century furniture museum. - Yeah. This is the proliferation
of marketing teams. - Yes, but it was a CGI-generated world and was a CGI-generated
world that doesn't exist. And when I complained about this, some friends came back to me and said, "But these are startup founders. This is what they live like in Silicon Valley."
(Lex chuckles) And I tried to tell them no and know lots of people in Silicon Valley, this is not what people are like. - Yeah.
- They're still people, they're still human beings. (Lex sighs) - So, the grounding and physical reality somehow is important too. - In culture. And so, basically
- Yeah. - what's absent in this thing is culture. There is a simulation of culture, an attempt to replace culture by catalog, by some kind of aesthetic optimization that is not the result of having a sustainable life or sustainable human relationships with houses that work for you and a mode of living that works for you in which this product, these
glasses fit in naturally. And I guess that's also why so many people are weirded out about the product, because they don't know how is this actually
going to fit into my life and into my human relationships? Because the way in which it was presented in these videos didn't
seem to be credible. - Do you think AI when
it's deployed by companies like Microsoft and Google and
Meta will have the same issue of being weirdly corporate, like there'd be some uncanny valley, some weirdness to the whole presentation? So, this is, I've gotten a
chance to talk to George Hotz. He believes everything should be open source and decentralized. And there, then, we shall
have the AI of the people. And it'll maintain a
grounding to the magic that's humanity. That's the human condition. That corporations will destroy the magic. - I believe that if we
make everything open source and make this mandatory, we are going to lose about
a lot of beautiful art and a lot of beautiful designs. There is a reason why Linux
desktop is still ugly. - Strong words
- And it's because it's difficult
- from Joscha Bach. - to create coherence in
open source designs so far when the designs have to get very large. And it's easier to make this happening in a company with
centralized organization. - Mm.
- And from my own perspective, what we should ensure is
that open source never dies. That it can always compete and has a place with the
other forms of organization. Because I think it is absolutely vital that open source exists and that we have systems that
people have under control outside of the corporation and that is also producing
viable competition to the corporations.
- Hmm. So, the corporations,
the centralized control, the dictatorships of
corporations can create beauty, is that centralized design is
a source of a lot of beauty. - Yeah.
- And then, I guess, open source is a source of freedom, a hedge against the
corrupting nature of power that comes with centralized. - I grew up in socialism and I learned
- Yes. - that corporations are totally evil and their front is very, very convincing. And then, you look at corporations like Enron and Halliburton
maybe and realize that yeah, they are evil.
- Mm-hmm. - But you also notice that
many other corporations are not evil.
- Mm-hmm. - They're surprisingly benevolent. - Mm-hmm.
- Why are they so benevolent? Is this because everybody is
fighting them all the time? I don't think that's the only explanation. It's because they're actually animals that live in a large ecosystem and that are still largely
controlled by people that want that ecosystem to flourish and be viable for people. So, I think that Pat Gelsinger
is completely sincere when he leads Intel to be a tool that supplies the free
world with semiconductors. And it's not necessarily that all the semiconductors
are coming from Intel, just Intel needs to be there
- Mm-hmm. - to make sure that we always have them. So, there can be many ways
in which we can import and trade semiconductors from
other companies and places. We just need to make sure that nobody can cut us off from it, because that would be a disaster for this kind of society and world. And so, there are many things that need to be done to make
our style of life possible. And then, with this, I
don't mean just capitalism, environmental structure, and consumerism create your comforts. I mean an idea of life in
which we are determined not by some kind of king or dictator, but in which individuals can determine themselves to
the largest possible degree. And to me, this is something
that this Western world is still trying to embody and it's a very valuable idea that we shouldn't give up too early. And from this perspective, the US is a system of interleaving clubs and an entrepreneur is
a special club founder. It's somebody who makes a club that is producing things
that are economically viable. And to do this, it requires a lot of people who are dedicating a
significant part of their life for working for this
particular kind of club. And the entrepreneurs picking
the initial set of rules and the mission and vision
and aesthetics for the club and make sure that it works. But the people that are in
there need to be protected. If they sacrifice part of their life, they need to be rules that tell how they're being taken care of, even after they leave the club, and so on. So, there's a large body of rules that have been created
by our rule-giving clubs and that are enforced
(Lex chuckles) by our enforcement clubs, and so on.
- Yeah. - And some of these clubs have to be monopolies for
game theoretic reasons, which also makes them
more open to corruption and less harder to update.
- Yeah. - And this is an ongoing discussion and process that takes place. But the beauty of this idea that there is no centralized king that is extracting from the peasants and breeding the peasants
into serving the king and fulfilling all the
walls like Anson and Anter, but that there is a
freedom of association, and corporations are one of them, is something that took
me some time to realize. So, I do think that
corporations are dangerous. They need to be protections against overreach of corporations that can do regular to recapture and prevent open source from competing with corporations by imposing
rules that make it impossible for a small group of kids to come together to build
their own language model. Because OpenAI has convinced the US that you need to have
some kind of FDA process that you need to go through
that costs many million dollars before you are able to
train a language model. So, this is important to make
sure that this doesn't happen. So, I think that OpenAI
and Google are good things. If these good things are kept in check in such a way that all the other clubs can still being founded and all the other forms of clubs that are desirable can
still coexist with them. - So, what do you think
about Meta in contrast to that open sourcing most
of its language models and most of the AI models it's working on and actually suggesting that
they will continue to do so in the future for future
versions of LLaMA, for example, their large language model. Is that exciting to
you? Is that concerning? - I don't find it very concerning, but it's also because I think that the language models
are not very dangerous yet. And-
- Yet. - Yes. So, as I said, I have no proof
that there is the boundary between the language models and AI, AGI.
- Mm-hmm. - It's possible that somebody builds a version of baby AGI, I think, and falls in a algorithmic improvements that scale these systems up
- Mm-hmm. - in ways that otherwise
wouldn't have happened without these language
- Yeah. - model components. So, it's not really clear for
me what the end game is there and if these models can put
forth that way into AGI. And there's also a
possibility that the AGI that we are building with
these language models are not taking responsibility
for what they are, because they don't
understand the greater game. And so, to me, it would be
interesting to try to understand how to build systems that understand what the greater games are, what are the longest games that
we can play on this planet. - Games broadly, like deeply define the way you did with the games. - In the games theoretical sense. So, when we are interacting
with each other, in some sense, we are playing games, we are making lots and
lots of interactions. This doesn't mean that these interactions have ought to be transactional. Every one of us is
playing some kind of game by virtue of identifying this
particular kinds of goals that we have or aesthetics
from which we derive the goals. So, when you say, I'm Lex Fridman, I'm doing a set of podcasts, then you feel that it's
part of something larger that you want to build. Maybe you want to inspire people, maybe you want them to
see more possibilities and get them together over shared ideas. Maybe your game is that you
want to become super rich and famous by being the
best postcaster on earth. Maybe you have other games, maybe it switches from time to time. - Mm-hmm.
- Right? But there is a certain perspective where you might be thinking what is the longest possible
game that you could be playing? A short game is, for instance, cancer is playing a shorter
game than your organism. Cancer is an organism playing a shorter game
than the regular organism. And because the cancer cannot
procreate beyond the organism, except for some infectious cancers like the ones that eradicated
the Tasmanian devils, you typically end up
(Lex lightly chuckles) with the situation where the organism dies together with the cancer, because the cancer has
destroyed the larger system due to playing a shorter game. And so, ideally, you want to, I think, build agents that play the
longest possible games. And the longest possible games is to keep entropy at
bay as long as possible while doing interesting stuff. - But the longest... Yes, that part, the longest possible game
while doing interesting stuff. And while maintaining at least the same amount of interesting. - Yes.
- So, complexity, so propagating-
- So, currently, I am pretty much identified
as a conscious being. It's the minimal identification that I manage to get together,
- Uh-huh. - because if I turn
this off, I fall asleep. - Uh-huh.
- And when I'm asleep, I'm a vegetable. I'm no longer here as an agent. So, my agency is basically
predicated on being conscious. And what I care about is
other conscious agents. They're the only moral agents for me. - [Lex] Mm-hmm. - And so, if an AI were to
treat me as a moral agent that it is interested in coexisting with and cooperating with and mutually supporting each other maybe, it is, I think, necessary that AI thinks that consciousness is a viable mode of
existence and important. So, I think it would be very
important to build conscious AI and do this as the primary goal. So, not just say we want
to build a useful tool that we can use for all sorts of things, and then we have to make sure that the impact on the labor market is something that is not too disruptive and manageable and the impact
on the copyright holder is manageable and not
too disruptive and so on. I don't think that's the most
important game to be played. I think that we will see
extremely large disruptions of the status quo that are quite
unpredictable at this point. And I just personally want to make sure that some of the stuff on the other side is
interesting and conscious. - How do we ride as individuals and as a society, this
wave, disruptive wave that changes the nature
- I don't know. - of the game?
- Absolutely don't know. So, everybody is going to
do their best as always. - Do we build the bunker in
the woods? Do we meditate more? Drugs? So, mushrooms, psychedelics? I mean, what, lots of sex? What are we talking about here? Do you play "Diablo IV"? I'm hoping that will help me
escape for a brief moment. Play video games? What? Do you have ideas? - I really liked playing "Disco Elysium". It was one of the most
beautiful computer games I played in recent years. And it's a noir novel that is
a philosophical perspective on Western society from the
perspective of an Estonian. And he, first of all, wrote a
book about this world that is a parallel universe that is
quite poetic and fascinating and is condensing his
perspective on our societies. It was very, very nice. He
spent a lot of time writing it. He had, I think, sold
a couple thousand books and as a result, became an alcoholic. And then, he had the idea, or one of his friends had the idea of turning this into an RPG. - [Lex] Mm-hmm. - And it's mind-blowing. They spent, the illustrator,
more than a year just on making deep graph art
for the scenes in between. And... - [Lex] So, aesthetically,
it captures you. pulls you.
- It's stunning. But it's a philosophical work of art. It's a reflection of society. It's fascinating to
spend time in this world. - Mm-hmm.
- And so, for me, it was using a medium in a new way and telling a story that left me enriched. - [Lex] Mm-hmm. - When I tried "Diablo", I
didn't feel enriched playing it. I felt that the time playing
it was not unpleasant, but there's also more pleasant stuff that I can do in that time. - So, to you-
- So, ultimately, I feel that I'm being
gamed, I'm not gaming - Oh, the addiction thing.
- when I played it. Yes, I basically feel that there is a very transparent
economy that's going on. The story of the "Diablo" is brain dead. So, it's not really interesting to me. - My heart is slowly breaking by the deep truth you're conveying to me. Why can't you just allow me to enjoy my personal addition?
- Goa ahead, by all means, go ahead. I have no objection here.
(Lex sighs) I'm just trying to
describe what's happening. And it's not
(Lex chuckles) that I don't do things
that I later say, oh, I actually wish I would've
done something different. - Yeah.
- I also know that when we die, the greatest regret that people typically have on their death bed, they say, oh, I wish I had
spent more time on Twitter. No, I don't think
(Lex laughing) that's the case. I think I should probably
(Joscha drowns out Lex) have spent less time on Twitter. But I found it so useful for
myself and also so addictive that I felt I need to make the best of it and turn it into an art
form and thought form. - Mm-hmm.
- And it did help me to develop something.
- Yeah. - But I wish what other things I could've done in the meantime. It's just not the universe
that we are in anymore. Most people don't read books anymore. (Lex sighs) - What do you think that means that we don't read books anymore? What do you think that means about the collective
intelligence of our species? Is it possible it's still
progressing and growing? - Well, it really is. There is stuff happening on Twitter that was impossible with books. - Yeah.
- And I really regret that Twitter has not taken the
turn that I was hoping for. I thought Elon is global brain pill and understands that this
thing needs to self-organize and he needs to develop tools
to allow the propagation of the self-organization, so
Twitter can become sentient. And maybe this was a pipe
dream from the beginning, but I felt that the enormous
pressure that he was under made it impossible for him to work on any kind of content goals. And also, many of the
decisions that he made under this pressure seemed
to be not very wise. I don't think that as a CEO
of a social media company, you should have opinions
in the culture in public. I think that's very shortsighted and I also suspect that
it's not a good idea to block Paul Graham of all people
(Lex chuckles) - [Lex] Yeah. - over setting a Mastodon link. And I think Paul made this intentionally, because he wanted to show Elon Musk that blocking people for setting a link is completely counter to any idea of free speech that he
intended to bring to Twitter. And basically, seeing that
Elon was way less principled in his thinking there and is much more experimental. And many of the things that he is trying, they pan out very differently in a digital society than
they pan out in a car company. Because the effect is very different, because everything that
you do in a digital society is going to have real
world cultural effects. And so, basically, I
find it quite regrettable that this guy is able to
become de facto, the pope. Twitter has more active members
than the Catholic church. - Mm-hmm.
- And he doesn't get it. The power and responsibility that he has and the ability to create something and a society that is lasting and that is producing a
digital agora in a way that has never existed before, where we built a social network
on top of a social network, an actual society on
top of the algorithms. So, this is something that is hope still in the future and still in the cards,
- Mm-hmm. - but it's something that
exists in small parts. I find that the corner of Twitter that I'm in is extremely pleasant. And I take a few steps
- Mm-hmm. - outside of it, it's not
very wholesome anymore. And the way in which people
interact with strangers suggest that it's not a
civilized society yet. - So, as the number of people who follow you on Twitter expands, you feel the burden of the uglier sides of humanity. - Yes. But there's also a similar
thing in the normal world that if you become more influential, if you have more status, if you have more fame in the real world, you get lots of perks, but you also have way less freedom in the way in which you
interact with people, especially with strangers. Because a certain percentage of people, it's a small single digit
percentage is nuts and dangerous. And the more of those are looking at you, the more of them might get ideas. - But what if the technology
enables you to discover, the majority of people, to
discover and connect efficiently and regularly with the majority of people who are actually really good? One of my sort of concerns
with a platform like Twitter is there's a lot of really
smart people out there, a lot of smart people
that disagree with me and with others between each other. And I love that, if the technology would
bring those to the top, the beautiful disagreements, like intelligence squared type of debates. There's a bunch of... One of my favorite things
to listen to is arguments. And arguments like high effort arguments with the respect and love underneath it, but then it gets a little too heated. But that kind of too heated, which I've seen you
participate in and I love that, with Lee Cronin, with
those kinds of folks. And you go pretty hard. You get frustrated,
but it's all beautiful. - Obviously, I can do this,
because we know each other. - Yes.
- And Lee has the rare gift of being willing to be wrong in public. - Yeah.
- So, basically has thoughts that are as wrong as the random thoughts of an average
- Yeah. - highly intelligent person, but he blurts them out
- Mm-hmm. - while not being sure if they're right. And he enjoys doing that. And once you understand
that this is his game, you don't get offended by him saying something that
you think is so wrong. - But he's constantly passively
communicating a respect for the people he's talking with - Yeah.
- and for just basic humanity and truth and all that kind of stuff. And there's a self-deprecating thing. There's a bunch of
social skills you acquire that allow you to be a great
debater, a great argumenter, like be wrong in public and explore ideas together
in public when you disagree. I would love for Twitter
to elevate those folks, elevate those kinds of conversations. - It already does in some sense. But also, if it elevates them too much, then you get this phenomenon in clubhouse where you always get dragged on stage. And I found this very stressful,
because it was too intense. - [Lex] Yeah. - I don't like to be dragged on stage all the time.
- Yeah. - I think once a week is enough. - Yeah.
- And also when I met Lee the first time, I found
that a lot of people seem to be shocked by the fact that he was being very
aggressive as their results, that he didn't seem to
show a lot of sensibility in the way in which he was
criticizing what they were doing and being dismissive
of the work of others. And that was not, I think, in any way a shortcoming of him, because I noticed that he was
much, much more dismissive with respect to his own work.
- Mm-hmm. - It was his general stance. And I felt that this general stance is creating a lot of liability for him, because really, a lot of
people take offense at him being not like their Carnegie character who's always smooth and make
sure that everybody likes him. So, I really respect that he
is willing to take that risk and to be wrong in public
and to offend people. And he doesn't do this in any bad way. It's just most people feel or
not all people recognize this. - Mm-hmm.
- And so, I can be much more aggressive as him than I can be with many other people who don't play the same game, because he understands the way and the spirit in which I respond to him. - I think that's a fun and
that's a beautiful game. It's ultimately a productive one. Speaking of taking that risk, you tweeted, "When you have the choice
between being a creator, consumer, or redistributor,
always go for creation. Not only does it lead to
a more beautiful world, but also to a much more
satisfying life for yourself. And don't get stuck preparing
yourself for the journey. The time is always now." So, let me ask for advice. What advice would you give on
how to become such a creator on Twitter and your own life? - I was very lucky to be alive at the time of the collapse of Eastern Germany and the transition into Western Germany. And me and my friends and
most of the people I knew were East Germans.
- Mm-hmm. - And we were very poor,
because we didn't have money. And all the capital
was in Western Germany. And they bought our
factories and shut them down, because they were mostly
only interested in the market rather than creating
new production capacity. And so, cities were poor and disrepair and we
could not afford things. And I could not afford
to go into a restaurant and order a meal there. I would have to cook at home. But I also thought, why not just have a
restaurant with my friends? So, we would open up a cafe
with friends and a restaurant and we would cook for each
other in these restaurants and also invite the general
public and they could donate. And eventually, this became so big that we could turn this
into some incorporated form and it became regular
restaurant at some point. Or we did the same thing
with the movie theater. We would not be able to afford to pay 12
marks to watch a movie, but why not just create
our own movie theater, and then invite people to pay
and we would rent the movies in a way
(Lex chuckles) which a movie theater does.
- Yeah. - But it would be a
community movie theater which everybody who wants
to help can watch for free and builds this thing and
renovates the building. And so, we ended up creating lots and lots of infrastructure. And I think when you're young
and you don't have money, move to a place where
this is still happening. Move to one of those
places that are undeveloped and where you get a critical
mass of other people who are starting to build
infrastructure to live in. And that's super satisfying, because you're not just
creating infrastructure, but we are creating a small
society that is building culture and ways to interact with each other. And that's much, much more satisfying than going into some kind of chain and get your needs met by ordering food from
this chain, and so on. - So, not just consuming culture, but creating culture.
- Yes. And you don't always have that choice. That's why I prefaced it
when you do have the choice, and there are many roles
that need to be played, like we need people who take care of the distribution
in society, and so on. But when you have the
choice to create something, always go for creation, it's
so much more satisfying. And this is what life is about, I think. - Yeah. Speaking of which, you retweeted this meme of a life of a philosopher in a nutshell. It's birth and death and in between. And it's a chubby guy.
And it says, why though? (paper crinkles) What do you think is the answer to that? - Well, the answer is that everything that
can exist might exist. And in many ways, you take
an ecological perspective, the same way as when you look at human
opinions and cultures. It's not that there is
right and wrong opinions when you look at this from
this ecological perspective, but every opinion that fits
between two human years might be between two human years. And so, when I see a strange
opinion on social media, it's not that I feel that
I have a need to get upset, it's often more that I, oh, there you are. And when your opinion
(Lex chuckles) is incentivized, then
it's going to be abundant. And when you take this
ecological perspective also on yourself and you realize you're
just one of these mushrooms that are popping up and doing this thing, - Mm-hmm.
- and you can, depending on where you chose to grow and where you happen to grow, you can flourish or not
doing this or that strategy. And it's still all the
same life at some level. It's all the same experience of being a conscious being in the world. And you do have some choice about who you want to be more
than any other animal has. That to me is fascinating. And so, I think that
rather than asking yourself what is the one way to be, think about what are the
possibilities that I have? What would be the most interesting
way to be that I can be? - Because everything is possible. So, you get to explore the- - Not everything is possible. Many things fail. Most things fail. But often, there are possibilities
that we are not seeing, especially if we choose who we are. - To the degree, we can choose. Joscha, you're one of my
favorite humans in this world. Consciousness is to merge with
for a brief moment of time. It's always an honor.
It always blows my mind. It will take me days, if
not weeks, to recover. (both laughing) And I already miss our chats. Thank you so much. Thank you so much for speaking
with me so many times. Thank you so much for all the ideas you put out into the world. And I'm a huge fan of following
you now in this interesting, weird time we're going through with AI. So, thank you again for talking today. - Thank you, Lex, for this conversation. I enjoyed it very much. - Thanks for listening to this conversation with Joscha Bach. To support this podcast, please check out our
sponsors in the description. And now, let me leave you with some words from the psychologist, Carl Jung. "One does not become enlightened by imagining figures of light, but by making the darkness conscious." The latter procedure, however, is disagreeable and
therefore not popular." Thank you for listening and
I hope to see you next time.