Marc Raibert: Boston Dynamics and the Future of Robotics | Lex Fridman Podcast #412
- So BigDog became LS3, which
is the big load-carrying one. - [Lex] Just a quick pause. It can carry 400 pounds. - It was designed to carry
400, but we had it carrying about 1,000 pounds at one time. - Of course you did, just to make sure. - We had one carrying the
other one, we had two of them so we had one carrying the other one. - So one of the things that
stands out about the robots Boston Dynamics have created is how beautiful the movement is, how natural the walking is, and running is even flipping when it's throwing so maybe you can talk
about what's involved in making it look natural. - [Marc] Well, I think
having good hardware is part of the story, and people who think you don't need to innovate hardware anymore are wrong. - The following is a
conversation with Marc Raibert, a legendary roboticist founder and longtime CEO of Boston Dynamics and recently the executive director of the newly created Boston
Dynamics AI Institute that focuses on research
and the cutting edge on creating future generations of robots that are far better than
anything that exists today. He has been leading the creation
of incredible legged robots for over 40 years at CMU, at MIT, the legendary MIT Leg Lab, and then of course, Boston Dynamics with amazing robots like
BigDog, Atlas, Spot and Handle. This was a big honor and pleasure for me. This is the Lex Fridman podcast. To support it, please
check out our sponsors in the description. And now, dear friends,
here's Marc Raibert. When did you first fall
in love with the robotics? - Well, I was always a
builder from a young age. I was lucky, my father
was a frustrated engineer, and by that I mean he wanted
to be an aerospace engineer, but his mom from the old country thought that that would
be like a grease monkey and so she said no. So he became an accountant. But the result of that was our basement was always full of tools and
equipment and electronics. And, you know, from a young age, I would watch him assembling a kit, an Aiko kit or something like that. I still have a couple
of his old Aiko kits. But it was really during graduate school when I followed a
professor back from class, it was Bertolt at MIT and I was taking an interim class. It's IAP, Independent Activities Period. And I followed him back to his lab. And on the table was a vyke
arm, robot arm taken apart in probably 1,000 pieces. And when I saw that, you
know, from that day on, I was a roboticist. - Do you remember the year? - [Marc] 1974. - 1974. So there's just this arm in pieces. - [Marc] Yeah. - And you saw the pieces
and you saw in your vision, the arm when it's put back together and the possibilities that holds. - Somehow it spurred my imagination. I was in the brain and
cognitive sciences department as a graduate student
doing neurophysiology. I'd been an electrical engineer as an undergrad at Northeastern. And the neurophysiology
wasn't really working for me. You know, it wasn't conceptual enough. I couldn't see really how by
looking at single neurons, you were gonna get to a place
where you could understand, you know, control systems or
thought or anything like that. And, you know, the AI lab
was always an appealing, this was before C-SAIL right? This was in the '70s so the AI lab was always an appealing idea. And so when I went back to
the AI lab following him and I saw the arm, I just thought, you know, this is it. - It's so interesting the tension between the BCS, brain
cognitive science approach to understanding intelligence and the robotics approach to
understanding intelligence. - Well, BCS is now morphed a bit, right? They have the Center for
Brains, Minds and Machines, which is trying to bridge that gap. And even when I was there, you know, David Maher was in the AI lab. David Maher had models of the brain that were appealing both to biologists, but also to computer people. So he was a visitor in
the AI lab at the time, and I guess he became full-time there. So that was the first
time a bridge was made between those two groups. Then the bridge kind of went away and then there was
another time in the '80s. And then recently, you know,
the last five or so years, there's been a stronger connection. - You said you were
always kind of a builder. What stands out to you in
memory of a thing you've built? Maybe a trivial thing that
just kind of like inspired, inspired you in the possibilities that this direction of work might hold? - I mean, we were just doing
gadgets when we were kids. I have a friend, we were
taking, you know the... I don't know if everybody remembers, but fluorescent lights had this little aluminum cylinder, I can't even remember
what it's called now, that you needed a starter, I think it was. And we would take those apart,
fill 'em with match heads, put a tail on it, and make
it into little rockets. - So it wasn't always
about function, it was- - [Marc] Well, a rocket
was pretty much function. - I guess that is pretty functional. But yeah, I guess that is a question. How much was it about function versus just creating something cool? - I think it's still a
balance between those two. There was a time though, when I was, I guess I was probably already a professor or maybe late in graduate school when I thought that
function was everything. And that, you know, mobility, dexterity, perception and intelligence, those are sort of the key
functionalities for robotics, that that's what mattered
and nothing else mattered. And I even had kind of this
platonic ideal that a robot, if you just looked at a robot and it wasn't doing anything, it would look like a pile of junk, which a lot of my robots
looked like in those days. But then when it started
moving, you'd get the idea that it had some kind of life or some kind of interest in its movement. And I think we purposely
even designed the machines, not worrying about the aesthetics
of the structure itself. But then you know, it turns out that the aesthetics of the thing itself add and combine with the lifelike things that the robots can do. But the heart of it is, you know, making them do things
that are interesting. - So one of the things that
underlies a lot of your work is that the robots you create,
the systems you have created for over 40 years now, have a
kind of, they're not cautious. So a lot of robots that people know about move about this world very
cautiously, carefully, very afraid of the world. A lot of the robots you built,
especially in the early days, were very aggressive under actuated. They're hopping, they're
wild, moving quickly. So, is there a philosophy underlying that? - Well, let me tell you about how I got started on legs at all. When I was still a graduate
student, I went to a conference, it was a biological legged
locomotion conference, so I think it was in Philadelphia. So it was all biomechanics
people, you know, researchers who would look at muscle and maybe neurons and things like that. They weren't so much computational people, but they were more biomechanics. And maybe there were 1,000 people there. And I went to a talk, one of the talks, all the talks were about the body of either animals or
people and respiration, things like that. But one talk was by a robotics guy, and he showed a six legged robot that walked very slowly. It always had at least
three feet on the ground, so it worked like a table or
a chair with tripod stability. And it moved really slowly. And I just looked at that and
said, "wow, that's wrong." That's not anything like
how people and animals walk because we bounce and fly, you know, we have to predict what's gonna happen in order to keep our balance when we're taking a running
step or something like that. We use the springiness in our legs, our muscles and our tendons and things like that as part of the story. You know, the energy circulates. We don't just throw it away every time. And now I'm not sure I understood all that when I first saw it, but I definitely got inspired to say, "you know, let's try the opposite." And I didn't have a clue as to how to make a hopping robot work. Not balance in 3D, in
fact, when I started, it was all just about
the energy of bouncing and I was gonna have a
springy thing in the leg and some actuator so that
you could get an energy regime going of bouncing. And the idea that balance
was an important part of it didn't come until a little later. And then, you know, I made the one-legged, the pogo stick robots. Now I think that we need
to do that in manipulation. If you look at robot
manipulation, we've been working, a community has been
working on it for 50 years. We're nowhere near human
levels of manipulation. I mean, we can, you know, it's come along, but I think it's all too safe. And I think trying to break
out of that safety thing of static grasping, you
know, if you look at the... A lot of work that goes on, it's about the geometry of the part, and then you figure out
how to move your hand so that you can position
it with respect to that and then you grasp it
carefully and then you move it. Well, that's not anything like
how people and animals work. We juggle in our hands,
we hug multiple objects and can sort them. So now to be fair, being
more aggressive is gonna mean things aren't gonna work
very well for a while so it's a longer term
approach to the problem and that's just theory now. You know, maybe that won't pay off, but that's sort of how I'm
trying to think about it, trying to encourage our group to go at it. - Well, yeah, I mean, we'll
talk about what it means to what is the actual thing
we're trying to optimize for a robot, you know, sometimes, especially with human robot interaction, maybe flaws is a good thing. Perfection is not necessarily
the right thing to be chasing. Just like you said, maybe being
good at fumbling an object, being good at fumbling might
be the right thing to optimize versus perfect modeling of the object and perfect movement of the
arm to grasp that object, maybe perfection is not supposed
to exist in the real world. - I don't know if you
know my friend Matt Mason, who is the director of the Robotics Institute
at Carnegie Mellon, and we go back to
graduate school together, but he analyzed a movie of Julia Child's doing a cooking thing. And she did, I think
he said something like there were 40 different ways
that she handled a thing, and none of them was grasping. She would nudge, roll,
flatten with her knife, things like that and none
of them was grasping. - So, okay, let's go
back to the early days. First of all, you've
created and led the Leg Lab. The legendary Leg Lab at MIT. So what was that first hopping robot? - But first of all, the Leg Lab actually
started at Carnegie Mellon. - [Lex] Carnegie Mellon. - So I was a professor there
starting in about 1986. And so that's where the first
topping machines were built, starting, I guess we got
the first one working in about 1982, something like that. That was a simplified one. Then we got a three
dimensional one in 1983, the Quadruped, that we
built at the Leg Lab, the first version was built
in about 1984 or five, and really only got going about '86 or so and took years of development
to get it to there. - Let's just pause here. For people who don't know,
I'm talking to Marc Raibert, founder of Boston
Dynamics, but before that, you were a professor developing some of the most
incredible robots for 15 years. And before that, of course,
a grad student and all that. So you've been doing this
for a really long time, so you like skipped over this, but like, go to the first hopping robot. There's videos of some of this. I mean, these are incredible robots. So you talked about the very first step was to get a thing hopping up and down. - [Marc] Right. - And then you realized, well, balancing is a thing
you should care about and it's actually a solvable problem. So can you just go through
how to create that robot? - [Marc] Sure. - What was involved in
creating that robot? - Well, I'm gonna start on
the... not the technical side, but the, I guess we could call it the motivational side or the funding side. So before Carnegie Mellon,
I was actually at JPL, at the Jet Propulsion Lab for three years. And while I was there, I
connected up with Ivan Sutherland, who is sometimes regarded as the father of computer graphics 'cause of work he did both at MIT and then University of Utah
and Evans and Sutherland. Anyway, I got to know him
and at one point he said, he encouraged me to do some
kind of project at Caltech, even though I was at JPL, you know, those are kind of related institutions. And so I thought about it and I made up a list of
three possible projects and I purposely made the top one and the bottom one really boring sounding and in the middle I put pogo stick robot. And when he looked at it, you
know, Ivan is a brilliant guy, brilliant engineer, and a
real cultivator of people. He looked at it and knew right away what the thing that was worth doing. And so he had an endowed
chair so he had about $3,000 that he gave me to build the first model, which I went to the shop
and with my own hands, kind of made a first
model, which didn't work and was just a beginning shot at it. And Ivan and I took that to Washington. And in those days you
could just walk into DARPA and walk down the hallway
and see who's there. And Ivan who had been
there in his previous life and so we walked around and
we looked in the offices. Of course I didn't know
anything, you know, I was basically a kid, but
Ivan knew his way around and we found Craig Fields in his office. Craig later became the director of DARPA, but in those days he
was a program manager. And so we went in, I had a
little Samsonite suitcase, which we opened, and it
had just the skeleton of this one-legged hopping
robot and we showed it to him and you could almost see the
drool going down his chin. - Because he was excited?
- Of excitement. And he sent me $250,000. He said, "okay, I wanna fund this." And I was between institutions, I was just about to leave
JPL and I hadn't decided yet where I was going next. And then when I landed
at CMU, he sent 250,000, which in 1980 was a lot of research money. - Did you see the possibility
of where this is going, why this is an important problem? - No. - The balancing, I mean, it's legged. It has to do with legged locomotion. I mean, it has to do
with all these problems that the human body solves when
we're walking, for example. Like all the fundamentals are there. - Yeah, I mean I think
that was the motivation to try and get more at the
fundamentals of how animals work. But the idea that it
would result in, you know, machines that were anything like practical like we're making now, that
wasn't anywhere in my head, no. You know, as an academic I was mostly just trying
to do the next thing, make some progress, impress
my colleagues if I could. - And have fun.
- [Marc] And have fun. - Pogo stick robot.
- [Marc] Pogo stick robot. - So what was on the technical side, what are some of the challenges
of getting to the point where we saw, like in the
video, the pogo stick robot that's actually successfully hopping and then eventually doing flips
and all this kind of stuff? - Well, in the very early days, I needed some better
engineering than I had, than I could do myself
and I hired Ben Brown. We each had our way of
contributing to the design and we came up with a thing
that could start to work. I had some stupid ideas about how the actuation
system should work. And you know, we sorted that out. It wasn't that hard to make it balanced once you get the physical
machine to be working well enough and have enough control
over the degrees of freedom. And then we very quickly,
you know, we started out by having it floating on
an inclined air table. And then that only gave us
like six foot of travel. So once it started working,
we switched to a thing that could run around the
room on a another device. It's hard to explain these
without you seeing them, but you probably know
what I'm talking about, a planarizer and then the next big step was to make it work in 3D which that was really the scary part with these simple things. You know, people had inverted
pendulums at the time for years and they could control them by driving a cart back and forth, but could you make it
work in three dimensions while it's bouncing and all that? But it turned out not
to be that hard to do, at least at the level of performance we achieved at the time. - So, okay, you mentioned
inverted pendulum, but like, can you explain
how a hopping stick in 3D can control... can balance itself? - [Marc] Yeah, sure. - What does the actuation look like? - You know, the simple story is that there's three things going on. There's something making it bounce and we had a system that was estimating how high the robot was off the ground. And using that, you know, there's energy that can be in
three places in a pogo stick. One is in the spring,
one is in the altitude, and the other is in the velocity. And so when at the top of the hop, it's all in the height. And so you could just
measure how high you're going and thereby have an idea
of a lot about the cycle and you could decide whether
to put more energy in or less so that's one element. Then there's a part that you decide where to put the foot. And if you think when
you're landing on the ground with respect to the center of mass, so if you think of a pole vaulter, the key thing the pole vaulter has to do is get its body to the right
place when the pole gets stuck. If they're too far forward, they kind of get thrown backwards. If they're too far back, they go over. And what they need to do is get it so that they go mostly
up to get over the thing. And you know, high jumpers
is the same kind of thing. So there's a calculation
about where to put the foot and we did something relatively simple. And then there's a third part to keep the body at an
attitude that's upright 'cause if it gets too far, you could hop and just
keep rotating around. But if it gets too far,
then you run out of motion of the joints at the hips. So you have to do that. And we did that by applying a torque between the legs and the body every time the foot's on the ground, you only can do it while
the foot's on the ground. In the air, the physics don't work out. - How far does it have to tilt before it's too late to
be able to balance itself? Or it's impossible to balance
itself, correct itself? - Well, you're asking
an interesting question because in those days we didn't actually optimize things and they probably could have
gone much further than we did and then had higher performance and we just kind of got, you know, a sketch of a solution and worked on that. And then in years since
some people working for us, some people working for others, people came up with all
kinds of equations for, or, you know, algorithms
for how to do a better job, be able to go faster. One of my students worked on
getting things to go faster. Another one worked on
climbing over obstacles 'cause when you're running,
it's one on the open ground, it's one thing if you're
running like up a stair, you have to adjust where you are otherwise things don't work out, right? You land your foot on
the edge of the steps. So there's other degrees of freedom to control if you're
getting to more realistic, practical situations. - I think it's really interesting to ask about the early days 'cause you
know, believing in yourself, believing that there's
something interesting here. And then you mentioned finding
somebody else, Ben Brown. What's that like finding other people with whom you can build this crazy idea and actually make it work? - Probably the smartest thing I ever did is to find the other people. I mean, when I look at it now, you know, I look at Boston Dynamics and all the really
excellent engineering there, people who really make stuff work. I'm only the dreamer. - So when you talk about pogo stick robot or legged robots, whether it's Quadrupeds or humanoid robots, did people
doubt that this is possible? Did you experience a lot of
people around you kind of? - I don't know if they doubted
whether it was possible, but I think they thought
it was a waste of time. - Oh, it's not even an
interesting problem. - I think for a lot of people,
you know, people who were, I think it's been both though. Some people I think I felt
like they were saying, "oh, why are you wasting your time on this stupid problem?" But then I've been at many
things where people have told me it's been an inspiration to go out and attack these harder things. And I think it has turned
out, I think legged locomotion has turned out to be a useful thing. - Did you ever have like doubt
about bringing Atlas to life, for example or with BigDog,
just every step of the way, did you have doubt? Like, this is too hard of a problem. - I mean, at first I wasn't an
enthusiast for the humanoids 'cause again, it goes back to saying what's the functionality and the form wasn't as
important as the functionality. And also, there's an
aspect to humanoid robots that's all about the cosmetics where there isn't really
other functionality. And that kind of is off-putting
for me as a roboticist, I think the functionality really matters. So probably that's why I
avoided the human robots, humanoid robots to start with. But I'll tell you now, you know, after we started working on 'em, you could see that the
connection and the impact with other people,
whether they're lay people or even other technical people, there's a special thing that goes on, even though most of the humanoid robots aren't that much like a person, - But we anthropomorphize
and we see the humanity. But also like with Spot, you
can see not the humanity, but whatever we find compelling about social interactions
there in Spot as well. - I'll tell you, you know,
I go around giving talks and take Spot to a lot
of them and it's amazing. The media likes to say
that they're terrifying and that people are afraid and YouTube commenters like to say that it's frightening. But when you take Spot out there, now maybe it's self-selecting, but you get a crowd of people
who wanna take pictures, wanna pose for selfies,
wanna operate the robot, wanna pet it, wanna put clothes on it. It's amazing. - Yeah, little Spot. So if we move around history a little bit. So you said, I think in the
early days of Boston Dynamics that you quietly worked on
making a running version of Aibo. - [Marc] Yeah, yeah.
- Sony's robot dog. - [Marc] Yeah. - It's just an interesting
little tidbit of history for me. Like what stands out to
your memory from that task? For people who don't know, that LittleDog robot moves slowly. How did that become BigDog?
What was involved there? What was the dance between how do we make this cute LittleDog versus a thing that can
actually carry a lot of payload and move fast and stuff like that? - [Marc] What the connection
was is that at that point, Boston Dynamics was mostly a physics-based simulation company. So when I left MIT to
start Boston Dynamics, there was a few years of overlap, but the concept wasn't
to start a robot company. The concept was to use this
dynamic simulation tool that we developed to do
robotics for other things. But working with Sony,
we got back into robotics by doing the Aibo runner
by, we programmed, we made some tools for programming
QRIO, which was a small, a humanoid this big that
could do some dancing and other kinds of fun stuff. And I don't think it
ever reached the market, even though they did show it. You know, when I look back, I say that we got us
back where we belonged. - Yeah, you rediscovered
the soul of the company. - That's right. - And so from there it
was always about robots. - [Marc] Yeah. - So you started Boston Dynamics in 1992. - [Marc] Right. - What are some fond
memories from the early days? - One of the robots that we
built wasn't actually a robot, it was a surgical simulator,
but it had force feedback. So it had all the techniques of robotics and you look down into this
mirror, it actually was, and it looked like you were looking down onto the body you were working on. Your hands were underneath the mirror so they were where you were looking and you had tools in your hands that were connected up to
these force feedback devices made by another MIT spin
out sensible technologies. So they made the force feedback device, we attached the tools and
we wrote all the software and did all the graphics. So we had 3D computer graphics. It was in the old days,
this was in the late '90s when you had a Silicon Graphics computer that was about this big. It was the heater in
the office, basically. - [Lex] Nice, nice. - And we were doing surgical
operations, anastomosis, which was stitching tubes
together, tubes like blood vessels or other things in their body. And you could feel, and you
could see the tissues move and it was really exciting. And the idea was to make a trainer to teach surgeons how to do stuff. We built a scoring system, because we'd interviewed
surgeons that told us, what you're supposed to do and what you're not supposed to do. You're not supposed to tear the tissue, you're not supposed to
touch it in any place except for where you're trying to engage. There were a bunch of rules. So we built this thing and
took it to a trade show, a surgical trade show and the surgeons were
practically lined up. Well, we kept a score and we posted their scores
like on a video game. And those guys are so competitive that they really, really loved doing it And they would come around
and they see someone's score was higher there, so they would come back. But we figured out shortly after that we thought surgeons were gonna pay us to get trained on these things and the surgeons thought we
should pay them in order to, so they could teach us about the thing. And there was no money from the surgeons. And we looked at it and thought, well maybe we could sell it to hospitals that would teach to train their surgeons. And then we said, well, at the time, we were probably a 12 person company or maybe 15 people, I don't remember. There's no way we could go
after a marketing activity. The company was all
bootstrapped in those years. We never had investors
until Google bought us, which was after 20 years. So we didn't have any resources
to go after hospitals. So sort of, at one day, Rob and I were looking at that and we said we'd built another simulator for knee arthroscopy and we said this isn't gonna work. And we killed it and we moved on. And that was really a
milestone in the company because we sort of understood who we were and what would work and what wouldn't, even though technically it was
really a fascinating thing. - What was that meeting like? Were you just like sitting
at a table, you know what? - Probably. - We're going to pivot completely. We're going to let go of this thing we put so much hard work into,
and then go back to the thing it came from. - It just always felt right
once we did it, you know. - Just look at each other and
said, "let's build robots." - [Marc] Yeah. - What was the first robot you built under the flag of Boston Dynamics? BigDog? - Well, there was the Aibo runner, but it wasn't even a whole robot. It was just legs that we...
we took off the legs on Aibo's and attached legs we'd made. And you know, we got that working and showed it to the Sony people. We worked pretty closely
with Sony in those years. One of the interesting things is that it was before
the internet and Zoom and anything like that. So we had six ISDN lines installed and we would have a telecon every week that worked at very low frame rate something like 10 hertz. English across the boundary
with Japan was a challenge trying to understand what
each of us was saying and have meetings every week
for several years doing that. And it was pleasure working with them. They were really supporters. They seemed to like us
and what we were doing. That was the real transition from us being a simulation company into being a robotics company again. - It was a quadruped, the legs
were four legs or two legs? - Four legs, yeah. - And what did you learn from
that experience of building basically a fast moving quadruped? - Mostly we learned that
something that small doesn't look very exciting
when it's running. It's like it's scampering and you had to watch a slow mo for it to look like it was interesting. If you watched it fast,
it was just like a- - That's funny. - One of my things was
to show stuff in video, even from the very early
days of the hopping machines. And so I was always focused
on how's this gonna look through the viewfinder and running Aibo didn't look
so cool through the viewfinder. - So what came next in terms of what was a big next milestone
in terms of a robot you built? - I mean, you gotta say
that BigDog was, you know, sort of put us on the map and got our heads really pulled together. We scaled up the company,
BigDog was the result of Alan Rudolph at DARPA starting a biodinautics program. And he put out a request for proposals and I think there were
42 proposals written and three got funded, one was BigDog, one was a climbing robot RiSE, and, you know, that put things in motion. We hired Martin Buehler, he was a professor in Montreal at McGill. He was incredibly important for getting BigDog out of
the lab and into the mud, which is a key step to really
be willing to go out there and build it, break it, fix it, which is sort of one of
our mottos at the company. - [Lex] So, testing in the real world. - [Marc] Testing. - For people who don't know BigDog, maybe you can correct me, but it's a big quadruped four
legged robot, it looks big. Could probably carry a lot of weight. Not the most weight that Boston Dynamics
have built, but a lot. - Well it's the first thing that worked. So let's see, if we go
back to the Leg Lab, we built a quadruped that
could do many of the things that BigDog did but it had a hydraulic
pump sitting in the room with hoses connected to the robot. It had a VAX computer in the next room. It needed its own room 'cause it was this giant
thing with air conditioning. And it had this very complicated
bus connected to the robot. And the robot itself
just had the actuators. It had gyroscopes for sensing
and some other sensors, but all the power and
computing was off board. BigDog had all that stuff
integrated on the platform. It had a gasoline engine for power, which was a very complicated
thing to undertake. It had to convert the
rotation of the engine into hydraulic power, which
is how we actuated it. So there was a lot of learning just on building the physical robot and the system integration for that. And then there was the controls of it. - So for BigDog, you
brought it all together onto one platform? - [Marc] Right. - And then so you could- - [Marc] You could take
it out in the woods. - Yeah, and you did. - We did. We spent a lot of time down
at the Marine Corps base in Quantico where there was a trail called the Guadalcanal Trail. And our milestone that DARPA had specified was that we could go on
this one particular trail that involved a lot of challenge. And we spent a lot of time, our team spent a lot of
time down there hiking. Those were fun days. - Hiking with a robot. So what did you learn about like, what it takes to balance a
robot like that on a trail, on a hiking trail in the woods, but basically forget the
woods, just the real world? That's the big leap into
testing in the real world. - Yeah. As challenging as the woods were, working inside of a home or
in an office is really harder because when you're in the woods, you can actually take
any path up the hill, all you have to do is avoid the obstacles. There's no such thing as
damaging the woods at least, you know, to first order whereas if you're in a house,
you can't leave scuff marks, you can't bang into the walls. The robots aren't very comfortable
bumping into the walls, especially in the early days so I think those were
actually bigger challenges once we faced them. It was mostly, you know, getting the systems to
work well enough together, the hardware systems to
work and the controls. In those days, we did
have a human operator who did all the visual perception going up the Guadalcanal Trail. So there was an operator
who was right there who was very skilled at, even though the robot was balancing itself and placing its own feet, if the operator didn't do the
right thing, it wouldn't go. But years later we went back
with one of the electric, the precursor to Spot. And we had advanced the controls and everything so much that an amateur, complete amateur could operate
the robot the first time up and down and up and down
whereas it had taken us years to get there in the previous robot. - So if you fast forward,
BigDog eventually became Spot? - So BigDog became LS3, which is the big load carrying one. - [Lex] Just a quick pause. It can carry 400 pounds. - It was designed to carry 400, but we had it carrying about
1,000 pounds at one time. - Of course you did, just to make sure. - We had one carrying the
other one, we had two of them. So we had one carrying the other one. There's a little clip of that. We should put that out somewhere. That's from like 20 years ago, but. - Wow. Wow, and it can go for
very long distances. It can travel 20 miles. - [Marc] Yeah. Gasoline. - Gasoline, yeah. Okay, sorry. So LS3, then how did that lead to Spot? - So BigDog and LS3 had engine power and hydraulic actuation. Then we made a robot
that was electric power. So there's a battery driving
a motor, driving a pump, but still hydraulic actuation. - Yeah. - Larry sort of asked us, "could you make something
that weighed 60 pounds that would not be so intimidating
if you had it in a house where there were people?" And that was the
inspiration behind the Spot pretty much as it exists today. We did a prototype the same size. That was the first all
electric, non hydraulic robot. - What was the conversation
with Larry Page like about? So he here's a guy that kind
of is very product focused and can see a vision for
like what the future holds. That's just interesting kind of, a side, what was the brainstorm about the future robotics with him like? - I mean, it was almost as
simple as what I just said. - Make it smaller? - He said, "yeah, geez, could you, do you think you could make a smaller one that wouldn't be so intimidating?" Well, yeah, like a BigDog. - [Lex] Yeah. - If it was in your house. And I said, "yeah, we could do that." And we started and did. - Is there a lot of technical challenges to go from hydraulic to electric? - You know, I had been
in love with hydraulics and still love hydraulics. You know, it's a great technology. It's too bad that somehow the world out there looks at
it like it's old fashioned or that it's icky and it's
true that it is very hard to keep it from having
some amount of dripping from time to time. But if you look at the performance, how strong you can get
in a lightweight package. And of course we did a
huge amount of innovation. Most of hydraulic control that is the valve that
controls the flow of oil had been designed in
the '50s for airplanes. It had been made robust
enough, safe enough that you could count on it so that humans could fly in airplanes and very little innovation had happened. You know, that might not be fair to the people who make the valves. I'm sure that they did innovate, but the basic design had stayed the same and there was so much more you could do. And so our engineers designed valves, the ones that are in Atlas for instance, that had new kinds of circuits, they sort of did some of the computing that could get you much
more efficient use. They were much smaller and lighter so that the whole robot could be smaller and lighter. We made a hydraulic power supply that had a bunch of components integrated in this tiny package. It's about this big,
the size of a football, weighs five kilograms and it produces five kilowatts of power. Of course it has to have
a battery operating, but it's got a motor, a pump, filters, heat exchanger to keep
it cool, some valves, all in this tiny little package. So hydraulics, you know,
could still have a ways to go. - One of the things that stands out about the robots Boston
Dynamics have created is how beautiful the movement is. How natural the walking is and
running is, even flipping is throwing is. So maybe you can talk
about what's involved in making it look natural? - [Marc] Well, I think
having good hardware is part of the story. And people who think you don't need to innovate hardware anymore
are wrong in my opinion. So I think one of the things,
certainly in the early years, for me, taking a dynamic approach where you think about what's the evolution of the motion of the thing
gonna be in the future and having a prediction of that that's used at the time that
you're giving signals to it as opposed to it all being servoing, which is servoing is
sort of backward looking. It says, "okay, where am I now, I'm gonna try and adjust for that." But you really need to
think about what's coming. - So how far ahead do
you have to look in time? - It's interesting. I think that the number is
only a couple of seconds for Spot. So there's a limited horizon type approach where you're recalculating
assuming what's gonna happen in the next second or second and a half. And then you keep iterating,
you know, at the next, even though a tenth of a
second later you'll say, "okay, let's do that again
and see what's happening." And you're looking at
what the obstacles are, where the feet are gonna be
placed, how to, you know, you have to coordinate a lot of things if you have obstacles and you're
balancing at the same time and it's that limited
horizon type calculation that's doing a lot of that. But if you're doing
something like a somersault, you're looking out a lot further, right? If you wanna stick the landing, you have to at the time of launch, have momentum and rotation,
all those things coordinated so that a landing is within reach. - How hard is it to stick a landing? I mean it's very much under actuated. Like once you're in the air, you don't have as much
control about anything. So how hard is it to get that to work? First of all, did flips
with a hopping robot. - If you look at the first
time we ever made a robot do a somersault, it was in a planar robot. You know, it had a boom
so it was restricted to the surface of a sphere.
We call that planar. So it could move for and
aft, it could go up and down and it could rotate. And so the calculation
of what you need to do to stick a landing isn't
all that complicated. You have to look at, you know, you have to get time to make the rotation. So how hard you jump, how
high you jump gives you time. You look at how quickly you can rotate. And so, if you get those two right, then when you land, you have
the feet in the right place and you have to get rid of all that rotational and linear momentum but that's not too hard to figure out. And we made, you know,
back in about 1985 or six, I can't remember, we had a simple robot doing somersault. To do it in 3D, really the
calculation is the same. You just have to be balancing in the other degrees of freedom. If you're just doing a somersault, it's just a planar thing. Roy Rob was my graduate
student and we were at MIT which is when we made a two legged robot do a 3D
somersault for the first time. There, in order to get
enough rotation rate you needed to do tucking also, withdraw the legs in
order to accelerate it. And he did some really fascinating work on how you stabilize more
complicated maneuvers. You remember he was a
gymnast, a champion gymnast before he'd come to me. So he had the physical
abilities and he was an engineer so he could translate
some of that into the math and the algorithms that
you need to do that. - He knew how humans do it. He just had to get robots to do the same. - Unfortunately though, humans don't really know how they do it. - [Lex] Yeah.
- Right. - We're coached, we have ways of learning, but do we really understand in a physical, in a physics way what we're doing? Probably most gymnasts
and athletes don't know. - So in some way, by building robots, you are in part understanding
how humans do like walking, most of us walk without
considering how we walk really. - [Marc] Right. - And how we make it so
natural and efficient, all those kinds of things. - Atlas still doesn't walk like a person and it still doesn't walk
quite as gracefully as a person even though it's been
getting closer and closer. The running might be close to a human, but the walking is still a challenge. - That's interesting, right? That running is closer to a human. It just shows that the more
aggressive and kind of, the more you leap into the unknown, the more natural it is. I mean, walking is kind
of falling always, right? - And something weird about the knee that you can kind of do
this folding and unfolding and get it to work out just- a human can get it to work out just right. There's compliance, main
springiness in the design that are important to how it all works. Well we used to have a
motto at Boston Dynamics in the early days, which was "you have to run before you can walk." (Lex chuckles) - That's a good motto because you also had Wildcat, which was along the way towards
Spot, which is a quadruped that went 19 miles an
hour on flat terrain. Is that the fastest you've ever built? - [Marc] Oh yeah. - [Lex] Might be the fastest
quadruped in the world. I don't know. - For a quadruped, probably. Of course it was probably the loudest too. So we had this little
racing go-kart engine on it and we would get people from,
you know, three buildings away sending us, you know, complaining
about how loud it was. - So at the Leg Lab, I believe most of the
robots didn't have knees. How do you figure out what is the right number of actuators? What are the joints to have? Do you need to have, you
know, we humans have knees and all kinds of interesting
stuff on the feet. The toe is an important
part, I guess for humans or maybe it's not. I injured my toe recently and it made running very unpleasant so that seems to be kind of important. So how do you figure out for efficiency, for function, for aesthetics
how many joints to have? How many actuates to have? - Well, it's always a balance between wanting to get
where you really wanna get and what's practical to
do based on your resources or what you know and all that. So I mean, the whole
idea of the pogo stick was to do a simplification. Obviously it didn't look like a human. I think a technical
scientist could appreciate that we were capturing some of the things that are important in human locomotion without it looking like it,
without having a knee, an ankle. I'll tell you the first
sketch that Ben Brown made when we were talking
about building this thing was a very complicated thing
with zillions of springs, lots of joints. It looked much more like
a kangaroo or an ostrich or something like that. Things we were paying a lot
of attention to at the time. So my job was to say, "okay, well let's do something
simpler to get started and maybe we'll get there at some point." - I just love the idea
that you two were studying kangaroos and ostriches. - Oh yeah, we did. We filmed and digitized data from horses. I did a dissection of
a ostrich at one point, which has absolutely remarkable legs. - Dumb question, do ostriches
have a lot of musculature on the legs or no? - Most of it it's up in the feathers, but there's a huge amount
going on in the feathers, including a knee joint. The knee joint's way up there. The thing that's halfway down the leg that looks like a backwards
knee is actually the ankle, the thing on the ground, which looks like the foot
is actually the toes. It's an extended toe. - Fascinating. - But the basic morphology is
the same in all these animals. - What do you think is the most beautiful movement of an animal? Like what animal do you think
is the coolest land animal? 'cause fish is pretty cool. Like the fish goes in the water,
but like legged locomotion. - You know, the slow mos of
cheetahs running are incredible. There's so much back motion and grace and of course they're moving very fast. The animals running away from the cheetah are pretty exciting. You know, the pronghorn, which, you know, they do this all four legs
at once jump called the pronk to kind of confuse the... especially if there's a group of them to confuse whoever's chasing them. - So they do like a
misdirection type of thing? - Yep, they do a misdirection thing. The front end views of
the cheetahs running fast where the tail is whipping
around to help in the turns to help stabilize in the
turns, that's pretty exciting. - Because they spend a lot
of time in the air, I guess. - [Marc] Yeah. - As they're running that fast. - [Marc] But they also turn very fast. - Is that a tail thing or is that... Do you have to have
contact with the ground? - Everything in the body
is probably helping turn 'cause they're chasing something
that's trying to get away that's also zigzagging around. But I would be remiss if
I didn't say, you know, humans are pretty good too. You watch gymnasts, especially these days, they're
doing just incredible stuff. - Well, like, especially
like Olympic level gymnasts. See, but there could be, I
think there could be cheetahs, in the Olympic club, we might be watching the average cheetah versus like, that could be
like a really special cheetah that can do like.
- You're right. - When did the knees first come into play in you building legged robots? - [Marc] In BigDog.
- BigDog. - Yeah, BigDog came first
and then LittleDog was later. And you know, there's
a big compromise there. Human knees have multiple muscles and you could argue that there's, I mean, it's a technical thing about negative work when you're contracting a joint, but you're pushing out,
that's negative work. And if you don't have
a place to store that it can be very expensive
to do negative work. And in BigDog, there was no place to store
negative work in the knees. But BigDog also had pogo
stick springs down below. So part of the action was to
comply in a bouncing motion. Later on in Spot, we took that out and as we got further and
further away from the Leg Lab, we had more energy driven controls. - Is there something to
be said about like knees that go forward versus backward? - Sure, there's this idea
called passive dynamics, which says that although you
can use computers and actuators to make a motion, a mechanical system can
make a motion just by itself if it gets stimulated the right way. So Tad McGeer in the, I
think in the mid '80s, maybe it was in the late '80s, started to work on that and
he made this legged system that could walk down an incline plane where the legs folded and
unfolded and swung forward. Do the whole walking motion
where the only thing, there was no computer, there were some adjustments
to the mechanics so that there were dampers
and springs in some places that helped the mechanical
action happened. It was essentially a mechanical computer. And the idea, the interesting idea there is that it's not all
about the brain telling, dictating to the body
what the body should do, the body is a participant in the motion. - So, a great design for a
robot has a mechanical component where the movement is
efficient even without a brain. - Yes. - How do you design that? - I think that these days
most robots aren't doing that. Most robots are basically
using the computer to govern the motion. Now, the brain though
is taking into account what the mechanical thing can
do and how it's gonna behave otherwise it would have
to really forcefully move everything around all the time, which probably some solutions do, but I think you end up
with a more efficient and more graceful thing if
you're taking into account what the machine wants to do. - So this might be a good place to mention that you're now leading up the
Boston Dynamics AI Institute newly formed, which is focused more on designing
the robots of the future. I think one of the things, maybe you can tell me the big
vision for what's going on, but one of the things is this idea that hardware still matters with organic design and so on. Maybe before that, can you zoom out and tell me what the vision
is for the AI Institute? - You know, I like to
talk about intelligence having two parts: athletic
part and a cognitive part. And I think, you know,
Boston Dynamics, in my view has sort of set the standard for what athletic intelligence can be. And it has to do with all the things we've been talking about. The mechanical design,
the real time control, the energetics and that kind of stuff. But obviously people have
another kind of intelligence and animals have another
kind of intelligence. We can make a plan. Our meeting started at 9:30, I looked up on Google Maps, how long it took to walk over here. It was 20 minutes so I decided, okay, I'd leave my house at
9:00, which is what I did. You know, simple intelligence, but we use that kind
of stuff all the time. It's sort of what we think
of as going on in our heads. And I think that's in
short supply for robots. Most robots are pretty dumb. And as a result, it takes
a lot of skilled people to program them to do everything they do and it takes a long time. And if robots are gonna
satisfy our dreams, they need to be smarter. So the AI Institute is designed to combine that
physicality of the athletic side with the cognitive side. So for instance, we're
trying to make robots that can watch a human do a task, understand what it's seeing,
and then do the task itself. So sort of OJT, on-the-job
training for robots as a paradigm. Now that's pretty hard and
it's sort of science fiction, but our idea is to work
on a longer timeframe and work on solving
those kinds of problems. And I have a whole list of
things that are kind of like in that vein. - Maybe we can just take many
of the things you mentioned, just take it as a tangent. Okay, first of all, athletic intelligence
is a super cool term. And that really is intelligence. We humans kind of take it for granted that we're so good at walking
and moving about the world. - And using our hands, you know. - [Lex] Using your hands. - The mechanics of interacting
with all these parts, these two things. - And you've never touched
those things before. - Never touched, well I've
touched ones like this. - Okay.
(Lex laughing) - Look at all the things I can do, right? I can juggle, I'm rotating it this way, I can rotate it without looking, I could fetch these
things out of my pocket and figure out which one was which and all that kind of stuff. And I don't think we have much of a clue how all that works yet. - Right, and I really like
putting that under the banner of athletic intelligence. What are the big open problems
in athletic intelligence? So Boston Dynamics with Spot, with Atlas just have shown time and time again, like push the limits of what we think is possible with robots. But where do we stand, actually,
if we kind of zoom out, what are the big open problems on the athletic intelligence side? - I mean, one question you
could ask isn't my question, but, you know, are they
commercially viable? Will they increase productivity? - [Lex] Yeah. - And I think we're
getting very close to that. I don't think we're quite there still. Most of the robotics
companies, it's a struggle. You know, it's really the
lack of the cognitive side that probably is the biggest
barrier at the moment, even for the physical successful robots But your question's a good one. I mean, you can always do a
thing that's more efficient, lighter, more reliable. I'd say reliability. I know that Spot, they've
been working very hard on getting the tail of
the reliability curve up and they've made huge progress. So the robots, there's
1,500 of them out there now. Many of them being used
in practical applications day in and day out, where
they have to work reliably. And it's very exciting
that they've done that. But it takes a huge effort to get that kind of
reliability in the robot. There's cost too. You know, you'd like to get the cost down. Spots are still pretty expensive, and I don't think that they have to be, but it takes a different
kind of activity to do that. I think that Boston Dynamics is owned primarily by Hyundai now. And I think that the skills
of Hyundai in making cars can be brought to bear in making robots that are less expensive and more reliable and those kinds of things. - So on the cognizant
side for the AI Institute, what's the trade off between
moonshot projects for you and maybe incremental progress? - That's a good question. I think we're using the
paradigm called stepping stones to moonshots. I don't believe, and that
was in my original proposal for the institute, stepping
stones to moonshots. I think if you go more than a year without seeing a tangible
status report of where you are, which is the stepping stone, and it could be a simplification, right? You don't necessarily have
to solve all the problems of your target goal, even though your target goal
is gonna take several years. Those stepping stone
results give you feedback, give motivation, because usually there's
some success in there. And so, that's the mantra
we've been working on. And that's pretty much how I'd say Boston Dynamics has worked, where you make progress
and show it as you go, show it to yourself, if not to the world. - What does success look like? Like what are some of the
milestones you're chasing? - Well, we watched, understand, through the project I mentioned before. We've broken that down
into getting some progress with what is meaningfully
watching something mean, breaking down an observation
of a person doing something into the components. You know, segmenting, yeah, you watch me do something, I'm gonna pick up this thing and put it down here and stack this on it. Well, it's not obvious if you
just look at the raw data, what the sequence of acts are, it's really a creative
intelligent act for you to break that down into the pieces and understand them in
a way so you could say, "okay, what skill do I need to accomplish each of those things?" So we're working on the front
end of that kind of a problem where we observe and translate the... It may be video, it may
be live into a description of what we think is going on, and then try and map that into
skills to accomplish that. It may been developing skills as well. So, we have kind of multiple steps at the pieces of doing that. - And this is usually video
of humans manipulating objects with their hands kind of thing? - Mh-hm, we're starting
out with bicycle repair, some simple bicycle repair task. - Oh, no, that seems complicated. That seems really complicated. - It is, but there's some
parts of it that aren't like putting the seat in, into the- you have a tube that goes
inside of another tube and there's a latch, that
should be within range. - Is it possible to observe, to watch a video like this
without having an explicit model of what a bicycle looks like? - I think it is, and I think
that's the kind of thing that people don't recognize. Now, let me translate it to navigation. You know, I think the basic
paradigm for navigating a space is to get some kind of sensor that tells you where an
obstacle is and what's open. Build a map and then go through the space. But if we were doing on-the-job training where I was giving you a task, I wouldn't have to say
anything about the room, right? We came in here, all we
did is adjust the chair, but we didn't say anything about the room and we could navigate it. So I think there's opportunities to build that kind of
navigation skill into robots. And we're hoping to be able to do that. - So operate successfully
under a lot of uncertainty? - Yeah and lack of specification. - [Lex] Lack of specification. - I mean, that's what sort
of intelligence is, right? Kind of dealing with,
understanding a situation, even though it wasn't explained. - So how big of a role
does machine learning play in all of this? Is this more and more learning based? - You know, since ChatGPT,
which is a year ago, basically, there's a huge interest in that and a huge optimism about it. And I think that there's a lot
of things that machine learn that kind of machine learning. Now, of course there's
lots of different kinds of machine learning. I think there's a lot of
interest and optimism about it. I think the facts on the ground are that doing physical
things with physical robots is a little bit different than language. And the tokens, you know, the
tokens sort of don't exist. Pixel values aren't like words, but I think that there's a
lot that can be done there. We have several people working on machine learning approaches. I don't know if you know, but we opened an office
in Zurich recently, and Marco Hutter, who's
one of the real leaders in reinforcement learning for robots, is the director of that office. He's still halftime at the
ETH, the university there, where he has an
unbelievably fantastic lab. And then he's halftime leading, will be leading off efforts
in the Zurich office. So we have a healthy learning component but there's part of me that still says, if you look out in the world at what the most impressive
performances are, they're still pretty much... I hate to use the word "traditional" but that's what everybody's calling it, traditional controls, like
model predictive control. You know, the Atlas
performances that you've seen are mostly model predictive control. They've started to do some learning stuff that's really incredible. I don't know if it's all been shown yet, but you'll see it over time and then Marco has done
some great stuff and others. - So especially for the
athletic intelligent piece, the traditional approach seems to be the one that
still performs the best. - I think we're gonna
find a mating of the two, and we'll have the best of both worlds and we're working on that
at the institute too. - If I can talk to you about teams. So you've built an incredible
team of Boston Dynamics before at MIT and CMU at Boston Dynamics, and now at the AI Institute. And you said that there's four
components to a great team, technical fearlessness,
diligence, intrepidness and fun. Technical fun. Can you explain each? Technical fearlessness,
what do you mean by that? - Sure. Technical fearlessness means being willing to take on a problem that
you don't know how to solve. And, you know, study it. Figure out an entry point,
maybe a simplified version or a simplified solution or something. Learn from the stepping stone and go back and eventually make a solution
that meets your goals. And I think that's really important. - The fearlessness comes into play because some of it has
never been done before. - Yeah, and you don't know how to do it. And there's easier stuff to do in life. So, you know, I mean, I don't
know, watch, understand, do, it's a mountain of a challenge. - So that's the really big
challenge you're tackling now. Can we watch humans at scale and have robots by watching humans become effective actors in the world? - Yeah, I mean, we have others like that. We have one called inspect, diagnose, fix. Like you call up the Maytag repairman. Okay, he's the one who
you don't have to call, but you call up the
dishwasher repair person and they come to your house and they look at your machine. It's already been actually figured out that something doesn't work, but they have to kind of examine it and figure out what's
wrong and then fix it and I think robots should
be able to do that. Boston Dynamics already has Spot robots collecting data on machines,
things like thermal data, reading the gauges, listening
to them, getting sounds, and that data are used to determine whether
they're healthy or not but the interpretation isn't
done by the robots yet. And certainly the fixing,
the diagnosing and the fixing isn't done yet, but I think it could be. And that's bringing
the AI and combining it with the physical skills to do it. - Yeah, and you're referring to the fixing in the physical world. I can't wait until they can
fix the psychological problems of humans and show up
and talk, do therapy. - Yeah, that's a different thing. - Yeah, it's different, well, it's all part of the same thing. Again, humanity. (Lex chuckles) Maybe, maybe. - You mean convincing you it's okay that the dishwasher's broken? - Oh, yeah. (both laughing) - The marketing approach. - Yeah, exactly. Don't sweat the small stuff. Yeah, as opposed to fixing the dishwasher, it'll convince you that it's okay that the dishwasher's broken. It's a different approach. Diligence. Why is diligence important? - Well, if you want a real robot solution, it can't be a very narrow solution that's gonna break at the first variation in what the robot does or the environment if it wasn't exactly as you expected it. So how do you get there? I think having an approach that leaves you unsatisfied until you've embraced the bigger problem is the diligence I'm talking about. And again, I'll point at Boston Dynamics. I think they've done it. Some of the videos that we
had showing the engineer, making it hard for the
robot to do its task, Spot opening a door, and
then the guy gets there and pushes on the door so it doesn't open the
way it's supposed to, pulling on the rope that's
attached to the robot so it's navigation has been screwed up. We have one where the
robot's climbing stairs and an engineer is tugging on a rope that's pulling it back down the stairs. You know, that's totally different than just the robot seeing the stairs, making a model, putting its
feet carefully on each step but that's what probably
robotics needs to succeed. And having that broader, that broader idea that you wanna come up
with a robust solution is what I meant by diligence. - So really testing it in all conditions, perturbing the system
in all kinds of ways. - [Marc] Right. - And as a result,
creating some epic videos. the legendary-
- [Marc] The fun part. The hockey stick. - [Lex] And then yes, tugging on Spot as it's trying to open the door. I mean it's great
testing, but it's also... I don't know, it's just somehow extremely compelling demonstration of robotics in video form. - I learned something very early on with the first three
dimensional hopping machine. If you just show a video of
it hopping, it's a so what, if you show it falling
over a couple of times and you can see how easily
and fast it falls over, then you appreciate what the robot's doing when it's doing its thing. So I think the reaction
you just gave to the robot getting kind of interfered with or tested while it's
going through the door, it's showing you the scope of the solution - The limits of the system, the challenges involved in failure. Showing both failure and success makes you appreciate the success. - [Marc] Yeah. - And then just the
way the videos are done in Boston Dynamics are incredible 'cause there's no flash, there's
no extra like production, it's just raw testing of the robot. - Well, you know, I was the final edit for most of the videos up
until about three years ago, or four years ago. And my theory of the
video is no explanation. If they can't see it, then
it's not the right thing. And if you do something worth
showing, then let them see it. Don't interfere with a bunch
of titles that slow you down or a bunch of distractions. Just do something worth
showing and then show it. - [Lex] That's brilliant.
- It's hard. It's hard though for
people to buy into that. - Yeah, I mean, people
always wanna add more stuff, but the simplicity of just
do something worth showing and show it, that's brilliant.
And don't add extra stuff. - People have criticized,
especially the BigDog videos where there's a human driving the robot and I understand the criticism now. At the time we wanted to just show, look, this thing's using
its legs to get up the hill so we focused on showing that, which was, we thought the story, the
fact that there was a human, so they were thinking about autonomy, whereas we were thinking
about the mobility. And so, we've adjusted to a lot of things that we see that people care
about, trying to be honest. We've always tried to be honest. - But also just show cool
stuff in this raw form, the limits of the system, see the system be perturbed and be robust and resilient and all that kind of stuff and dancing with some music. Intrepidness and fun. So intrepid. - I mean, it might be the
most important ingredient. And that is, you know, robotics is hard. It's not gonna work right, right away so don't be discouraged
is all it really means. So usually when I talk about these things, I show videos and I show
a long string of outtakes, and you have to have
courage to be intrepid when you work so hard
to build your machine, and then you're trying and it just doesn't do what
you thought it would do what you want it to do. And you know, you have to
stick to it and keep trying. - How long, I mean...
we don't often see that, the story behind Spot and Atlas, how many failures was there
along the way to get, you know, a working Atlas, a working
Spot in the early days, even a working BigDog? - There's a video of Atlas
climbing three big steps and it's very dynamic and it's really exciting,
a real accomplishment. It took 109 tries and we have a video of every one of them, you know, we shoot everything. Again, this is at Boston Dynamics. So it took 109 tries, but once it did it, it had
a high percentage of success so it's not like we're cheating by just showing the best one, but we do show the evolved performance not everything along the way, but everything along
the way is informative and it shows sort of, there's, you know, stupid
things that go wrong. Like the robot just when you say "go" and it collapses right there on the start, that doesn't have to do with the steps or the perception didn't work right so you miss the target when
you jump or something breaks and there's oil flying
everywhere, but that's fun. - Yeah, so it's hardware failures and it maybe some software. - Lots of control of
evolution during that time. I think it took six weeks to
get that, those 109 trials, because there was programming going on. It was actually robot learning, but there were humans in the
loop helping with the learning. So all data driven, but... - Okay. And you always are learning
from that failure, so. - [Marc] Right. - And how do you protect
Atlas from not getting damaged from 109 attempts? - It's remarkable. One of the accomplishments of Atlas is that the engineers have made a machine that's robust enough that it
can take that kind of testing where it's falling and stuff and it doesn't break every time. It still breaks. And part of the paradigm is to
have people to repair stuff. You gotta figure that in if you're gonna do this kind of work. You know, I sometimes criticize the people who have their gold plated thing and they keep it on the shelf and they're afraid to kind of use it. I don't think you can make progress if you're working that way. You need to be ready to have it break and go in there and fix it. It's part of the thing,
you know, plan your budget so you have spare parts and
accrue and all that stuff. - Yeah, if it falls 109 times, it's okay. Wow. So intrepid truly, and
that applies to Spot, it applies to all the other robots. - It applies to everything. I think it applies to
everything anybody tries to do that's worth doing. - [Lex] Yeah. And especially with systems
in the real world, right? - [Marc] Yeah. - And so fun. - Fun, technical fun. I usually say "have technical fun." I think that life as an
engineer is really satisfying. I think you get to, you
know, to some degree, it can be like crafts work where you get to do
things with your own hands or your own design or
whatever your media is. And it's very satisfying to
be able to just do the work. Unlike a lot of people
who have to do something that they don't like doing. I think engineers typically
get to do something that they like and there's a
lot of satisfaction from that. Then there's, you know, in many cases you can have impact on the world somehow because you've done something
that other people admire, which is different from just the craft fun of building a thing. So that's the second way that
being an engineer is good. I think the third thing
is that if you're lucky to be working in a team where
you're getting the benefit of other people's skills that
are helping you do your thing. You know, none of us has
all the skills needed to do most of these projects. And if you have a team where you're working well with the others, that can be very satisfying and then if you're an engineer,
you also usually get paid. And so you kind of get paid four times in my view of the world. So what could be better than that? - Get paid to have fun. I mean, what do you
love about engineering? When you say engineering, what does that mean to you exactly? What is this kind of big thing
that we call engineering? - I think it's both being a scientist or getting to use science at the same time as being kind of an artist or a creator because you're making some- scientists only get to describe,
to study what's out there and engineers get to make
stuff that didn't exist before. And so it's really, I
think a higher calling. Even though I think most, you know, the public out there thinks science is top and engineering is somehow secondary, but I think it's the other way around. - And at the cutting edge, I think when you talk about robotics, there is a possibility to do art in that you do like the
first of its kind thing. So then there's the production at scale, which is its own beautiful thing. But when you do the first new robot or the first new thing, that's a possibility to
create something totally new. - Bringing metal to life or
a machine to life is kind of, is fun. And it was fun doing the dancing videos where we got a huge public response and we're gonna do more. We're doing some at the
institute, and we'll do more. - Well that metal to life
moment, I mean, to me that's still magical. Like when inanimate objects comes to life, that's still like, to me, to this day is still an incredible moment that human intelligence can create systems that instill life or whatever that is into inanimate objects. It's truly magical, especially when it's at the scale that humans can perceive and
appreciate like directly. - But I think sort of with it going back to the pieces of that, you design a linkage that
turns out to be half the weight and just as strong. That's very satisfying, yeah. You know, there are people who do that and it's a creative act. - What to you is most
beautiful about robotics? (Marc chuckles) Sorry for the big romantic question. - I think having the robots move in a way that's evocative of
life is pretty exciting. - [Lex] So the elegance of movement? - Yeah. Or if it's a high performance
act where it's doing it, faster, bigger than other robots, usually we're not doing it
bigger, faster than people, but we're getting there in
a few narrow dimensions. - So faster, bigger, smoother, more elegant, more graceful. - I mean, I'd like to
do dancing that starts, you know, we're nowhere near the dancing capabilities of a human. We've been having a ballerina in who's kind of a well-known ballerina and she's been programming the robot. We've been working on the
tools that can make it so that she can use her way of talking, way of doing a choreography
or something like that more accessible to get the robot do things and it's starting to produce
some interesting stuff. - Well, we should mention that
there is a choreography tool. - [Marc] There is. - I mean, I guess I saw versions of it, which is pretty cool. You can kind of, at slices of time control different parts at the high level, the movement of the robot, Spot. - We hope to take that
forward and make it more tuned to how the dance world wants to talk, wants to communicate and
get better performances. I mean, we've done a lot, but there's still a lot possible and I'd like to have performances where the robots are dancing with people. So right now, almost everything
that we've done on dancing is to a fixed time base. So once you press go, the robot does its thing
and plays out its thing. It's not listening, it's not watching but I think it should do those things. - I think I would love to
see a professional ballerina, like alone in a room with a robot, slowly teaching the robot,
just actually the process of a clueless robot trying to figure out a small little piece of a dance. So it's not like, because right now Atlas and Spot have done
like perfect dancing. - [Marc] Right. - To a beat and so on, to a degree, but like the learning process of interacting with
the human would be like incredible to watch. - One of the cool things
going on, you know, that there's a class at Brown University called Choreo Robotics. Sydney Skybetter is a
dancer, choreographer, and he teamed up with Stephanie Tellex, who's a computer science professor, and they taught this class and I think they have some
graduate students help him teach it where they have two Spots and people come in. I think it's 50-50 of computer science
people and dance people. And they program performances
that are very interesting. I show some of them
sometimes when I give a talk. - And making that process of
a human teaching the robot more efficient, more intuitive, maybe language, part movement,
that'd be fascinating. That'd be really
fascinating because I mean, one of the things I've kind of realized is humans communicate with movement a lot. It's not just language. There's body language,
there's so many intricate little things and- - [Marc] Totally. - And like that, to watch a
human and Spot communicate back and forth with movement. I mean, there's just so many
wonderful possibilities there. - Yeah, but it's also a challenge. You know, we get asked to
have our robots perform with famous dancers and they can, they have 200 degrees
of freedom or something. Every little ripple and thing and they have all this head and neck and shoulders and stuff. And the robots mostly
don't have all that stuff. And it's a daunting
challenge to not look stupid, you know, physically stupid next to them. So we've pretty much avoided
that kind of performance, but we'll get to it. - I think even with the
limited degrees of freedom, we could still have some
sass and flavor and so on. You can figure out your own thing. Even if you can't.
- Yeah. And we can reverse things. Like if you watch a
human do robot animation, which is a dance style where
you jerk around sort of, and you pop and pop and
lock and all that stuff. I think the robots
could show up the humans by doing unstable oscillations and things that are
faster than a person can. So that's sort of on my plan but we haven't quite gotten there yet. - You mentioned about building teams and robotics teams and so on. How do you find great engineers? How do you hire great engineers? - I think you even need
to have an environment where interesting engineers, well, you know, it's a chicken egg. If you have an environment
where interesting engineering is going on, then
engineers wanna work there. And you know, I think it took a long time to develop that at Boston Dynamics. In fact, when we started, although, I had the
experience of building things in the Leg Lab, both at CMU and at MIT, we weren't that sophisticated
of an engineering thing compared to what Boston Dynamics is now. But it was our ambition to do that. And you know, Sarcos was
another robot company, so I always thought of us as being this much on the computing side and this much on the hardware
side and they were like this. And then over the years we, you know, I think we achieved the same or better levels of engineering. Meanwhile, you know, Sarcos got acquired and then they went through
all kinds of changes and I don't know exactly
what their current status is, so it took many years
is part of the answer. I think you gotta find people who love it. In the early days, we paid a little less so we only got people who were doing it because they really loved it. We also hired people who might not have professional degrees, people who were building
bicycles and building kayaks. We have some people who come from that kind of the maker world and that's really important
for the kind of work we do to have that be part of the mix, - Whatever that is. Whatever the magic ingredient that makes a great builder
maker, that's the big part of it. - People who repaired
the cars or motorcycles or whatever in their
garages when they were kids. - There's a kind of like
the robotics students, grad students and just
roboticists that I know and I hangout with, there's
a kind of endless energy and like, they're just happy. Like say I compare another
group of people that are like, people that skydive professionally. There's just like
excitement in general energy that I think probably
has to do with the fact that they're just constantly,
first of all, fail a lot. And then the joy of building a thing that eventually works. - Yeah, talking about being
happy, there used to be a time when I was doing the
machine shop work myself back in those JPL and Caltech days when if I came home smelling
like the machine shop, you know, 'cause it's an oily place my wife would say, "oh,
you had a good day today" because you could tell
that that's where I'd been. - You've actually built something, you've done something
in the physical world. Yeah, and probably the videos help, right? The videos help show off what robotics is. - Oh, you know, at Boston
Dynamics, it put us on the map. I remember interviewing some sales guy and he was from a company and he said, "well, no one's ever heard of my company, but we have products,
really good products. You guys, everybody knows who you are, but you don't have any products at all" which was true. And we thank YouTube for that. YouTube came, we caught the YouTube wave and it had a huge impact on our company. - I mean, it's a big impact
on not just on your company, but on robotics in general
and helping people understand, inspire what is possible with robots. - [Marc] Yeah.
- Inspire imagination, fear and everything, the full
spectrum of human emotion was aroused which is great, which is great for the
entirety of humanity and also it's probably
inspiring for young people that want to get into AI and robotics. Let me ask you about some competitors. - [Marc] Sure. - You've been complimentary
of Elon and Tesla's work on Optimus Robot with
their humanoid robot. What do you think of their efforts there with the humanoid robot? - You know, I really admire
Elon as a technologist. I think that what he did with Tesla, it was just totally mind boggling that he could go from
this totally niche area that less than 1% of anybody
seemed to be interested to making it so that
essentially every car company in the world is trying
to do what he's done so you gotta give it to him. Then look at SpaceX, you know,
he's basically replaced NASA if you could, that might
be a little exaggeration, but not by much. So you gotta admire the guy and I wouldn't count him out for anything. I don't think Optimist
today is where Atlas is, for instance. I don't know, it's a
little hard to compare 'em to the other companies. You know, I visited Figure. I think they're doing well
and they have a good team. I've visited Apptronik and I
think they have a good team and they're doing well. But Elon has a lot of resources.
He has a lot of ambition. I'd like to take some credit
for his ambition, I think. I think if I read between the lines, it's hard not to think that him seeing what Atlas is doing is a
little bit of an inspiration. I hope so. - Do you think Atlas and
Optimus will hang out at some point? - I would love to host that, you know. Now that I'm not at Boston Dynamics, I'm not officially connected. I am on the board, but I'm
not officially connected. I would love to host a- - [Lex] Robot meetups? - A robot meetup, yeah. - Does the AI Institute
work with Spots and Atlas's? Is it focused on Spots mostly
right now as a platform? - We have a bunch of different robots. We bought everything we could buy. So we have Spots. I think we have a good size fleet of them. I don't know how many it
is, but a good size fleet. We have a couple of ANYmal robots. You know, ANYmal is a company
founded by Marco Hutter, even though he's not that involved anymore but we have a couple of those. We have a bunch of arms
like, you know, Franca's and US Robotics 'cause you know, even
though we have ambitions to build stuff and we are
starting to build stuff, you know, day one getting off the ground, we just bought stuff. - I love this like robot
playground you've built. - Yeah, you can come over
and take a look if you want. - That's great. So it's like all these kinds
of robots, legged, arms. - It doesn't feel that much like it, well, there's some areas
that feel like a playground, but it's not like they
all frolic together. - Sure.
(both laughing) Hey, again, maybe you'll
arrange it a robot meetup. But in general, what's
your view on competition in this space for especially like humanoid and legged robots? Are you excited by the competition or the friendly competition? - I think that it doesn't... You know, I don't think
about competition that much. I'm not a commercial guy. I think for many years at Boston, the many years I was at Boston Dynamics, we didn't think about competition. We were just kind of doing our thing. It wasn't like there
were products out there that we were competing with. Maybe there was some
competition for DARPA funding, which we got a lot of,
got very good at getting, but even there, in a couple of cases where we might have competed, we ended up just being the robot provider that is for the LittleDog program. We just made the robots, we
didn't participate as developers except for developing the robot. And in the DARPA robotics
challenge, we didn't compete. We provided the robots. So in the AI world now, now that we're working on cognitive stuff, it feels much more like a competition. The entry requirements in
terms of computing hardware and the skills of the
team and hiring talent, it's a much tougher place. So I think much more about competition now on the cognitive side. On the physical side, it doesn't feel like it's that much about competition yet. Obviously with 10 humanoid
companies out there, 10 or 12, I mean, there's probably others that I don't know about. They're definitely in competition,
will be in competition. - How much room is there for quadruped and especially a humanoid
robot to become cheaper? So like cutting costs and
like how low can you go and how much of it is
just mass production? So questions of, you know, Hyundai, like how to produce versus
like engineering innovation, how to simplify it. - I think there's a huge way to go. I don't think we've seen the bottom of it or the bottom internal's lower prices. I think you should be totally optimistic that at asymptote things
don't have to be anything like as expensive as they are now. Back to competition, I
wanted to say one thing. I think in the quadruped space, having other people selling quadrupeds is a great thing for Boston Dynamics because the question, I believe the question
in the user's minds is, "which quadruped do I want?" It's not, "oh, do I want a quadruped? Can a quadruped do my job?" It's much more like that, which is a great place for it to be. - [Lex] Yeah. - Then you're just, you know, doing the things you normally do to make your product
better and compete, selling and all that stuff. And that'll be the way it is
with humanoids at some point. - Well, there's a lot of humanoids and you're just not even, it's like- iPhone versus Android and people just buying both
and it's kind of just... - [Marc] Yeah. You're creating the category. Or the category is happening. I mean, right now, the use
cases, that's the key thing. Having realistic use cases
that are money making in robotics is a big challenge. You know, there's the warehouse use case. That's probably the only thing that makes anybody any money
in robotics at this point. - There's gotta be a moment. - There's old fashioned robots. I mean, there's fixed
arms doing manufacturing. I don't wanna-
- [Lex] Yes. - Say that they're not making money. - Industrial robotics, yes. But there's gotta be a
moment when social robotics starts making real money,
meaning like a Spot type robot in the home and there's tens
of millions of them in the home and they're like, you know, I don't know how many dogs
there are in the United States as pets. - [Marc] Many. - It feels like there's something we love about having a intelligent
companion with us that remembers us, that's excited to see us,
all that kind of stuff. - But it's also true that the
company's making those things there've been a lot of failures
in recent times, right? There's that one year when I
think three of 'em went under. So it's not that easy to do that, right? Getting performance, safety and cost all to be where they need
to be at the same time is... that's hard. - But also some of it is, like you said, you can have a product, but
people might not be aware of it. So like, also part of it is the videos or however you connect with
the public, the culture and like, create the category, make people realize this
is the thing you want because there's a lot of negative
perceptions you can have. Do you really want a system
with a camera in your home walking around? If it's presented correctly and if there's like the right
kind of boundaries around it that you understand
how it works and so on. A lot of people would want
to, and if they don't, they might be suspicious of
it so that's an important one. Like, we all use smartphones and that has a camera
that's looking at us. - Yeah, it has two or three or four. - And it's listening, very few people are suspicious about it. They kind of take it for granted and so on and I think robots would
be the same kind of way. - I agree. - So as you work on the
cognitive aspect of these robots, do you think we'll ever get to human level or super human level intelligence? There's been a lot of
conversations about this recently, given the rapid development
in large language models. - I think that intelligence
is a lot of different things. And I think some things, computers are already smarter than people and some things they're not even close. And, you know, I think you'd need a menu of detailed categories
to come up with that. But I also think that the conversation that seems to be happening
about AGI puzzles me. It's sort of, so I ask you a question, do you think there's anybody
smarter than you in the world? - Absolutely, yes. - Do you find that threatening? - No. - So I don't understand, even if computers were
smarter than people, why we should assume that that's a threat, especially since they
could easily be smarter, but still available to
us or under our control, which is basically how
computers generally are. - I think the fear is that they would be 10x, 100x smarter and operating under different
morals and ethical codes than humans like naturally do. And so almost become
misaligned in unintended ways and therefore harm humans in
ways we just can't predict. And even if we program them to do a thing on the way of doing that thing, they would cause a lot of harm. And when they're 100 times, 1,000 times, 10,000 times smarter than us,
we won't be able to stop it or won't be able to even see the harm as it's happening until it's too late. That kind of stuff. So you can construct all kinds of like, possible trajectories
of how the world ends because of superintelligent systems. - Okay. It's a little bit like that
line in the "Oppenheimer" movie where they contemplate
whether the first time they set off a reaction,
all matter on earth is gonna go up. I don't remember what
the verb they used was for the chain reaction, right? - [Lex] Mh-hm. - Yeah, I guess it's
possible, but I don't think... I personally don't think it's
worth worrying about that. I think that it's an opportunity, balancing opportunities in risk. I think if you take any technology, there's opportunity in risk
and it's easy to point, I'll point at the car, they
pollute and about what? 1.25 million people get killed every year around the world because of them. Despite that, I think
they're a boon to humankind, very useful. Many of us love them and those technical
problems can be solved. I think they're becoming safer. I think they're becoming less polluting, at least some of them are and every technology you can name has a story like that, in my opinion. - What's the story behind
the Hawaiian shirt? Is it a fashion statement,
philosophical statement? Is it just a statement of rebellion? Engineering statement? - It was born of me being a contrarian. - [Lex] Yes. So it's a symbol. - Someone told me once
that I was wearing one when I only had one or two and they said, "oh, those things are so old fashioned, you can't wear that Marc." And I stopped wearing
them for about a week. - [Lex] Yeah.
- And then I said, "I'm not gonna let them
tell me what to do." And so every day since pretty much. That was years ago, that was 20 years ago, 15 years ago probably. - That says something
about your personality. That's great. - It took me a while to
realize I was a contrarian, but you know, it can be a useful tool. - Have you had people tell
you on the robotics side that like, "I don't
think you could do this." The kind of negative motivation? - I'd rather talk about, there's a guy, when we were doing a lot of
DARPA work, there was a Marine Ed Tovar who's still around, what he would always say
is when someone would say, "oh, you can't do that"
he'd say, "why not?" - [Lex] Yeah.
(Lex chuckles) - And it's a great
question I ask all the time when I'm thinking, oh,
we're not gonna do that. And I say, "why not?" And I give him credit for opening my eyes to resisting that.
- [Lex] I love it. So yeah, yeah. The Hawaiian shirt is almost
like a symbol of why not. Okay. What advice would you give to young folks that are trying to figure out what they wanna do with their life? How to have a life they can be proud of? How they can have a career
they can be proud of? - When I was teaching at MIT, for a while, I had undergraduate advisees where people would have to meet with me once a semester or something
and they frequently would ask, what they should do. And I think the advice I used
to give was something like, well, if you had no constraints on you, no resource constraints,
no opportunity constraints and no still skill constraints, what could you imagine doing? And I said, well start there and see how close you can
get to what's realistic, for how close you can get. You know, the other version of that is, try and figure out what
you wanna do and do that because I don't think a lot of people think that
they're in a channel right and there's only limited opportunities, but it's usually wider than they think. - Yeah, the opportunities
really are limitless, but like at the same time,
you want to pick a thing right, and it's the diligence. And really, really pursue it, right? And really pursue it. - [Marc] Yeah. - Because sometimes like the
really special stuff happens after years of pursuit. - Yeah. Oh absolutely, it can take a while. - I mean, you've been doing
this for 40 plus years. - Some people think I'm in a rut. And in fact, some of the
inspiration for the AI Institute is to say, "okay, I've
been working on locomotion for however many years it
was, let's do something else." And it's a really fascinating
and interesting challenge. - And you're hoping to show it off also in the same way as Boston Dynamics? - Just about to start
showing some stuff off, yeah. I hope we have a YouTube channel. I mean, one of the challenges is it's one thing to show
athletic skills on YouTube showing cognitive function is a lot harder and I haven't quite figured
out yet how that's gonna work. - [Lex] There might be a way. - There's a way.
- [Lex] There's a way. - Why not? - I also do think sucking at
a task is also compelling. Like the incremental improvement. A robot being like
really terrible at a task and then slowly becoming better. Even in athletic intelligence, honestly, like learning to walk and falling and slowly figuring that out. I think there's something
extremely compelling about that. - [Marc] You're right. - We like flaws, especially
with the cognitive tasks. It's okay to be clumsy.
It's okay to be confused and a little silly and
all that kind of stuff. It feels like in that
space is where we can... - There's charm. - There's charm. There's charm and there's
something inspiring about a robot sucking and then becoming less
terrible slowly at a task. - No, I think you're right. - That kind of reveals
something about ourselves and ultimately that's one of
the coolest things about robots is it's kind of a mirror about
what makes humans special. Just by watching how hard it is to make a robot do the
things that humans do, you realize how special we are. - [Marc] Mh-hm. - What do you think is the
meaning of this whole thing? Why are we here? Marc, do you ever ask
about the big questions as you try to create these humanoid human-like intelligence systems? - I don't know. I think you have to have
fun while you're here. That's about all I know. It would be a waste not to, right? - The ride is pretty short,
so might as well have fun. Marc, I'm a huge fan of yours. It's a huge honor that
you would talk with me. This is really amazing and your work for many
decades has been amazing. I can't wait to see what
you do at the AI Institute. I'm gonna be waiting impatiently for the videos and the demos
and the next robot meetup for maybe Atlas and Optimus to hang out. - I would love to do
that. That would be fun. - Thank you so much for talking. - Thank you, it was fun talking to you. - Thanks for listening
to this conversation with Marc Raibert. To support this podcast, please check out our
sponsors in the description. And now let me leave you with some words from Arthur C. Clarke. "Whether we're based
on carbon or on silken makes no fundamental difference. We should each be treated
with appropriate respect." Thank you for listening and
hope to see you next time.