Ben Goertzel: Artificial General Intelligence | Lex Fridman Podcast #103
the following is a conversation with Ben Gerel one of the most interesting Minds in the artificial intelligence Community he's the founder of Singularity net designer of opencog AI framework formerly a director of research at the machine intelligence Research Institute and chief scientist of Hansen robotics the company that created the Sophia robot he has been a central figure in the AGI Community for many years including in his organizing and contributing to the conference and artificial general in Ence the 2020 version of which is actually happening this week Wednesday Thursday and Friday it's virtual and free I encourage you to check out the talks including by yosa Bak uh from episode 101 of this podcast quick summary of the ads two sponsors the Jordan Harbinger show and Master Class please consider supporting this podcast by going to Jordan Harbinger docomo a masterclass.com Lex click the links buy all the stuff it's the best way to support this podcast and the journey I'm on in my research and startup this is the artificial intelligence podcast if you enjoy it subscribe on YouTube review it with five stars on Apple podcast support on patreon or connect with me on Twitter Alex fredman spelled without the e just f r i d m as usual I'll do a few minutes of as now and never any ads in the middle that can break the flow of the conversation this episode is supported by the Jordan Harbinger show go to Jordan Harbinger dcom Lex it's how he knows I sent you on that page there's links to subscribe to it on Apple podcast Spotify and everywhere else I've been binging on his podcast Jordan is great he gets the best out of his guests Dives deep calls them out when it's needed and makes the whole thing fun to listen to he's interviewed Kobe Bryant Mark Cuban Neila grass Tyson gri Kasparov and many more more this conversation with Kobe is a reminder how much focus and hard work is required for greatness and Sport business and life I highly recommend the episode if you want to be inspired again go to Jordan har.com Lex it's how Jordan knows I sent you this Show sponsored by masterclass sign up at masterclass.com Lex to get a discount and to support this podcast when I first heard about master class I thought it was too good to be true for 180 bucks a year you get an all access pass to watch courses from to list some of my favorites Chris Hatfield on space exploration Neil degrass Tyson on scientific thinking and communication will WR creator of the greatest city building game ever Sim City and Sims on game design Carlos Santana on guitar grick Kasparov the greatest chess player ever on chess Daniel Neo on poker or many more Chris Hatfield explaining how rockets work and the experience of being launched into space alone is worth the money once again sign up on masterclass.com Lex to get a discount and to support this podcast and now here's my conversation with Ben gitzo would books authors ideas had a lot of impact on you um in your life in the early days you know what got me into Ai and science fiction and such in the first place wasn't a book but the original Star Trek TV show which my dad watched with me like in its first run it would have been 1968 69 or something and that that was incredible CU every every show they visited a different a different alien civilization with different culture and weird mechanisms but that that got me into science fiction and there wasn't that much science fiction to watch on TV at that stage so that got me into reading the whole the whole literature of Science Fiction you know from from the beginning of the previous Century until that time and the I mean there was so many science fiction writers who were in inspirational to me I'd say if I had to pick two it would have been H Stanis LM the the Polish writer yeah Solaris and then he had he had a bunch of more obscure writings on on superhuman AIS that were engineered Solaris was sort of a superhuman naturally occurring in intelligence than Philip K dick who you know ultimately my fandom for Philip K dick is one of the things that brought me together with David Hansen my collaborator on on on robotics project so you know Stannis slm was was very much an intellectual right so he he had a very broad view of intelligence going beyond the human and into what I would call you know open-ended super intelligence the the Solaris super inell ocean was intelligent in some ways more generally intelligent than people but in a complex and confusing way so that human beings could never quite connect to it but it was but it was still palpably very very smart and then the the what Golem four supercomputer in one of one of lm's lm's books this was engineered by people but eventually it became very intelligent in a different direction than humans and decided that humans were kind of trivial and not that interesting so it it put some impenetrable shield around itself shut itself off from humanity and then issued some philosophical screed about the pathetic and hopeless nature of of humanity and and all human thought and then and then disappeared now Philip K dick he was a bit different he was human focused right his main thing was you know human compassion and the human heart and soul are going to be the constant that will keep us going through whatever aliens aliens we discover or telepathy machines or or or super AIS or or or whatever it might be so he didn't believe in reality like the reality that we see may be a simulation or or or a dream or or something else we can't even comprehend but he believed in love and compassion is something persistent through the various simulated realities so those those two science fiction writers had had a huge impact on me then a little older than that I got into dovi and friedi n and rambod and a bunch of uh more more literary type writing we talk about some of those things so on the Solara side stof lamb uh this kind of idea of they being intelligences out there that are different than our own do you think there are intelligences maybe all around us that were not able to even detect so this kind of idea of uh maybe you can comment also on Steven Wolfram thinking that there's computations all around us and we're just not smart enough to kind of detect their their intelligence or appreciate their intelligence yeah so my friend Hugo degaris who I've been talking to about these things for for for many decades since the early 90s he had an idea he called SII the search for intraparticulate intelligence so the concept there was as AIS get smarter and smarter and smarter you know assuming the laws of physics as we know them now are still are still what these super intelligences perceived to hold and are bound by as they get smarter and smarter they're going to shrink themselves little and little because special relativity make makes it to sort of communicate between two specially distant points so they're going to get smaller and smaller but then ultimately what does that mean the minds of the super super super intelligences they're going to be packed into the the interaction of of Elementary particles or quirks or the partons inside quirks or whatever it is so what we perceive as random fluctuations on the quantum or subquantum level may actually be the thoughts of the micro micro micro miniaturized super intelligences because there's no way we can tell random from structured but with an algorithmic information more complex than our brains right we can't tell the difference so what we think is random could be the thought processes of some really tiny super minds and if so there's not a damn thing we can do about it except you know try to upgrade our intelligences and expand our mind so that we can we can perceive more of what's around us but if th if those random fluctuations like even if we go to like quantum mechanics if that if that's actually uh super intelligent systems aren't we then part of the soup of super intelligence that we're aren't we just like like a finger of the entirety of the body of the superintelligence system we could be I mean a finger is a is a strange metaphor I mean we we we a finger is dumb is what I mean is uh is but a finger is also useful and is controlled with with intent by by the brain where we may be much less than that right I mean I mean yeah we may be just some random EPA phenomenon that that they don't care about too much like think about the the shape of the crowd emanating from a sports Stadium or something right there there's some Emer shape to the crowd it's there you could take a picture of it it's kind of cool it's irrelevant to the main point of the sports event or where the people are going or or or or what's on the minds of the people making that shape in the crowd right so we we may just be some semi arbitrary higher level pattern popping out of of a lower level hyper intelligent self-organization and I'm me so so be it right I mean that's one thing that still a fun ride yeah I mean the older I've gotten the more respect I've achieved for our fundamental ignorance I mean M mine and everybody else's I I look at my my two dogs two beautiful little toy poodles and you know they watch me sitting at the computer typing they just think I'm sitting there wiggling my fingers to exercise them maybe or guarding the monitor on the desk that they have no idea that I'm communicating with other people Halfway Around the World let let alone you know creating complex algorithms running in in RAM on some computer server in St Petersburg or something right they although they're right there they're right there in the room with me so what things are there right around us that we just too stupid or close-minded to comprehend probably probably quite a lot you're very your very poodle could be uh could also be communicating across multiple Dimensions with with other with other beings and you're too you're too unintelligent to understand the kind of communication mechanism they're going through there there there have been various uh TV shows and science fiction novels pusing cats Dolphins uh mice and whatnot are actually super intelligence is here to observe that I I would I would guess as one of the other quantum physics Founders said those theories are not crazy enough to be true the reality is probably crazier than that beautifully put so on The Human Side uh with uh Philip K dick and and uh in general where do you fall on this idea that uh love and just the basic Spirit of human nature persists throughout these multiple realities um are you on the side like the thing that inspires you about artificial intelligence is it the human side of somehow persisting through all of the different systems we engineer or is it or is AI inspire you to create something that's greater than human that's beyond human that's almost nonhuman I would say my motivation to create AGI comes from from both of those directions actually so when I when I first became passionate about AGI when I was it would have been two or three years old after watching robots on Star Trek I mean then it was really a combination of intellectual curiosity like can a machine really think how how would you do that and yeah just ambition to create something much better than all the clearly limited and and fundamentally defective humans I saw around me then as I got older and got more in mesed in in in the human world and you know got married had children some my parents begin to age I started to realize well not only will AGI let you go far beyond the limitations of the human but it could also like stop us from dying and and suffering and and and feeling pain and and tormenting ourselves mentally you can see AGI has amazing capability to do good for humans as humans alongside with with its capability to go far far beyond the human level so I mean both both aspects are are there which makes it uh even more exciting and important so you mentioned the what did you pick up from those guys I mean that that that would probably go beyond the beyond the scope of of a brief interview certainly I mean both of those are amazing thinkers who one will necessarily have a a complex relationship with right so I mean dovi on the on the minus side he's kind of a religious fanatic and he sort of helped squash the Russian nihilist movement which was very interesting because what what nihilism meant originally in in in that period of the mid late 1800s in Russia was not not taking anything fully 100% for granted it was really more like what we'd call beanis now where you don't want to adopt anything as a dogmatic certitude and always leave mind open and how dovi parody nihilism was was was was a bit different right he parody is people who believe absolutely nothing so they M they must assigned an equal probability weight to to every proposition which which which doesn't really work so on the one hand I I didn't really agree with dovi on on his sort of religious point of view on on on the on the other hand if you look at his understanding of human nature and sort of the human mind and and and heart and and soul it's it's it's really unparalleled and he had an amazing view of how human beings you know construct a world for themselves based on their own understanding and and their own mental predisposition and I think if if you look in the brothers karamazov in particular the the Russian literary theorist M Bakin wrote about this as a polyphonic mode of fiction which means it's not third person but it's not first person from any one person really there are many different characters in the novel and each of them is sort of telling part of the story from their own point of view so the reality of the whole story is is an intersection like synergetically of the many different characters worldviews and that really it's a beautiful metaphor and even a reflection I think of how all of us socially create our reality like each of us sees the world in a certain way each of us in a sense is making the world as we see it based on on our own minds and understanding but it's poony like like in like in music where multiple instruments are coming get coming together to create the sound the Ultimate Reality that's created comes out of each of our subjective understandings you know intersecting with each other and that that was one of the many beautiful things in in DVI so maybe a little bit to mention you have a connection to Russia and the Soviet culture I mean I'm not sure exactly what the nature of the connection is but there at least the spirit of your thinking well my my my ancestry is three4 Eastern European Jewish so I mean my three of my great-grandparents immigrated to New York from Lithuania and sort of Border regions of of Poland which were in and out of Poland in around the around the time of world World War I and they were they were socialists and and Communists as well as Jews mostly menic not not Bolshevik and they sort of they fled at just the right time to the US for their own personal reasons and then almost all or maybe all of my extended family that remained in Eastern Europe was killed either by hitlin or or Stalin's minions at some point so the branch of the family that immigrated to the US was was was pretty much the the only one right so how much of the spirit of the people is in your blood still like do you when you look in the mirror do you see uh what do you see meat I see a bag of meat that I want to transcend by uploading into some sort of superior reality but very yeah I mean yeah very clearly well I mean I'm I'm not religious in a traditional sense but clearly the the Eastern European Jewish tradition was what I what I was raised in I there was my grandfather Leo well was a a physical chemist who worked with Lis Pauling and a bunch of the other early greats and in quantum mechanics I mean he was he was uh into x-ray defraction he was on the material science side experimentalist rather than a theorist his sister was was also a physicist and my my father's Father Victor gzel was a PhD in in Psychology who had the unenviable job of giving Psychotherapy to the Japanese jaes in internment camps in the US in in in World War II like to counsel them why they shouldn't kill themselves even though they'd had all their stuff taken away and been imprisoned for no good reason so I mean there yeah there there's a lot of uh Eastern European jewishness in my in my background one of my great uncles was I guess conductor of San Francisco Orchestra so there there's a lot of Mickey Sul and bunch of music music in there also and clearly this culture was all about learning and and understanding understanding the world and also not quite taking yourself too seriously while you do it right there's a lot of Y Yiddish humor in there so I I do appreciate that that culture although the whole idea that like the Jews are the chosen people of God never resonated with me too much the graph of the gzel family I mean just the people I've encountered just doing some research and just knowing your work through through the decades uh it's kind of fascinating I'm just the the the number of phds yeah yeah I mean f my dad is a sociology Professor who recently retired from from ruers University but that clearly that gave me a head start in life I mean my my grandfather gave me all his quantum mechanics books when I was like seven or eight years old you know I remember going through them and it was all the old Quant mechanics like r Rutherford Adams and stuff so I got to the part of wave functions which I didn't understand although I was very bright kid and I realized he he didn't quite understand it either but at least like he pointed me to some Professor he knew at at upen nearby who who understood these things right so that's that that's an unusual opportunity for a kid to have right and my my dad he was programming for tram when I was 10 or 11 years old on like HP 3000 Main frames at ruers University so I got to do linear regression in Fortran on on Punch Cards at when when I was in in in middle school right because he was doing I guess analysis of demographic and and and sociology data so yes certainly certainly that gave me a head start and a push towards science be beyond what would have been the case with many many different situations when did you first fall in love with AI is it the is it the programming side of Fortran is it the maybe the sociology psychology that you picked up from your dad or I when I was probably 3 years old when I saw a robot on Star Trek it was turning around in a circle going eror error error error because Spock and Kirk had tricked into a mechanical breakdown by presenting with a logical Paradox and I was just like well this makes no sense this AI is very very smart It's been traveling all around the universe but these people could trick it with a simple logical Paradox like what if you know if the human brain can get beyond that Paradox wh wh why why can't why can't can't this AI so I I I felt the the screenwriters of Star Trek had misunderstood the nature of intelligence and I complained to my dad about it and he he wasn't he wasn't going to say anything one way or the other but you know in before I was born when my dad was at Antioch College in uh in the middle of the US he he led uh he led a protest movement called slam student League against mortality they were protesting against death wandering across the campus so he he he he was into some futuristic things even back then but whether AI could confront logical paradoxes or not he did he didn't know but that you know when I 10 years after that or something I discovered Douglas Hoffer's book gordal shabbach and that was sort of to the same point of AI and Paradox and logic right because he was over with over and over with gle's incompleteness theorem and Canon AI really fully model itself reflexively or does that lead you into some Paradox can the human mind truly model itself reflexively or does that lead you into some Paradox so when I think that book Gord lerach which I think I read when it first came out it would have been 12 years old or something I remember it was like 16-hour day I read it cover to cover and then ReRe it ReRe it I reread it after that because there was a lot of weird things with little formal systems in there that were hard for me at the time but that was the first book I read that gave me a feeling for AI as like a practical academic or engineering discipline that that people were working in because before I read Gord shach I was into AI from the point of view of a of a science fiction fan and I I had the idea well it may be a long time before we can achieve immortality in superhuman AGI so I should figure out how to build a spacecraft traveling close to the speed of light go far away then come back to the Earth in a Million years when technology is more advanced and we can build these things reading G shach well it didn't all ring true to me a lot of it did and but I could see like there are smart people right now at various universities around me who are actually trying to work on building what I would Now call AGI although Hoff didn't didn't call it that so really it was when I read that book which would have been probably Middle School that then I started to think well this this is something that I could I could practically work supposed to flying away and waiting it out you can actually be the one of the people that actually uh EXA and if you think about I I was interested in what we'd Now call nanotechnology and in the human immortality and time travel all the same cool things as every other like science fiction loving kid but AI seemed like if Hoff did it was right you just figure out the right program sit there and type it like you don't you don't need to you don't need to spin Stars into weird configurations or get government approval to cut people up and Fiddle with their DNA or something right it's just programming and then of course that can achieve anything else that there's another book from back then which was by fine bomb Gerald fbom who was a who was a physicist at at at Princeton and that was the Prometheus project and this book was written in the late 1960s though I encountered it in the mid '70s but what this book said is in the next few decades humanity is going to create superhuman thinking machines molecular nanotechnology and human immortality and then the challenge we'll have is what to do with it do we use it to expand human consciousness in a positive direction or or do we use it just to further vapid uh consu consumerism and what he proposed was that the UN should do a survey on this and the UN should send people out to every little village in in remotest Africa or South America and explain to everyone what technology was going to bring the next few decades and the choice that we had about how to use it and let everyone on the whole planet vote about whether we should develop you know super AI nanotechnology and and and immortality for expanded Consciousness or for rampant rampant consumerism and needless to say that didn't quite happen and I think this guy died in the mid 80s so he didn't even see his ideas start to become become more mainstream but it's interesting many of the themes I'm engaged with now from AGI and immortality even to trying to democratize technology as I've been pushing for with Singularity my work in the blockchain world many of these themes were there in you know fine bomb's book in uh in the late 60s even and of course Valentin churchin uh a Russian writer who who I and a great Russian physicist who I got to know when we both lived in New York in the late 90s and early Arts I mean he he had a book in the late 60s in in Russia which was the phenomenon of science which laid out laid out all these all these same things as well and Val died in I don't remember 2004 five or something of parkinsonism so yeah it's easy easy for people to lose track now of the fact that the the futurist and singularitarianism almost mainstream and they're on TV all the time I mean these these are not that new right they're sort of new in the history of the human species but I mean these were all around in Fairly mature form in in the middle of the last century were written about quite articulately by fairly mainstream people who are professors at at top universities it's just until the enabling Technologies got to a a certain point then you you couldn't make it real so and even in the 70s I was sort of seeing that and and living living through it right from Star Trek to Douglas Hoffer things were getting very very practical from the late 60s to the late 70s and you know the first computer I bought you could only program with heximal machine code and you had to solder it together and then then like a few years later there's Punch Cards and a few years later you could get like Atari 400 and commodore Victor 20 and you could you could type on the keyboard and program in higher level languages along alongside the Assembly Language so these ideas have been building up a while and I guess my generation got to feel them build up which is different than people coming into the field now now for whom these things have just been part of the Ambiance of of culture for their whole career even or even their even their whole life well it's fascinating to think about you know there being all of these ideas kind of swimming you know almost with a noise all around the world all the different generations and then some kind of nonlinear thing happens where they percolate up and and uh capture the imagination of the mainstream and that seems to be what's happening with AI now I mean n who you mentioned had the idea of the Superman right but he he didn't understand enough about technology to think you could physically engineer a Superman by piecing together Mo molecules in a certain way he he was a bit vague about how how the how the Superman would appear but he was quite deep at thinking about what the State of Consciousness and the mode of cognition of of a Superman would be he he was a very astute analyst of you know how the human mind constructs the illusion of a self how it constructs the illusion of Free Will how how it constructs values like like good and evil out of its own you know desire to maintain and Advance its own organism he understood a lot about how human minds work then he understood a lot about how postum Minds would work I mean this Superman was supposed to be a mind that would basically have complete root access to its own brain and Consciousness and be able to architect it its own its own value system and inspect and fine-tune all all of its own its own biases so that's a lot of powerful thinking there which then fed in and and sort of seated all of postmodern Continental philosophy and all sorts of of things have been very valuable in development of culture and indirectly even even of Technology but of course without the technology there it was all some quite abstract thinking so now now we're at a time in history when a lot of these ideas can be can be made real which is amazing amazing and scary right it's kind of interesting to think what do you think n would if he was born a a century later or transported through time what do you think you would say about AI I mean well those are quite different if he's born a century later or transported through time well he'd be he'd be on like Tik Tok and Instagram and he would never write the great works he's written so let's transport him through time maybe also sparra would be a music video right I mean I mean I mean who knows yeah but if he was transported through time do you think uh that'd be interesting actually to go back uh you just made me realize that it's possible to go back and read ni with an eye of is there some thinking about artificial beings I'm sure there he has in he had inklings I mean with Frankenstein before him I'm sure he had inklings of artificial beings somewhere in the text it'd be interesting to see to try to read his work to see if he hadn't if if uh uh Superman was actually an AGI system like if he had inklings of that kind of thinking didn't he didn't no I I would say not I mean he had he had a lot of inklings of modern cognitive science which are very interesting if you look in like the the third part of of the collection that's been titled the will to power I mean in book three there there's there's very deep analysis of thinking processes but he he wasn't so much of a physical tinkerer type type guy right was very abstract and do you think uh what do you think about the will to power do you think human what do you think drives humans is it is it uh oh an Unholy mix of things I I don't think there's one pure simple and elegant objective function D driving humans by by by by any means well do you think um if we look at I know it's hard to look at humans in an aggregate but do you think overall humans are good or or uh do we have both good and evil within us that uh depending on the circumstances depending on the whatever can can can percolate to the top good and evil are very ambiguous complicated and in some ways silly Concepts but if we we could dig into your question from a couple directions so I think if you look in evolution humanity is shaped both by individual selection and what biologists would call group selection like tribe level selection right so individual selection has driven us in a selfish DNA sort of way so so that each of us does to a certain approximation what will help us propagate our our DNA to to Future Generations I mean that that that that's why I've got four kids so far and and probably that's not the last one yeah on the other hand I like the ambition tribal like group selection means humans in a way will do what what will advocate for the Persistence of the DNA of their whole their whole tribe or their their social group and in biology you you have both of these right like a and you can see say an ant colony or beehive there's a lot of group selection in in in the evolution of those social animals on the other hand say a a big cat or some very solitary animal it's a lot more biased toward toward individual selection humans are an interesting balance and I think this reflects itself in what we would view as selfishness versus altruism to to to some extent so we just have both of those objective functions contributing to the the makeup of of our brains and then as n analyzed in his own way and others have analyzed in different ways I mean we abstract this as well we have both good good and and and evil with within us right because a lot of what we view as evil is really just selfishness and a lot of what we view as good is altruism which means doing doing what's good for the for the tribe and on that level we have both of those just baked baked into us and that's that's how it is of course there are psychopaths and sociopaths and people who you know get gratified by the suffering of others and that's that that that that's that's a different thing yeah those are exceptions but on the whole I think at core we're not purely selfish we're not purely altruistic we we are a mix and that's that's the nature of it and we also have a complex constellation of values that are just very specific to our our Evol evolutionary history like we you know we we love waterways and and and mountains and the the ideal place to put a house as in a mountain overlooking the water right and you know we we we we care a lot about our our kids and we care a little less about our cousins and even less about our fifth cousins I mean there are many particularities to human values which whether they're good or evil depends on your on on on your perspective really see I I I spent a lot of time in Ethiopia in Adis Ababa where we have one of our AI development offices for my Singularity net project and when I walk through the streets in Otis you know there's so there's people Lying by the side of the road like just living there by the side of the road dying probably of curable diseases without enough food or medicine and when I walk by them you know I feel terrible I give them money when I come back home to the developed world they're not on my mind that much I I do donate some but I mean I I also spend some of the limited money I have enjoying myself in frivolous ways rather than donating it to those people who are right now like starving dying and and suffering on on the roads side so does that make me evil I mean it makes me somewhat selfish and somewhat altruistic and we each we each balance that in in in our own way right so that's that that whether that will be true of all possible agis is a is a is a is a subtler question so you you that's how humans are so you have a sense you kind of mentioned that there's a selfish I'm not going to bring up the whole irand idea of uh selfishness being the core virtue that's an whole interesting kind of tangent that I think will just distract our I I I have to make one amusing comment or comment that has amused me anyway so the the yeah I I I have extraordinary negative respect for for Ein Rand negative what's a negative respect but when I worked with a company called jesin which was evolving flies to have extraordinary long lives in in in Southern California so we we had flies that were evolved by artificial selection to have five times a lifespan of normal fruit flies but the population of super long live flies was physically sitting in a spare room at an IR Rand Elementary School in Southern California so that was just like wow if if I saw this in a movie I wouldn't believe it right well yeah the universe has a sense of humor in that kind of way that fits in there humor fits in somehow into this whole absurd existence but you you mentioned the balance between selfishness and altruism as kind of being innate do you think it's possible that's kind of an emergent Fe phenomena those peculiarities of our value system how much of it is innate how much of it is something we collectively kind of like a dfki novel bring to life together as a civilization I mean the the answer to nature versus nurture is usually both and of course it's nature versus nurture versus self-organization as you mentioned so clearly they are evolutionary roots to individual and group selection leading to a mix of selfishness and altruism on the other hand different cultures manifest that in in in in different ways while we we all have basically the same biology and if you look if if you look at sort of pre-vedic the yanam Mamo in Venezuela which which their their culture is focused on on killing killing other tribes and you have other Stone Age tribes that are are mostly Peaceable and have big tabos against violence so you you can certainly have a big difference in in how culture manifests these innate biological characteristics but still you know there's probably limits that are given by biology I I used to argue this with my great-grandparents who were marxists actually because they they believed in the withering away of the state like they they believe that you know as you move from capitalism to socialism to Communism people would just become more socialm minded so that a state would be unnecessary and people would just give give everyone would give everyone else what what they needed Now setting aside that that's not what the various Marxist experiments on the planet seem to be heading toward in in practice just a as a theoretical point I was very dubious that that human nature could go there like at that time when my great-grandparents are alive I was just like you know I'm a cynical teenager I I think humans are humans are just jerks the state is not going to wither away if you don't have some structure keeping people from screwing each other over they're going to do it and so now I actually don't quite see things that way I mean I think the my feeling now subjectively is the culture aspect is more significant than I thought it was when I when I was a teenager and I think you could have a human society that was dialed dramatically further toward you know self-awareness other awareness compassion and sharing than our current society and of course greater material abundance helps but to some extent material abundance is a subjective perception also because many Stone Age cultures perceived themselves as living in great material abundance that they had all the food and water they wanted they lived in a beautiful place that they had sex lives that they had children I mean they they they they had abundance without any factories right so I I think Humanity probably would be capable of fundamentally more positive and and joy-filled mode of of social existence than than what we have now clearly Marx didn't quite have the right idea about about how to how to get there I mean he missed he missed a number of of key aspects of uh of human society and and its Evolution and if we look at where we are in society now how to get there is is a quite a quite different question because they're very powerful forces pushing people in in different directions than a positive joyous comp compassionate existence right so if we were tried to um you know Elon Musk is uh dreams of colonizing Mars at the moment so we maybe he'll have a chance to start a new civilization uh with a new governmental system and certainly there's quite a bit of chaos we're sitting now I don't know what the date is but this is uh June there's quite a bit of chaos and all different forms going on in the United States and all over the world so there's a hunger for new types of governments new types of leadership new types of of systems what uh and so what are the forces at play and how do we move forward yeah I mean colonizing Mars first of all it's it's a super cool thing to do we we we should be doing it so you're you're you love the idea yeah I mean it's more important it's more important than making chocolatier chocolates and and sexier lingerie and and many of the things that we spend a lot more resources on as a as a species right so I mean we certainly should do it I think that the possible Futures in which a Mars colony makes a critical difference for Humanity are are are are very few I mean I I I I I think I mean assuming we make a Mars colony and people go live there in a couple decades I mean their supplies are going to come from Earth the money to make the colony came from Earth and whatever powers are supplying the the the goods there from from Earth are going to in effect be in in control of that of that Mars colony of course there are outlier situations where you know Earth gets nuked into Oblivion and somehow Mars has been made self- sustaining by that point and and then Mars is what allows Humanity to persist but I think that those are very very very unlikely you don't think it could be a first step on a long journey of course it's a first step on a long journey which which is which is awesome I'm guessing the colonization of the rest of the physical universe will probably be done by agis that are better designed to live in space than by by the meat machines that that that we are but I mean who knows we may cryopreserve ourselves in some Superior way to what we know now and like shoot ourselves out to Alpha centurum Beyond I mean that's all cool it's very interesting and it's much more valuable than most things that you spending its resources on on the other hand with aggi we can get to a singularity before the Mars colony becomes sustaining for sure possibly before it's even operational so your intuition is that that's the problem if we really invest resources and we can get to faster than a legitimate full like self- sustaining colonization of Mars yeah and it's it's very clear that we will to me because there's so much economic value in getting from Nar AI toward toward AGI whereas the Mars colony there's less economic value until you get quite far far far out into the into the future so I think that's very interesting I just think it's it's somewhat somewhat off to the side I mean Ju Just as I think say you know art and music are are very very interesting and I want to see resources go into amazing art and music being being created and I i' rather see that than a lot of the garbage that Society spends their money on on the other hand I don't think Mars colonization or inventing amazing new genres of music is is not one of the things that is most likely to make a critical difference in the evolution of human or non-human life in in in in this part of the universe o o over the next decades you think AGI is really AI is is by far the most important thing that's on the horizon and then technologies that have direct ability to enable AGI or to accelerate AGI are also very important for example say qu Quantum Computing I don't think that's critical to achieve AGI but certainly you could see how the right qualum Computing architecture could massively accelerate AGI similar other other types of of nanotechnology right now the quest to cure aging and end disease while not in the big picture as important as as as AGI of course it's important to to all of us as as as individual humans and if someone made a super longevity pill and distributed it tomorrow I mean that would be huge and a much larger impact than a Mars colony is is is going to have for quite some time but perhaps not as much as an AGI system no because if you get if you can make a benevolent AGI then all the other problems are solved I mean the if then the AGI can be once it's as generally intelligent as humans it can rapidly become massively more generally intelligent than humans and and then that that AGI should be able to solve science and engineering problems but much better than than than human beings as long as it is in fact motivated to do so that's why I said a a benevolent AGI there could be other kinds maybe it's good to step back a little bit I mean we've been using that term AGI people often cite you as the Creator or at least the popularizer of the term AGI artificial general intelligence can you tell the origin story of the term sure so yeah I would say I I launched the term AGI upon the world for for for what what it's worth without ever fully being in in in love with the term right what happened is I was editing a book and this process started around 2001 or 2 I think the book came out 2005 finally I was editing a book which I provisionally was titling real Ai and I mean the goal was to gather together fairly serious academic is papers on the topic of making thinking machines that could really think in the sense like people can or or or even more broadly than people can right so then I was reaching out to other folks that i' had encountered here or there who were in interested in in in that which included some some other folks out of the who I knew from the transhumanist and singularitarianism I think he may have been have just started doing his PhD with uh Marcus Hooter who at that time hadn't yet published his book Universal AI which sort of gives a mathematical foundation for artificial general intelligence so I reached out to Shane and Marcus and Peter Vos and my pay Wang who was another former employee of mine who had been Douglas hoffstead PhD student who had his own approach to AGI and a bunch of some Russian folks reached out to these guys and they contributed papers for the book but that was my provisional title but I never loved it because in the end you know I was doing some what we would Now call narrow AI as well like applying machine learning to genomics data or chat data for sentiment analysis and I mean that work is real and in a sense in a sense it's it's really AI it's just a different kind of kind of AI Ray KW wrote about narrow AI versus strong AI but that seemed weird to me because first of all narrow and strong are not Anton that's right I mean but secondly strong AI was used in the cognitive science literature to mean the hypothesis that digital computer AIS could have true Consciousness like like human beings so there was already a meaning to strong AI which was complexly different but related right so we were tossing around on an an email list whether what title title it should be and so we we talked about narrow AI broad AI wide AI narrow AI General Ai and I think it it was either Shane leg or Peter Vos on the private email discussion we had you said but why don't we go with AGI artificial general intelligence and pay Wang wanted to do GI General artificial intelligence cuz in Chinese it goes in that order right but we figured gay wouldn't work in in in US culture at that time right so so we we went with the AGI AGI we used it for the for the title of that book and part of Peter and Shane's reasoning was you have the G factor in Psychology which is IQ general intelligence right so you have a meaning of GI general intelligence in Psychology so then you're looking like artificial GI so then oh that makes a lot of sense we use that for the we use that for the title of the book and so I think I maybe both Shane and Peter think they invented the term but but then later after the book was published this guy Mark grid came up to me and he's like well I I publish an essay with the term AGI in it in like 1997 or something and so I'm just waiting for some Russian to come out and say they published that in 1953 right I mean that term that term is not dramatically inovative or anything it's one of these obvious in hindsight things which is also annoying in a way because you know Josh habach who you you interviewed is a close friend of mine he likes the term synthetic intelligence which I like much better but it hasn't it hasn't actually caught on right because I mean artificial is a bit off to me because AR artifice is like a tool or something but but not all AGI are going to be tools I mean they may be now but we're aiming toward making them agents rather than than tools and in a way I don't like the distinction between artificial and natural because I mean we're we're part of nature also and machines are part of are are part of nature I mean you can look at evolved versus engineered but that that's a different that's a different distinction then it should be engineered general intelligence right and then General well if you look at Marcus Hooter's book Universal what he argues there is is you know within the domain of computation Theory which is limited but interesting so if you assume computable environment it's a computable reward functions then he articulates what would be a truly general intelligence a system called aixi which is quite beautiful I I and that's that's the middle name of of my latest child actually is it what's the first name first name is quiry Q rxi which my my wife came out with but that that's an acronym for Quantum organized r expanding intelligence and his middle name is his middle name is exes actually which is uh mean means the the former principle underlying I exe but in any case you're giving Ela musk a new child a run I I I did it first he he he cop he copied me with this this new freakish name but now if I have another baby I'm GNA have to out outdo him it's become an arms Ras of weird geeky baby names yeah we'll see what the babies think about it right but I mean my oldest son zarathustra loves his name and my daughter sharizad loves her name so so far basically if you give your kids weird names they live up to it well you're obliged to make the kids weird enough that they like the names right it directs their upbringing in a certain way but yeah anyway I mean what Marcus showed in that book is that a truly general intelligence theoretically is possible but would take infinite computing power so then the artificial is a little off the General is not really achievable within physics as as as we know it and I mean physics as we know it may be limited but that's what we have to work with now intelligence infinitely General you mean like yeah information processing perspective yeah yeah in intelligence is not very well- defined either right I mean what what what what does it mean I mean in AI now it's fashionable to look at it as maximizing and expected reward over the future but that's that sort of definition is path olical in various ways and my my friend David wein bomb AKA Weaver he had a beautiful PhD thesis on open-ended intelligence trying to conceive intelligence in a without a reward without yeah he's just looking at it differently he's looking at complex self-organizing systems and looking at an intelligent system as being one that you know revises and grows and improves itself in conjunction with with its with its environment without necessarily there being one objective function it's trying to maximize although over certain intervals of time it may act as if it's optimizing a certain objective function very much Solaris from Stan's novels right so yeah the point is artificial general and intelligence don't work they're all bad on the other hand everyone knows what AI is yeah and AI seems immediately comprehensible to people with with a technical background so I think that the term is served as sociological function and now now now it's out there everywhere which which which baffles me it's like KFC I mean that's that's it you're we're stuck with agii probably for a very long time until AGI systems take over and rename themselves yeah and that I mean that we'll be we're stuck with gpus too which mostly have nothing to do with Graphics anymore right I wonder what the AGI system will call us humans I was maybe Grandpa yeah GPS yeah Grandpa Processing Unit yeah biological Grandpa processing units uh okay so um maybe also just a comment on AGI representing before even the term existed representing a kind of community now you've talked about this in the past sort of AI has come in waves but there's always always been this community of people who dream about creating uh General human level super intelligence systems uh can you maybe give your sense of the history of this community as it exists today as it existed before this deep learning re Evolution all all throughout the winters and the Summers of AI sure uh first I would say as a side point the winters and Summers of AI are greatly exaggerated by by Americans yeah and in the if you look at the publication record of the artificial intelligence Community since say the 1950s you would find a pretty steady growth in advance of ideas and and and papers and what's thought of as an AI winter or summer was sort of how much money is the US military pumping into AI which was was meaningful on the other hand there was AI going on in Germany UK and in Japan and in Russia all over the place while US military got more and less less in enthused about AI so what I mean that happened to be just for people who don't know the US military happened to be the main source of funding for AI research so another way to phrase that is it's up and down of uh funding for artificial intelligence research true and I would say the correlation between funding and intellectual Advance was not 100% right because I mean in in Russia as an example or in Germany there was less dollar funding than in the US but many foundational ideas were were laid out but it was more Theory than than implementation right and us really excelled at sort of breaking through from theoretical papers to working implementations which which did go up and down somewhat with US military funding but still I mean you can look in the 1980s Dietrich derer in Germany had self-driving cars on the autoban right and I mean this it was a little early with regard to the car industry so it didn't catch on such as has has happened now but I mean that whole advancement of self-driving car technology in Germany was Prett pretty much independent of AI military Summers and and Winters in the US so there there's been more going on in AI globally than not only most people on the planet realize but than most new AI phds realize because they've come out within a certain Sub sub field of of AI and haven't had to look so much so much beyond that but I I would say when I got when I got my PhD in 1989 in in mathematics I was interested in AI already Philadelphia by yeah I started at myu then I transferred to to Philadelphia to Temple University good old North Philly North Philly yeah yeah yeah the pearl of pearl of the US right yeah you never stopped at a red light then because you were afraid if you stopped at the red light some more car Jackie so strive through every red light yeah it is a every every day driving or bicycling to Temple from my house was it was like a new new adventure right but yeah when I the reason I didn't do PC and AI was what people were doing in the academic AI field then was just astoundingly boring and seemed wrong-headed to me it was really like rule-based expert systems and production systems and I actually I loved mathematical logic I had nothing against logic as the cognitive engine for an AI but the idea that you could type in the knowledge that AI would need to think seemed just completely stupid and and and and wrong-headed to me I mean you can use logic if you want but somehow the system has got to be automated learning right it should be learning from experience and the AI field then was not interested in learning from experience I mean some researchers certainly were I mean I I remember in mid 80s I discovered a book by John Andreas which was it was about uh reinforcement learning system called perus p rr- p USS which was an acronym that I can't even remember what it was for but purpose anyway but he I mean that was a system that was supposed to be an AGI and basically by some sort of fancy like Markoff decision process learning it was supposed to learn everything just from the bits coming into it and learn to maximize its reward and become become intelligent right so that was there in Academia back then but it was like isolated scattered weird people but all these scattered weird people in that period I mean they they laid the intellectual grounds for what happened later so you look at John Andreas at University of Canterbury with his purpose reinforcement learning marov system he was the PG supervisor for John clar in in in New Zealand now John clear worked with me when I was at wado University in in 1993 in in in New Zealand and he worked with Ian Whitten there and they launched WCA which was the first open- Source machine learning toolkit which was launched in I guess '93 or '94 when I was at wada University written in Java unfortunately written in Java which was a cool language back then though right I guess it's still well it's not cool anymore but it's powerful I find like most programmers now I find Java unnecessarily bloated but back then it was like Java or C++ basically and Java oriented so it's Java was easier for students yeah amusingly a lot of the work on wo when we were in New Zealand was funded by a u sorry A New Zealand government grant to use machine learning to predict the menstrual cycles of cows so in the US all the grant funding for AI was about how to kill how to kill people or spy on people in New Zealand it's all about cows or kiwi fruits right yeah so so yeah anyway I mean Andre John Andreas had his probability Theory based reinforcement learning Proto AGI John clear was trying to do much more ambitious probabilistic AGI systems now John clear helped do wo which was the first open- Source machine learning tool get so the predecessor for tensor flow and torch and and all these things also Shane leg was was at at wado working with working with with with John CLE and Ian Ian wh and and this whole group and and then working with my own company my company webm an AI company I had in in the late '90s with a team there at wado University which is how Shane got his head full of of AGI which led him to go on and with Gabus found Deep Mind so what you can see through that lineage is you know in the 80s and 70s John Andreas was trying to build probalistic reinforcemen AGI systems the technology the computers just weren't there to support it his ideas were were very similar to what people are doing now but you know although he's long since passed away and didn't become that famous outside of Canterbury I mean the lineage of ideas passed on from him to his students to their students you can go Trace directly from there to me and and to Deep Mind right so that there was a lot going on in AGI that did ultimately lay the groundwork for what we have today but there was there wasn't a community right and so when I when I started trying to pull together an AGI community it was in the I guess the early Arts when I was living in in Washington DC and making a living doing AI Consulting for VAR various US government agencies and I I organized the first Agia Workshop in 2006 and I mean it wasn't it wasn't like it was literally in my basement or something I mean it was it was in the conference room at the Marriott in Bethesda it's not not that not that edgy or underground unfortunately but still how many people attended about 60 or something that's not bad I mean DC has a lot of AI going on probably until the last five or 10 years much more than Silicon Valley although it's just quiet because of the nature of what what Happ what happens in in in DC their business isn't driven by PR mostly when something starts to work really well it's taken black and becomes even even more quiet right yeah but yeah the thing is that really had the feeling of a group of star eyed Mavericks like huddled in a basement like plotting how to overthrow the the narrow AI establishment and you know for the first time in some cases coming together with others who shared their passion for AGI and the technical seriousness about about working on it right and that I mean that's very very different than than what we have today I mean now now it's a little bit different we we have AGI conference every year and then there's several hundred people rather than than 50 now now it's more like this is the main Gathering of people who want to achieve AGI and think that uh large scale nonlinear regression is is is is not the golden path to to AGI so I mean it's AK and your all Network yeah yeah yeah well certain architectures for for for learning using neural network so yeah the AGI conferences are sort of now the main concentration of people not obsessed with deep neural Nets and deep reinforcement learning but but still interested in in in a in AGI not not not not the only ones I mean there there's other little conferences and and groupings interested in uh human level Ai and and cognitive cognitive architectures and so forth but yeah it's it's been a big shift like back back then you couldn't really it would be very very edgy then to give a university Department seminar the mentioned AGI or human level AI it was more like you had to talk about something more short-term and immediately practical then you know in the bar after the seminar you could  about AGI in the same breath as uh as time travel or or the simulation hypothesis or something right whereas now now AGI is not only in the academic seminar room like you have Vladimir Putin knows what AGI is and he's like Russia needs to become the leader in AGI right so national leaders and CEOs of large corporations I mean the CTO of Intel just Ratner this was years ago Singularity Summit conference 2008 or something he's like We Believe Ray KW The Singularity will happen in 2045 and it will have Intel Inside So I mean so it's gone it's gone from being something which is the pursuit of like crazed Mavericks crackpots and and science fiction Fanatics to being you know a a marketing term for large large corporations and and national leaders right which is a astounding transition but yet in the in the course of this transition I think a bunch of sub communities have formed and the community around the AGI conference series is certainly one of them it hasn't grown as big as I might have liked it to on on the other hand you know sometimes a a modest ized Community can be better for making intellectual progress also like you go to a society for Neuroscience conference you have 35 or 40,000 neuroscientists on the one hand it's it's amazing on the other hand you're not going to talk to the leaders of the of the of of the field there if if you're an outsider yeah in the same in the same sense the triple AI the artificial intelligence uh the main kind of generic artificial intelligence Comm uh conference it's too big it's uh too amorphous like it it doesn't make it and and nip has become a a company advertising Outlet now right so I so yeah so I mean to to comment on uh the role of AGI in the research Community i' still if you look at neurs if you look at cvpr if you look at these uh eye clear you know um AGI is still seen as the outcast I would still I would say in these main machine learning in these main artificial intelligence uh conferences amongst the researchers I don't know if it's an accepted term yet I what I've seen bravely you mention Shane leg is Deep Mind and then open AI are the two places that are I would say unapologetically so far I think it's actually changing unfortunately but so far they've been pushing the idea that the goal is to create an AGI well they have billions of dollars behind them so I mean that that they in the public mind that that certainly carries some oomph right I mean I mean but they also have really strong researchers right they they do they're great teams I mean Deep Mind in particular yeah and they have I mean Deep Mind has Marcus hutter walking around I mean there's all these folks who basically their full-time position involves dreaming about creating AGI yeah I mean Google brain has a lot of amazing AGI oriented people also I mean and and I mean so I'd say from a public marketing view Deep Mind and open AI are the two large wef funded organizations that have put the term and concept AGI out there sort of as part of their Public Image but I mean there there're certainly not there are other groups that are doing research that seems just as as as aiish to me I mean including a bunch of groups in in Google's main main Mountain View office so yeah it's true AGI is somewhat away from the mainstream now but if you compare to where it was right you know 15 years ago there's there's there's there's been an amazing mainstreaming you could say the same thing about super longevity research which is one of my my application areas that I'm excited about I mean I've been talking about this since the '90s but working on this since 2001 and back then really to say you're trying to create therapies to allow people to live hundreds or thousands of years you you were way way way way out of out of the industry academic mainstream but now you know Google had had project Calico Craig ventor had human longevity Incorporated and once once the Suits come marching in right I mean once once there's big money in it then people are forced to take it take it seriously because that's the way Mo modern society works so it's still not as mainstream as cancer research just as AGI is is not as mainstream as automated driving or something but the degree of mainstreaming that's happened in the last uh you know 10 to 15 years is is astounding to those of us who've been at it for a while yeah but there's a marketing aspect to the term but in terms of actual full force research that's going on under the header of AGI it's currently I would say dominated maybe you can disag degree dominated by neural networks research that the nonlinear regression as you mentioned um like what's your sense with with open Cog with your work in in general I was uh logic based systems and expert systems for me always seemed uh to capture a deep element of intelligence that needs to be there like you said it needs to learn it needs to be automated somehow but that seems to be missing from a lot of the a lot of research currently um so what's your sense I guess one way to ask this question what's your sense of what kind of things will an AGI system need to have yeah that that's a very interesting topic that I thought about for for a long time and I I think there are many many different approaches that can work for getting to to human level AI so I I I don't I don't think there's like one golden algorithm one one golden design that that that can that can work and I mean flying machines is the the much warant analogy here right like I mean you have airplanes you have helicopters you you you you have balloons you have stealth bombers that don't look like regular airplanes you you've got all blimps Birds too Birds yeah and and bugs right and uh you I mean and there are certainly many kinds of flying machines that and there's a catapult that you can just launch there's bicycle powered like uh flying machines right nice yeah yeah so now these are all analyzable by a basic theory of of aerodynamics right now so one issue with AGI is we don't yet have the analog of the theory of aerodynamics and that's that's what Marcus hter was trying to make with the Axe and his general theory of general intelligence but that theory in its most clearly articulated Parts really only works for either infinitely powerful machines or almost or insanely yeah impractically powerful machines so I mean if if you were going to take a theory based approach to AGI what you would do is say well let's let's take what's called say axe TL which is which is hutter's Axe machine that can work on merely insanely much processing power rather than infinitely much what does TL stand for uh time time and length Okay so you're basically how how constrained somehow yeah yeah yeah so how how axe Works basically is each each each action that it wants to take before taking that action it looks at all its history yeah and then it looks at all possible programs that it could use to make a decision yeah and it decides like which decision program would have let it make the best decisions according to its reward function over its history and he uses that decision program to take to make the next decision right it's not afraid of infinite resources it's searching through the space of all possible computer programs in between each action and each next Action Now XL searches through all possible computer programs that have runtime less than T and length less than l so it's which is still an impracticably humongous space right so what what you would like to do to make an AGI and what will probably be done 50 years from now to make an AGI is say okay well we we we have some constraints we have these processing power constraints and you know we have the space and time constraints on on on on the program we have energy utilization constraints and we have this particular class environments class of environments that we care about which may be say you know manipulating physical objects on on the surface the Earth communicating in in in human language I mean whatever our particular not not annihilating Humanity whatever our particular requirements happen to be if you formalize those requirements in some formal specification language you should then be able to run a automated program specializer on Axl specialize it to the the Computing resource constraints and the particular environment and goal and then it will spit out like the the specialized version of Axl to your resource restrictions in your environment which will be your AGI right and that that that I think is how our super AGI will create new AGI systems right but but and that's a very rough seems really inefficient that's a very Russian approach by the way like the whole field of program specialization CA came out of Russia can you backtrack so what is program specialization so it's basically well say take take take sorting for example you can have a generic program for sorting lists but what if all your lists you care about are length 10,000 or less got it you can run an automated program specializer on your sorting algorithm and it will come up with the algorithm that's optimal for sorting lists of length 1,000 or Le or 10,000 or less right it's kind of like isn't that the kind of the process of evolution is uh it's a program specializer to the environment so you're you're kind of evolving human beings or Liv exactly I your Russian Heritage is is showing there I mean so without Alexander vitv I mean there and Peter Anin and so on I mean there's a yeah there there's a long history of of thinking about Evolution Evolution that way that way also right so well my my my my point is that what we're thinking of as a human level general intelligence you know if you start from narrow AIS like are being used in the commercial AI field now then you're thinking okay how do we make it more and more General on the other hand if you start from aexi or Schmid Uber's girdle machine or these infinite infinitely powerful but practically infusible AIS then getting to a human level AGI is a matter of specialization it's like how do you how do you take these maximally General learning processes and how do you how do you specialize them so that they can operate within the resource constraints that you have but will achieve the particular things that that you care about because we we are not we humans are not maximally General intelligences right if I ask you to run a maze in 750 Dimensions you'll probably be very slow whereas at two Dimensions you you're probably you're probably way better right so I mean we're special be because our our hippocampus has a two-dimensional map in it right and it does not have a 750 dimensional map in it so I mean we we are you know A peculiar mix of generality and and and specialization right we'll probably start quite General at Birth uh not obviously still narrow but like more General than we are at age uh 20 and 30 and 40 and 50 and 60 I don't think that I I I think it's more complex than that because I mean the young in in some sense a young child is less biased and the brain has yet to sort of crystallize into appropriate structures for processing aspects of the physical and social world on on on the other hand the young child is very tied to their sensorium whereas we can we can deal with abstract mathematics like 750 dimensions and the young child cannot because they they haven't they haven't grown what pette called the the formal capabilities they they haven't learned to abstract yet right and and the ability to abstract gives you a different kind of generality than than what than what a baby has so there there's both more specialization and more generalization that comes with with the development process actually I mean I guess just the the trajectories of the specialization are most controllable at the young age I guess is uh as one way to put it do you have do you have kids no they're not as controllable as you think so you think it's uh interesting I I I I I think honestly I think a human adult is much more generally intelligent that than a human baby babies are very stupid I mean I mean I mean they're cute they're cute which is which is why we put up with their repetitiveness and and stupidity but and they have what the Zen guys would call a a beginner's mind which is a beautiful thing but that doesn't necessarily correlate with with a high level of of in of Intelligence on the plot of like cuteness and stupidity there there's a the there's a process that allows us to put up with their stupidity they get become more by the time you're an ugly old man like me you got to get really really smart to compensate okay cool but yeah going back to your your original question so the the way I look at human level AGI is yeah how do you specialize you know unrealistically inefficient superhuman Brute Force learning processes to the specific goals that humans need to achieve and the specific resources that that that we have and both of these the goals and the resources the environments I me all all this is important and on the on the resources side it's important that the hardware resources we're bringing to Bear are very different than the human brain so the way the way I would want to implement AGI on a a bunch of neurons in a vat that I could rewire arbitrarily is quite different than the way I would want to create AGI on say a modern server Farm of CPUs and gpus which in turn may be quite different than the way I would want to implement AGI on you know whatever quantum computer will will have in in 10 years supposing someone makes a robust Quantum turning machine or or something right so I I I I think you know there there's been co-evolution of the the patterns of organization in the human brain and and the physiological particulars of of of the human brain o o over time and when you look at neural network works that is one powerful class of learning algorithms but it's also a class of learning algorithms that evolve to exploit the particulars of the human brain as as a computational substrate if you're looking at the computational substrate of a modern server Farm you won't necessarily want the same algorithms that you want that you want on the on the human brain and you know from the right level of abstraction you you you could look at maybe the best algorithms on a brain and the best algorithms on a modern computer network as implementing the same abstract learning and representation processes but you know finding that level of abstraction is its own AI research project then right so that's about the hardware side and and the software side which follows from that then regarding what are the requirements I I wrote the paper years ago on what I called the embodied communication prior which was quite similar in intent to Yoshua benio's recent paper on the Consciousness prior except I I didn't want to wrap up Consciousness in it because to me the qualia problem and subjective experience is a very interesting issue also which which we can chat about yeah but I would rather keep that philosophical debate distinct from the debate of what kind of biases do you want to put in a general intelligence to give it humanlike general intelligence and I'm not sure yosha Benjo is really addressing that kind of I's just using the term I I love yosua to to pieces like he he's by far my favorite of the the Lions of of deep learning but he's s such such a good-hearted Guyer for sure I am not I am not sure he has Plumb to the depths of the philosophy of of of Consciousness no he's using it as a sexy T yeah yeah yeah so I I what I called it was the embodied communication prior and can you maybe explain it a little bit yeah yeah what what I meant was you know what are we humans evolved for you can say Being Human but that's that's very abstract right I mean we our minds control individual bodies which are autonomous agents moving around in a world that's composed largely of solid objects right and we've also evolved to communicate via language with other you know solid object agents that are going around doing things collectively with us in a in a world of solid objects and these things are very obvious but if you compare them to the scope of all possible intelligences or even all possible intelligences that are physically realizable that actually can strange things a lot so if you start to look at you know how would you realize some Specialized or constrained version of universal general intelligence in a system that has you know limited memory and limited speed of processing but who general intelligence will be biased toward controlling a solid object agent which is Mobile in a solid object world for manipulating solid objects and communicating via language with other similar agents in that same world right then starting from that you're starting to get a requirements analysis for for for human human level general intelligence and then that that leads you into cognitive science and you can look at say what are the different types of memory that the human mind and brain has and this this has matured over the last decades and I got into this a lot in so after getting my PhD in math I was an academic for eight years I was in Departments of mathematics computer science and psychology when I was in the psychology department the University of Western Australia I was focused on cognitive science of of memory and and perception actually I was I was teaching neuron Nets and deep neural Nets and it was multi-layer perceptrons right psychology yeah a cognitive science it was cross disciplinary among engineering math psychology philosophy Linguistics Compu computer science but yeah we we were teaching psychology students to try to model the data from Human cognition experiments using multi-layer perceptrons which was the early version of a deep neural network very very yeah recurrent back propop was very very slow to train back then right so this is the study of these constraint systems that are supposed to deal with physical object so if you look if you look at if you look at cognitive psychology you can see there's multiple types of memory which are to some extent represented by different subsystems in the human brain so we have episodic memory which takes into account our life history and everything that's happened to us we have declarative or semantic memory which is like facts and beliefs abstracted from the particular situations as they occurred in there's sensory memory which to some extent is sense modality specific and then to some extent is is Unified ac across across sense modalities there's procedural memory memory of how to do stuff like how how to swing the tennis racket right which is there's motor memory but it's also a little more more abstract than than motor memory it involves cerebellum and cortex working working together then then there's there's memory linkage with emotion between with which has to do with linkages of Cortex and and and and lyic system there's specifics of spatial and temporal modeling connected with memory which has to do with you know hippocampus and Thalamus connecting to Cortex and the basil ganglia which influences Golds so we have specific memory of what goals sub goals and Sub sub goals we want to perceive in which context in the past human brain has substantially different subsystems for these different types of memory and substantially differently tuned learning like differently tuned modes of long-term potentiation to do with the types of neurons and neurotransmitters and the different parts of the brain correspond to these different types of knowledge and these different types of memory and learning in the human brain I mean you can back these all into embodied communication for controlling agents and in Worlds of of of solid objects now so if you look at building an AGI system one way to do it which starts more from cognitive science than Neuroscience is to say okay what are the types of memory that that are necessary for this kind of world yeah yeah necessary for this this sort of intelligence what types of learning work well with these different types of memory and then how do you connect all these things together right and of course the human brain did it incrementally through Evolution because each of the subn networks of the brain I mean it's not really the lobes of the brain it's the sub networks Each of which is is widely distributed which of the sub each of the sub networks of the brain Co involved with the other subn networks of of the brain both in terms of its patterns of organization and the particulars of the neurophysiology so they all grew up communicating and adapting to each other it's not like they were separate black boxes that were then were then glommed together right whereas as Engineers we would tend to say let's make let's make the declarative memory box here and the procedural memory box here and the perception box here and W wire them together and when you can do that it's it's interesting I mean that's how a car is built right but on the other hand that's clearly not how biological systems are are are made the parts coevolve so as to adapt and and work together so this that's by the way how every human engineered system that flies that was were using that analogy before is built as well so do you find this at all appealing like there's been a lot of really exciting which I find strange that it's ignored uh work in cognitive architectures for example throughout the last few decades do you find yeah I mean I I I had a lot to do with that with that community and you know Paul Rosen Blum who was one of the and John lared who built the sore architecture are friends of mine and uh I I learned SAR quite well and actar and these different cognitive architectures and how I was looking at the AI World about 10 years ago before this whole commercial deep learning explosion was on the one hand you had these cognitive architecture guys who were working closely with psychologists and cognitive scientists who had thought a lot about how the different parts of a humanlike mind should should work together on the other hand you had these learning theory guys who didn't care at all about the architecture but were just thinking about like how how do you recognize patterns and large amounts of data and in some sense what you needed to do was was to get the learning that the learning theory guys were doing and put it together with the architecture that the cognitive architecture guys were doing and then you would have what you needed now you can't unfortunately when you look at the details you can't just do that without totally rebuilding what what is happening on both the cognitive architecture and the learning side so I mean they tried to do that in sore but what they ultimately did is like take a neural deep neural net or something for perception and you includ as one of the one of the black boxes it's one it becomes one of the boxes the learning mechanism becomes one of the boxes opposed to fundamental that that doesn't quite work you could look at some of the stuff Deep Mind has done like the the differential neural computer or something that sort of has a neural net for deep learning perception it has another neural net which is like a a memory matrix it's St say the map of the London Subway or something so probably Demis aabus was thinking about as like part of Cortex and part of hippocampus because hippocampus has a spatial map and when he was a neuroscientist he was doing a bunch on cortex hippocampus inter ction so there the DNC would be an example of folks from the deep neural net world trying to take a step in the cognitive architecture Direction by having two neurom modules that correspond roughly to two different parts of the human brain that deal with different kinds of memory and learning but on the other hand it's super super super crude from the cognitive architecture view right Ju Just as what what John L and sore did with neural Nets was super super crude from from a from a learning point of view because the learning was like off to the side not affecting the core representations right and mean you weren't learning the representation you were learning the data that feeds into the rep you you were learning abstractions of perceptual data to feed into the the representation that was was not learned right so yeah this this was this was clear to me a while ago and one of my hopes with the AGI Community was to sort of bring people from those two directions together that didn't happen much in terms of not yet and or what I was going to say is it didn't happen in terms of bringing like the Lions of cognitive architecture together with the Lions of deep learning it did work in the sense that a bunch of younger researchers have had their heads filled with both of those ideas this this comes back to a a saying my dad who was a university Professor often quoted to me which was a science advances one funeral at a time which I'm I'm I'm trying to avoid like I'm I'm 53 years old and I'm trying to invent amazing weird ass new things that that that nobody ever ever thought about which which we'll talk about in in in a few in a few minutes but there but there but there there is that aspect right like the people who've been at AI a long time and have made their career developing one aspect like a cognitive architecture or or a deep learning approach it can be hard on once you're old and have made your career doing one thing it can be hard to mentally shift gears I mean I I I try quite hard to remain flexible minded been successful somewhat in changing maybe uh have you changed your mind on some aspects of what it takes to build an AGI like technical things the hard part is that the world doesn't want you to the world or your own brain the world well that one point is your brain doesn't want to the other part is that the world doesn't want you to like like the people who have followed your ideas get mad at you if if if if you change your mind and and you know the media wants to pigeonhole you as as Avatar of of of of a certain a certain idea but yeah I i' I've changed my mind on on a bunch of things I mean when I started my career I really thought Quantum Computing would be necessary for for AGI and I I doubt it's necessary now although I think it will be a super major enhancement but I mean I'm also I'm now in the middle of embarking on a complete rethink and rewrite from scratch of our opencog AGI system together with uh alexe poov and his team in St Petersburg who's working with me in Singularity net so now we're trying to like go back to basics take take everything we learned from working with the current opencog system take everything everybody else has learned from working with with their with their their Proto AGI systems and and design design the best framework for the for the next stage and I do think there's a lot to be learned from the recent successes with deep neural Nets and deep reinforcement systems I mean people made these essentially trivial systems work much better than I thought they would and there's a lot to be learned from that and I want to incorporate that knowledge appropriately in our our opencog 2.0 system on on on the other hand I also think current deep neural net architectures as such will never get you any anywhere near AGI so I think you you avoid the pathology of throwing the baby out with the bath water and like saying well these things are garbage because foolish journalists overblow them as as as being the path to AGI and and a few researchers over overblow them as as well yeah there there's there's a lot of interesting stuff to be learned there on even though those are not not the golden path so maybe this is a good chance to step back you mentioned open coock 2.0 but go back to open K 0.0 which exist yeah uh yeah maybe talk to the history of open absolutely and you're thinking about these ideas I would say opencog 2.0 is a term we're throwing around sort of tongue and cheek because the existing opencog system that we're working on now is not remotely close to what we consider we'd consider a one a 1.0 right I mean I mean it's it it's an early it's it's been around what 13 years or something but it's still an early stage research system right and actually we're we are going back to the beginning in terms of theory and implementation because we feel like that's the right thing to do but I'm sure what we end up with is going to have a huge amount in common with with with with the current system I mean we all still like that the general approach so that so first of all what is open Cog sure open Cog is an open-source software project that I launched together with several others in 2008 and probably the first code written to that was written in 2001 or two or something that was developed uh as a proprietary co-base within my AI company nov LLC then we decided to open source it in two in 2008 cleaned up the code throughout some things add added some new things and what language is it written in it's C++ primarily there's a bunch of scheme as well but most of it C++ and it's separate from that something we'll also talk about is singularity net so it was it was born as a non networked thing correct correct well there are many levels of networks in inv involved here right no connectivity to the internet or no at at Birth yeah I mean Singularity net is a separate project and a separate body of code and you can use Singularity net as part of the infrastructure for a distributed opencog system but they are there are different layers yeah got it so opencog on the one hand as a software framework could be used to implement a variety of different Ai architectures and and and algorithms but in practice there's been a group of developers which I've been leading together with lenus vus nil giler and a few others which have been using the opencog platform and infrastructure to to implement certain ideas about how to make an an AGI so there there's been a little bit of ambiguity about opencog the software platform versus opencog the the the AGI design because in theory you could use that software to do you could use it to make a neural net you could you could use it to make a lot of different what kind of stuff does the software platform provide like in terms of utilities tools like what what yeah let me first tell about opencog as a software platform and then I'll tell you the specific AGI R&D we've been building building on top of it yep so the core component of opencog as a software platform is what we call the adom space which is a weighted labeled hypergraph atom atom space atom space yeah yeah not not Adam like Adam and Eve although that would be cool too yeah so you have a hyper graph which is like a so a graph in this sense is a bunch of nodes with links between them a hyper graph is like a graph but links can go between more than two nodes so you have a link between three nodes and in fact in fact open cogs Adam space would properly be called a metagraph because you can have links pointing to link LS or you could have links playing the whole subgraphs right so it's a it's an extended hyper graph or or a metagraph and is metagraph a technical term it is now a technical term interesting but I don't think it was yet a technical term when we started calling this a generalized hypergraph in but in in any case it's a weighted labeled generalized hypergraph or weighted labeled metagraph the the weights and labels mean that the nodes and links can have numbers and and symbols attached to them so they can have Types on them they can have numbers on them that represent say a truth value or an importance value for for a certain purpose and of course like with all things you can reduce that to a hypergraph and then the hypergraph can be reduce hypergraph to a graph and you could reduce a graph to an adjacency Matrix so I mean there's always multiple representations but there's a layer of representation that seems to work well here got it right right right and so similarly you could have a link to a whole graph because a whole graph could repres present say a body of information and I could say I reject this body of information then one way to do that is make that link go to that whole subgraph representing the body of information right I mean there's many there are many alternate representations but that's anyway what we have an opencog we have an atom space which is this weighted labeled generalized hypergraph knowledge store it lives in Ram there's also a way to back back it up to disk there are ways to spread it among among multiple different Mach Maes then there are various utilities for dealing with that so there's a pattern matcher which lets you specify a sort of abstract pattern and then search through a whole atom space w labeled hypergraph to see what sub hypergraphs May match that that pattern for an example so that's then there's there's something called the the Cog server in opencog which lets you run a bunch of different agents or processes these in auler and each of these agents basically it reads stuff from the adom space and it writes stuff to the adom space so th this is sort of the basic operational model like that's the software framework right and and of course that's there's a lot there just from a scalable software engineering standpoint so you could use this I don't know if you've have you looked into the Steven wols physics project recently with the hypog grass and stuff could you theoretically use like the software framework to play certainly could although Wolfram would rather die than use anything but Mathematica for for his work well that's yeah but there's a big community of people who are uh you know would love integration and like like you said the young minds love the idea of integrating of connecting yeah that's right and I would add on that note the idea of using hypergraph type models in physics is not very new like if if you look at the Russians did it first well I'm sure they did and a guy named Ben drus who's a mathematician a professor in Louisiana or somewhere had a beautiful book on Quantum sets and hypergraphs and algebraic topology for discreet models of physics and carried it much farther than than than Wolfram has but he's he's not rich and famous so so it didn't didn't get in the headlines but yeah wolf from aide yeah certainly that's a good way to put it the whole opencog framework you could use it to model biological networks and and simulate biology processes you could use it to model physics on on on discrete graph models of of of physics so you can you could use it you could use it to do say biologically realistic neur neural networks for for for example and that's so that that's a framework what do agents and processes do do they grow the graph do they what kind of computations just to get get a sense are they supposed in theory they could do anything they want to do they're just C++ processes on on the other hand the computation framework is sort of designed for agents where most of their processing time is taken up with reads and writs to the atom space and so that's that's a very different processing model then say the matrix multiplication based model as underlies most deep most deep Learning Systems right so so you could I mean you could create an agent that just factored numbers for a billion years it would run within the opencog platform but it would be pointless right the I mean the point of doing opencog is because you want to make agents that are cooperating via reading and writing into this this weighted labeled hypergraph right and so that and that that has both cognitive architecture importance because then this hypergraph is being used as a sort of shared memory among different cognitive processes but it also has you know software and Hardware implementation implications cuz current GPU architectures are not so useful for opencog whereas a graph chip would be incredibly useful right and I think graph core has those now but they're not ideally suited for this but I think in the next let's say three to five years we're going to see new chips where like a graph is put on the Chip And and you know the back and forth between multiple processes acting simdi and Mimi on that on that graph is going to be fast and then that may do for opencog type architectures what gpus did for for deep neural architectures as a small tangent can you comment on thoughts about neuromorphic Computing so like Hardware implementations of all these different kind of uh are you interested are you excited by that possi I'm excited by graph processors because I think they can massively speed up speed up opencog which is a class of architectures that that I'm that that I'm working on I think if you know in principle neomorphic Computing should be amazing I haven't yet been fully sold on any of the systems that that are out there like memists should be amazing too right so a lot of these things have obvious potential but I haven't yet put my hands on the system that that seemed to manifest that Marxism should be amazing but the the the current systems not been great I mean look for example if you wanted to make a biologically real real istic Hardware neural network like taking making a circuit in Hardware that emulated like the hjin Huxley equation or the isak kevich equation like equ differential equations for biologically realistic neuron and putting that in Hardware on the chip that would seem that it would make more feasible to make a large scale truly biologically realistic neural network now what's been done so far is not like that so I guess personally as a researcher I mean I've done a bunch of work in cognitive neuro in sorry in computational Neuroscience where I did some work with IPA in in in DC intelligence Advanced research project agency we were looking it how do you how do you make a biologically realistic simulation of seven different parts of the brain cooperating with each other using like realistic nonl dynamical models of neurons and how do you get that to simulate what's going on in the Mind of a geoint intelligence analyst while they're trying to find terrorists on a map right so if you want to do something like that having neuromorphic Hardware that really let you simulate like a realistic model of the neuron would would would would be would be amazing but that's that's sort of with my computational Neuroscience haton right with an AGI haton I'm just more interested in these hypergraph knowledge representation based architectures which which would benefit benefit more from from various types of of graph processors because the main processing bottleneck is Reading Writing to Ram it's reading writing to the graph in Ram the main processing bottleneck for this kind of Proto AGI architecture is not multiplying matrices and and for that reason gpus which are really good at multiplying matrices don't don't don't apply as as well there there are Frameworks like gunrock and others that try to boil down graph processing to Matrix operations and and they're cool but you're still putting a square peg in in in into a round hole in a certain way the same is true I mean current Quantum machine learning which is very cool it's also all about how to get Matrix and Vector operations in in quantum mechanics and I see why that's natural to do I mean quantum mechanics is all unitary matrices and and vectors right on on the other hand you could also try to make graph Centric quantum computers which I I think is is is is is is is where things will go and then then we can have then we can make like take the opencog implementation layer implement it in a uncollapsed state inside a quantum computer but that that may be the singularity squared right I'm I'm not I'm not I'm I'm not I'm not sure we need that to get get to human human level human level that's already beyond the the first singular but uh can we just yeah let's go back to opencog no yeah and the hypergraph and open yeah that's the software framework right so the the next thing is is our cognitive architecture tells us particular algorithms to put there got it can we Backtrack on the kind of do is this graph designed is it uh in general supposed to be sparse and the operations constantly grow and change the graph yeah the graph is sparse and but is it constantly adding links and so on it is a self modifying hypergraph so it's not so the write and read operations you're referring to this isn't just a fixed graph to which you change way it's constant growing graph yeah that yeah that that that's that's true so it's it is different model then say current deep neural Nets S have a fixed neural architecture and you're updating the weights although there have been like Cascade correlational neural architectures that grow new new nodes and links but the most common neural architectures now have a fixed neural architecture you're updating the weights and in open Cog you can update the weights and that certainly happens a lot but adding new nodes adding new links REM removing nodes and links is an equally critical part of the Systems Operations got it so now when you start to add these cognitive algorithms on top of this opencog architecture what does that look like so what yeah so that the within this framework then creating a cognitive architecture is basically two things it's it's choosing what type system you want to put on the nod and links in the hypergraph what types of nodes and links you want M and then then it's choosing what collection of Agents what collection of AI algorithms or processes are going to run to to operate on on on on this hypergraph and of course those two decisions are are closely closely connected to each other so in terms of the type system there are some links that are more neuronet like they just like have weights to get updated by heavan learning and activation spreads along them there are other links that are more logic like and nodes that are more logic like so you could have a variable node and you can have a node representing a universal or existential quantifier as in in predicate logic or or term logic so you can have logic like nodes and links or you can have neural like nodes and links you can also have procedure like nodes and links as as in say uh combinatory logic or or or Lambda calculus representing programs so you can have nodes and links representing many different types of semantics which means you could make a horrible ugly mess or you can make a system where these different types of knowledge all interpenetrate and synergize with each other beautifully right so you so the so the hypergraph can contain programs yeah it can contain programs although it can in the current version it is a very inefficient way to guide the execution of programs which is one thing that we are aiming to resolve with our our rewrite of the of the system now so what to you is the most beautiful aspect of open Cog just you personally some aspect that uh captivates your imagination from beauty or power uh yeah what what fascinates me is finding a common representation that underlies abstract declarative knowledge and sensory knowledge and movement knowledge and and procedural knowledge and episodic knowledge finding the right level of representation where all these types of knowledge are stored in a sort of universal and interconvertible yet practically manipulable way right so that that that that's to me to me that's the core because once you've done that then the different learning algorithms can help each other out like what you want is if you have a logic engine that helps with declarative knowledge and you have a deep neural net that gathers perceptual knowledge and you have say an evolutionary Learning System that learns procedures you want these to not only interact on the level of sharing results and passing inputs and outputs to each other you want the logic engine when it gets stuck to be able to share its intermediate state with the neural net and with the evolutionary learning algorithm so so that they can help each other out of of bottlenecks and help each other solve combinatorial explosions by intervening inside each other's cognitive processes but that can only be done if the intermediate state of a logic engine The evolutionary learning engine and a deep neural net are represented in in the same form and that's what we figured out how to do by putting the right type system on top of this weighted labeled hypergraph so is there can you maybe elaborate on what that what are the different characteristics of a type system that that can uh coexist uh amongst all these different kinds of knowledge that needs to be represented and is I mean like is it hierarchical um just any kind of insights you can give on that kind of type system yeah yeah so this this this gets very nitty-gritty and and mathematical of course but what one key part is switching from predicate logic to term logic what is predicate logic what is term logic logic so term logic was invented by Aristotle or at least that's the the oldest oldest recollection we we we we have we have of it but term logic breaks down basic logic into basically simple links between nodes like a inheritance link between between node a and and node B so in term Logic the basic deduction operation is a implies b b implies C therefore a implies C whereas in predicate logic the basic operation is modus ponents like a a implies B therefore B so there're there it's a slightly different way of breaking down logic but by breaking down logic into term logic you get a nice way of breaking logic down into into into nodes and links so your Concepts can can become nodes The Logical relations become links and so then inference is like so if this link is a implies B this link is B implies C then deduction builds a link a implies C and and your problemistic algorithm can assign assign a certain weight there now you may also have like a heavy and neural link from a to c which is the degree to which thinking the degree to which a being the focus of attention should make B the focus of attention right so you could have then a neural link and and you could you could have a symbolic like logical inheritance Link in your term logic and they have separate meaning but they they could be used to to guide each other as well like if if there's a large amount of neural weight on the link between A and B that may direct your logic engine to think about well what is the relation are they similar is is there an inheritance relation are they s are they similar in some context on the other hand if there's a logical relation between A and B that may want that may direct your neural component to think well when I'm thinking about a should I be directing some attention to be also because there's a logical relation so in terms of logic there's a lot of thought that went into how do you break down logic relations including basic sort of propositional logic relations as Aristotelian term logic deals with and then quantifier logic relations also how do you break those down elegantly in into a hyper graph because you I mean you can boil logic Expressions into a graph in many different ways many of them are very ugly right right we we tried to find elegant ways of sort of hierarchically breaking down complex logic expression in into no into nodes and links so so that if you have say different nodes representing you know Ben AI Lex interview or whatever the logic relations between those things are compact in in the in the node and Link representation so that when you have a neural net acting on those same nodes and links the neural net and the logic engine can can sort of inter interoperate with each other and also interpretable by humans is that is that an important that's tough in simple cases it's interpretable by humans but honestly you know I would say logic systems give more potential for yeah transparency and comprehensibility than neuron net systems but you still have to work at it because I mean if if if I show you a predicate logic proposition with like 500 nested Universal and existential quantifiers and 217 variables that's no more comprehensible than the weight Matrix of a neural network right so the I'd say the logic Expressions that an AI learns from its experience are mostly totally opaque to human beings and maybe even harder to understand than they're on that because I mean when you have multiple nested quantifier bindings it's a very high level of abstraction there is a difference though in that within logic it's a little more straightforward to pose the problem of like normalize this and boil this down to a certain form I mean you can do that in neural Nets too like you can distill a neural net to a simpler form but that's more often done to make a neural net that'll run on an embedded device or something it's it's harder to distill a net to a comprehensible form than it is to simplify logic expression to a comprehensible form but but it doesn't come for free like what's what what's in the ai's mind is is incomprehensible to to a human unless you do some special work to make it comprehensible so on the on the procedural side there's some different and sort of interesting Voodoo there I mean if if you're familiar in computer science there's something called the curry Howard correspondence which is a onetoone mapping between proofs and programs so every program can be mapped into a proof every proof can be mapped into a program you can model this using category Theory and a bunch of a bunch of of of nice math but we want to make that practical right so that so that if you if you have an executive program that like Mo moves a robot's arm or figures out in what order to say things in a dialogue that's a procedure represented in opencog hypergraph but if you want to reason on how to how to improve that procedure you need to map that procedure into logic using Curry Howard the isomorphism so then the logic the logic engine can reason about how to improve that procedure and then map that back into the procedural representation that is efficient for execution so again that comes down to not just can you make your procedure into a bunch of nodes and links because I mean that can be done trivially a a C++ compiler has nodes and links inside it can you boil down your procedure into a bunch of nodes and links in a way that's like hierarchically decomposed and simple enough you can reason about yeah yeah that given the resource constraints at hand you can map it back in forth to your to your term logic like fast enough and without having a bloated logic expression right so there's just a lot of there's a lot of nitty-gritty particulars there but I'm I'm by the same to and if you if you ask a chip designer like how do you make the Intel i7 chip so good right there's a there's a long list of of technical answers there which which will take take a while to go through right and this has been Decades of work I mean the the first AI system of this nature I tried to build was called Web mind in the mid1 1990s and we had a big graph a big graph operating in Ram implemented with Java 1.1 which was a terrible terrible implementation idea and then each each node had its own processing so like that there the core Loop looped through all nodes in the network and that each node enact what it what its little thing was doing and we had logic and neural Nets in there but and evolutionary learning but we hadn't done enough of the math to get them to operate together very cleanly so it was really it was quite a horrible mess so as as well as shift doing an implementation where the graph is its own object and the agents are separately scheduled we've also done a lot of work on how do you represent programs how do you represent procedures you know how do you represent genotypes for evolution in a way that the interoperability between the different types of learning associated with these different types of knowledge actually works and that's been quite difficult it's taken decades and it's totally off to the side of what the commercial mainstream of the of the AI I field is doing which isn't thinking about representation at all really although you could see like in the DNC they had to think a little bit about how do you make representation of a map in this memory Matrix work together with a representation needed for say visual pattern recognition in the hierarchical neural network but I would say we have taken that direction of taking the types of knowledge you need for different types of learning like declarative procedural attentional and how do you make these types of knowledge represent in a way that allows cross learning across these different types of memory we've been prototyping and experimenting with this within opencog and before that web mine since the mid mid 1990s now disappointingly to all of us this has not yet been cashed out in an A in an AGI system right I mean we've used this system within our consult Consulting business so we've built natural language processing and robot control and financial analysis we've built a bunch of sort of vertical Market specific proprietary AI projects they use opencog on on the back end but we we haven't that's not the AGI goal right that's that's it's interesting but it's not the AGI goal so now what what we're looking at with our rebuilded the system 2.0 yeah we're also calling it true AGI so we're not quite sure what the what what the name what the name is yet that we we made a website for 2 ai. but we we haven't put anything on there yet so we may come up with an an even even better name but it's kind of like the real AI starting point for your a but I like true better because true has like you can be true-hearted right you can be true to your girlfriend so true true has true has a number and it also has logic in it right because logic is is a key so yeah with with with the with the true AGI system we're sticking with the same basic architecture but we're we're we're trying to build on what we've learned one thing we've learned is that you know we need type checking among dependent types to be much faster and among probalistic dependent types to be much faster so as it is now you can have complex Types on the nodes and links but if you want to put like if you want types to be first class citizens so that you can have the types can be variables and then and then you do type checking among complex higher order types you can do that in the system now but it's very slow this is stuff like it's done in in Cutting Edge program languages like like agda or something these obscure research languages on the other hand we've been doing a lot time together deep neural Nets with symbolic learning so we did a project for Cisco for example which was on this was Street Scene analysis but they had deep neural models for a bunch of cameras watching Street Scenes but they trained to different model for each camera because they couldn't get the transfer learing to work between camera a and Camera B so we took what came out of all the deeper models for the different cameras we fed it into an opencog symbolic representation then we did some pattern Mining and some reasoning on what came out of all the different cameras within the symbolic graph and that worked well for that application I mean Yugo latapi from Cisco gave a talk touching on that at last year's AGI conference it was in Shenzhen on the other hand we learned from there it was kind of clunky to get the Deep neural models to work well with the symbolic system because we were using torch and torch keeps uh a sort of State computation graph but you needed like real time access to that computation graph within our hyper graph and we we we certainly did it Alexa poov who leads our St Petersburg team wrote a great paper on cognitive modules in opencog explaining sort of how do you deal with the torch compute graph inside opencog but in the end we realized like that just hadn't been one of our design thoughts when we when we built opencog right so between wanting really fast dependent type checking and wanting much more efficient interoperation between the computation graphs of deep neural net Frameworks and opencog hypergraph and adding on top of that wanting to more effectively run an opencog hypergraph distributed across Ram in 10,000 machines which is we're doing dozens of machines now but it's just not we we we didn't architect it with that sort of modern scalability in mind so these performance requirements are what have driven us to want to to rearchitecturing with the current infrastructure that was you know built in the phase 2001 to 2008 which is which is is is hard is hardly shocking right so well I mean the three things you mentioned are really interesting so what do you think about in terms of interoperability uh communicating with the computational graph of Neal networks what do you think about the representations that neural networks form um they're bad but there's many ways that you could that you could deal with that so I've been wrestling with this a lot in some work on on supervised grammar induction and I have a simple paper on that that I'll give it the next a AGI conference the online portion of which is next week actually so what is grammar induction so this isn't AGI either but it it's sort of on the verge between nari and AGI or something unsupervised grammar induction is the problem throw your AI system a huge body of text and have it learn the grammar of the language that produced that text so you're you're not giving it labeled examples so you're not giving it like a thousand sentences where the parses were marked up by graduate students so it's just got to infer the grammar from from from the text it's like it's like the Rosetta Stone but worse right because you only have the one language yeah and you have to figure out what what is the grammar so that's not really AGI because I mean the the way a human learns language is not that right I mean we we learn from language that's used in context so it's a social embodied thing we see we see how a given is grounded in in observation there's an interactive element I guess to yeah yeah on the other hand so I'm I'm more interested in that I'm more interested in making an AGI system learn language from its social and embodied experience on the other hand that's also more of a pain to do and that that would lead us into Hansen Robotics and their robotics work I've done which we'll talk about in a few minutes but just as an intellectual exercise as a learning exercise trying to learn grammar from a corpus is very very interesting right and and that's been a field in AI for a long time no one can do it very well so we've been looking at Transformer neural networks and tree Transformers which are amazing these came out of uh of Google Google brain actually and actually on that team was Lucas Kaiser who used to work for me in in one the period 200 5 through 8 through 8 or something so it's been fun to see my former sort of AGI employee disperse and do all these amazing things way too many sucked into Google actually well yeah anyway we'll talk about that too Lucas Kaiser and a bunch of these guys they they create Transformer networks that classic paper like attention is all you need and all these things following on from that so we're looking at Transformer networks and like these are able to I mean this is what underlies gpt2 and gpt3 and so on which are very very cool and have absolutely no cognitive understanding of any of the Texs are at like they're they're very intell they're very intelligent idiots right so uh sorry to take but a small bring us back but do you think gpt3 understands no not all it understands nothing he's a complete idiot but brilliant idiot you don't think GPT uh 20 will understand langage no no no AB size is not going to buy you understanding any more than a faster car is going to get you to Mars yeah okay it's a completely different kind of thing I mean these networks are very cool and as an entrepreneur I can see many highly valuable uses for them and as as an as an artist I I love them right so I mean I I we we're using our own neurom model which is along those lines to control the Philip K dick robot now and it's amazing to like train train a neurom model on the robot Philip K dick and see it come up with like craze stoned philosopher pronouncements very much like what Philip kadik might have said right like that so these models are are are super cool and I'm I'm working with Hansen robotics now on using a similar but more sopade one for Sophia which which we we we have we haven't launch launched yet but so I think it's cool but no these but it's not understanding these are recognizing a large number of shallow patterns that they're not they're not forming an abstract representation and that's the point I was coming to when we're looking at grammar induction we tried to mine patterns out of the structure of the Transformer Network and you can but the patterns aren't what you want they're they're nasty so I mean you if you do supervised learning if you look at sentences where you know the correct parts of a sentence you can learn a matrix that Maps between the internal representation of the Transformer and the parts of the sentence and so then you can actually train something that will output the sentence parse from the Transformer Network's internal State and we we did this I think uh Christopher Manning some some some others have now done this also but I mean what you get is that the representation is horribly ugly and is scouted all over the network and doesn't look like the rules of grammar that you know are the right rules of grammar right it's kind of ugly so what what we're actually doing is we're using a symbolic grammar learning algorithm but we're using the Transformer neural network as a sentence probability Oracle so like when when you if you have a rule of grammar and you wen't sure if it's a correct rule of grammar or not you can generate a bunch of sentences using that rule of grammar and a bunch of sentences violating that rule of grammar and you can see the the Transformer model doesn't think the sentences obeying the rule of grammar are more probable than the Sens is disobeying the rule of grammar so in that way you can use the neural model as a sense probability Oracle to guide guide a symbolic grammar learning process and that's to work better than trying to milk the grammar out of the neural network that doesn't have it in there so I think the thing is these neural Nets are not getting a semantically meaningful representation internally by and large so one line of research is to try to get them to do that and in infog gam was trying to do that so like if if you look back like two years ago there was all these papers on like Ed Edward this probalistic programming neural NET Framework that Google had which came out of infog so the the idea there was like you you could train an infogan neural net model which is a generative associative Network to recognize and generate faces and the model would automatically learn a variable for how long the nose is and automatically learn a variable for how wide the eyes are or or how big the lips are or something right so it automatically learn the the these variables which have a semantic meaning so that that was a rare case where a neuronet trained with a fairly standard Gan method was able to actually learn the semantic representation so so for many years many of us tried to take that the next step and get a Gant type neural network that that would have not just a list of semantic latent variables but would have say a baset of semantic latent variables with dependencies between them the whole programming framework Edward was was was made for that I mean no one got to work right and it could you think it's possible yeah do you think I don't I don't know it might be that back propagation just won't work for it because the gradients are too screwed up maybe you could get to work using CES or some like floating Point evolutionary algorithm right we tried we didn't get it to work eventually we just paused that rather than gave it up we paused that and said well okay let's let's try more innovative ways to learn implic to learn what are the representations implicit in that Network without trying to make it grow in inside that Network and I I described how we're doing that in language you can do similar things in Vision right so use it as an oracle yeah yeah yeah so you can that's one way is you use a structure learning algorithm which is symbolic and and and then you use the the Deep neural net as an oracle to guide the structure learning algorithm the other way to do it is like infog gam was trying to do and try to tweak the neural network to have the symbolic representation inside it I I tend to think what the brain is doing is more like using the Deep neural net type thing as an oracle like I think the the visual cortex or the cerebellum are probably learning a non semantically meaningful opaque Tangled representation and then when they interface with the more cognitive parts of the cortex the cortex is sort of using those as an Oracle and learning the abstract representation so if you do Sports say take for example serving in tennis right I mean my tennis serve is okay not not great but I learned it by trial and error right and I mean I learned music by trial Nar 2 I I just sit down and play but then if you're an athlete which I'm not a good athlete I mean then you'll watch videos of yourself serving and your coach will help you think about what you're doing and you'll then form a declarative representation but your cerebellum maybe didn't have a declarative representation same way with music like I will hear something in my head I'll sit down and play the thing like I heard it and then then I will try to study what my fingers did to see like what what did you just play like how how did you do that right because if you're composing you may want to see how you did it and then declaratively morph that in some way that your fingers wouldn't wouldn't think of right but the the physiological movement may come out of some opaque like cbella rein rein reinforcement learned thing right and so that's I think trying to milk the structure of a neuronet by treating as an oracle maybe more like how your declarative mind postprocesses what what what your your visual or or or motor cortex I mean I mean in Vision it's the same way like you can recognize beautiful art much better than you can say why you think that piece of art is beautiful but if you're trained as an art critic you do learn to say why and some of it's  but some of it isn't right some of it is learning to map sensory knowledge into declarative and and and lingu and linguistic knowledge yet without necessarily making the sensory system itself use use a transparent and easily communicable representation yeah that's fascinating to think of NE networks as like dumb question anwers that you can just milk to build up uh a knowledge base and there could be multiple networks I suppose from different uh yeah yeah so I think if if a group like deep mind or open AI were to build AGI and I think Deep Mind mind is like a thousand times more likely from from from from what I could tell but cuz they've hired a lot of people with broad Minds in many different approaches and and angles on on AGI worse open AI is also awesome but I see them as more of like a pure deep reinforcement learning shop time I got you there's a lot of there you're right there's um I mean there's so much interdisciplinary work at Deep Mind like neuroscience together with Google brain which granted they're not working that closely together now but you know my oldest son zarra is doing his PhD in machine learning applied to automated theorem proving in in Prague under Joseph Urban so the the first paper deep math which applied deep neural Nets to guide theor improving without of Google brain I mean by now by now the the automated theorem proving Community is gone way way way beyond anything go Google was doing but still yeah that but anyway if that Community was going to make an AGI probably one way they would do it was you know take 25 different neural modules architected in different ways maybe resembling different parts of the brain like a Bas basil ganglia model cerebellum Model A Thal palus model few few hippocampus models number of different models representing parts of the cortex right take all of these and then wire them together to to to co- Trin and like learn them together like that that would be an approach to creating an an an AGI one could Implement something like that efficiently on top of our our true AGI like opencog 2.0 system once it exists although obviously Google has has their own highly efficient implementation architecture so I think that's a decent way to build AGI I was very interested in that in the mid90s but I mean the knowledge about how the brain works sort of pissed me off like was it wasn't there yet like you know in the hippocampus you have these concept neurons like the so-called grandmother neuron which everyone laughed at it it's actually there like I have some Lex Friedman neurons that fire differentially when I when I see you and not when I see any other person right yeah so how how do these Lex Friedman neurons how do they coordinate with the distributed representation of Lex Freedman I have in my cortex right there's some back and forth between cortex and hippocampus that lets these discreet symbolic representations in hippoc campus correlate and cooperate with the distributed representations in cortex this probably has to do with how the brain does its version of abstraction and quantifier logic right like you can have a single neuron hippocampus that that activates a whole distributed activation pattern in in cortex well this this maybe how the brain does like symbolization and abstraction as in in functional programming or something but we can't measure it like we we we don't have enough electrodes stuck between the the cortex and the and the hippoc campus and any known experiment to measure it so I got I got frustrated with that direction not cuz it's impossible because we just don't understand enough yet we don't of course it's a valid research direction and you can try to understand more and more and we are measuring more and more about what what happens in the brain now than ever before so it's it's quite interesting on the other hand I sort of got more of an engineering mindset about AGI I'm like well okay we don't know how the brain works that well we don't know birds fly that well yet yet either we have no idea how a hummingbird flies in terms of the the aerodynamics of it on the other hand we know basic principles of like flapping and and and pushing the air down and we know the basic principles of how the different parts of the brain work so let's take those basic principles and engineer something that embodies those Bas basic principles but you know is welld designed for the hardware that that we have on on hand right right now so do you think we can create AGI before we understand how the brains I think I think that's probably that's probably what will happen and maybe the AGI will help us do better brain Imaging that will then let us build artificial humans which is very very interesting to us because we are humans right I mean building artificial humans is is super worthwhile I I just think it's probably not the shortest path to AGI so it's fascinating idea that we would build AGI to help us understand ourselves uh you know a lot of people ask me if uh you know the young people interested in doing artificial intelligence they look at sort of uh you know doing graduate level even undergrads but graduate level research uh they see what the artificial intelligence Community stands now it's not really AGI type research for the most part yeah so the the natural question they ask is what advice would you give I mean maybe I could ask uh if people were interested in working on uh open Cog or in some kind of direct or indirect connection to open Cog or AGI research what would you recommend opencog first of all is open- source project there's a there's a Google group uh dis discussion list there's a GitHub repository so if anyone's interested in lending a hand with that aspect of of of AGI introduce yourself on the open opencog email list and uh there's a slack as well I mean we're we're certainly interested to have uh you know in inputs into our redesign process for a new version of opencog but but also we're doing a lot of very interesting research I mean we're working on on data analysis for covid clinical trials we're working with Hansen robotics we're doing a lot of cool things with the current version of of of opencog now so there there's certainly opportunity to jump into opencog or or various other open- source a AGI oriented projects so would you say there's like Masters and phg thesises in there plenty yeah plenty of of course I mean the challenge is to find a supervisor who wants to Foster that that that sort of research but it's way easier than it was when I got my PhD right so okay great we talked about open Cog which is kind of uh one the software framework but also the actual uh attempt to build an AGI system and then there is this exciting idea of Singularity net so maybe can you say first what is singularity net sure sure Singularity net is a platform for realizing a decentralized network of of artificial intelligences so Marvin Minsky the AI Pioneer who who I knew a little bit he had the idea of a society of Minds like you should achieve an AI not by writing one algorithm or one program but you should put a bunch of different AIS out there and the different AIS will interact with each other each playing their own role and then the totality of the Society of AIS would would be the thing that displayed the human level intelligence and I had when he was alive I had many debates with with Marvin about about this idea and he I think uh he really thought the mind was more like a society than than I do like I think I think you could have a a mind that was as disorganized as a human society but I think a humanlike mind has a bit more central control than that actually like I mean we have this Thalamus and the medulla and lyic system we we have a sort of top- down control system that guides much of much of what we do more so than than a society does so I think he stretched that metaphor a little too far but I but I also think there's there's something interesting there and so in the in the '90s when I started my first sort of non-academic AI project web mind which was an AI startup in New York in the Silicon alley area in in the late 90s what I was aiming to do there was make a distributed Society of AIS the different parts of which would live on different computers all around the world and each one would do its own thinking about the data local to it but they would all share information with each other and Outsource work with each other and cooperate and the intelligence would be in in in the whole Collective and I organized a conference together with Francis hean at free University of Brussels in 2001 which was the global brain zero conference and we're we're planning the next version of the global brain one conference at the Free University of Brussels for next year 2021 so 20 20 years after then we maybe we can have the next one 10 years after that like exponentially faster until the singularity comes right uh the timing is right yeah yeah yeah exactly so the yeah the idea with the global brain was you know maybe the AI won't just be in a program on one guy's computer but the AI will be you know in the internet as a whole with the cooperation of different AI modules living in different places so one of the issues you face when architecting a system like that is you know how how is the whole thing controlled do you have like a centralized control unit that pulls the puppet strings of all the different modules there or do you have a fundamentally decentralized Network where the Society of of AIS is controlled in some democratic and self-organized Way by all the AIS in in that Society right and Francis and I had different view of many things but we both we both wanted to make like a global Society of AI Minds with a decentralized or organ organization mode now the main difference was he wanted the individual AIS to be all incredibly simple and all the intelligence to be on the collective level whereas I thought that was cool but I thought a more practical way to do it might be if some of the agents in the Society of Minds were fairly generally intelligent on their own so like you could have a bunch of open cogs out there and a bunch of simpler learning systems and then these are are all cooperating and coordinating together soort of like in the brain okay the brain as a whole is the general intelligence but some parts of the cortex you could say have a fair rid of general intelligence on their own whereas say parts of the cerebellum or lyic system have very little general intelligence on their own and they're contributing to general intelligence you know by way of their connectivity to to other other modules do you see instantiations of the same kind of you know maybe different versions of open Cog but also just the same version of open Cog and maybe many instantiations of it as part as that's what David and H and I want to do with many Sophia and other robots yeah yeah each one has its own individual mind living on the server but there's also a collective intelligence infusing them and a part of the mind living on the edge in each robot right yeah so so the the thing is at that time as well as webmind being implemented in Java 1.1 as like a massive distributed system yeah that you know the there blockchain wasn't there yet so how how them do this decentralized control you know we sort of knew it we knew about distributed systems we knew about encryption so I mean we had the key principles of what underlies blockchain now but I mean we didn't put it together in the way that's it's been done now so when when vitalic butterin and colleagues came out with aium blockchain you know many many year years later like 2013 or something then I was like well this is interesting like this is solidity scripting language it's kind of dorky in a way and I don't see why you need a turn complete language for this purpose but on the other hand this is like the first time I could sit down and start to like script infrastructure for decentralized control of the AIS in a society of Minds in a tractable way like you could hack the Bitcoin cbase but it's it's really annoying whereas sady is is ethereum scripting language is just nicer and and easier to use I'm very annoyed with it by this point but like Java I mean these languages are amazing when when they first come out so then I came up with the idea that turned into Singularity that okay let's let's make a decentralized agent system where a bunch of different AIS you know wrapped up in say different Docker containers or lxc containers different AIS can each of them have their own identity on the blockchain and the coordination of this community of AIS has no Central controller no dictator right the and there's no Central repository of information the coordination of the Society of Minds is done entirely by the decentralized network in a in a decentralized way by the by the algorithms right because you know the model of Bitcoin is in math We Trust right and so that that that's what you need you need the Society of Minds to trust only in math not trust only in one one centralized server so the AI systems themselves are outside of the blockchain but then the communication at the moment yeah yeah we I would have loved to put the ai's operations on chain in some sense but in ethereum it's just too slow you you you you you can't you can't do it somehow it's the basic communication between AI systems that's uh yeah yeah so basically an AI is just some software in singular an AI is just some software process living in a container M and there's input and output there's a proxy that lives in that container along with the AI that handles the interaction with the rest of of of Singularity net and then when one AI wants to contribute with another one in the network they set up a number of channels and the setup those channels uses the ethereum blockchain but once the channels are set up then data flows along those channels without without having to be having to be on the blockchain all that goes on the blockchain is the fact that some data went along that channel so you can do so there's not a shared knowledge uh it's well the the identity of each agent is on the blockchain right on the ethereum blockchain if one agent rates the reputation of another agent that goes on the blockchain and agents can publish what apis they will fulfill on the on the blockchain but the actual data for AI and the results AI is not on the blockch do you think it could be do you think it should be um in some cases it should be in some cases maybe it shouldn't be but I mean I I I think that uh so I'll give you an example using ethereum you can't do it using now there's more modern and faster blockchains where you could you could start to do that in in in in in some cases two years ago that was less so it's a very rapidly evolving ecosystem so like one example maybe you can comment on uh something I worked a lot on is autonomous vehicles you can see each individual vehicle as a AI system and uh you can see vehicles from uh Tesla for example and then uh Ford and GM and all these has also like larger I mean they all are running the same kind of system on each sets of vehicles so it's individual AI systems and individual vehicles but it's all different station is the same AI system within the same company so you know you can Envision a situation where all of those AI systems are put on Singularity net right yeah and how how do you see that happening and what would be the benefit and could they share data I gu I guess one of the biggest things is that the power there is in a decentralized control but uh the benefit would have been is is really nice if they can somehow share the knowledge in an open way if they choose to yeah yeah yeah those are those are all all quite good points so I I think the the benefit from being on the on the decentralized network as we envision it is that we want the AIS and the network to be Outsourcing work to each other and making a API calls to to each other frequently I got you so the real benefit would be if that AI wanted to Outsource some cognitive processing or data processing or data pre-processing whatever to some other AIS in the network which specialize in in something different and this this really requires a different way of thinking about AI software development right so just like object-oriented programming was different than imperative programming and now object or programmers all use these Frameworks to do things rather than just libraries even you know shifting to agent-based programming where your AI agent is asking other like live realtime evolving agents for feedback in what they're doing that's a different way of thinking I mean it's it's not a new one there was loads of papers on agent-based programming in the 80s and onward but if you're willing to shift to an agent-based model of development then you can put less and less in your AI and rely more and more on interactive calls to other AIS running in in the network and of course that's not fully manifested yet because although we've rolled out a nice working version of singular unet platform there's there's only 5200 AIS running in there now there's not tens of thousands of of AI so we don't have the critical mass for the whole Society of Mind to be doing doing what what we want what we want the magic really happens when it's just a huge number of Agents yeah yeah exact exactly in terms of data we're partnering closely with another blockchain project called ocean protocol and ocean protocol that's uh the project of Trent Miki who developed Big Chain DB which is a blockchainbased database so ocean protocol is basically blockchainbased big data and nams at make making it efficient for for different AI processes or or statistical processes or whatever to to share L large large data sets or one process can send a clone of itself to work on the other guy's data set and send results back and so forth so by getting ocean and and you know you have data lake so this is the data ocean right so by getting ocean and Singularity net to to interoperate we're aiming aiming to take into account of of the Big Data aspect also but it's it's quite challenging because to build this whole decentralized blockchainbased infrastructure I mean your competitors are like Google Microsoft Alibaba and Amazon which have so much money to put behind their centralized infrastructures plus they're solving simpler algorithmic problems because making it centralized in some ways is is is easier right so they're they're very major computer science challenges and I think what what you saw with the whole icoo boom in in the blockchain and cryptocurrency world is a lot of young hackers who are hacking Bitcoin or ethereum and they see well why don't we make this decentralized on blockchain then after they raise some money through an Ico they realize how hard it is it's like like actually we're wrestling with incredibly hard computer science and software engineering and distributed systems problems which are can be solved but they're just very difficult to solve and in some cases the individuals who started those projects were not were not well equipped to to actually solve the problems that they wanted so you think would you say that's the main bottleneck if uh if you look at the future of currency uh you know the question is currency the main B bck is politics like it's government and the bands of armed thugs that will shoot you if you bypass their their currency restrictions that's right so like your sense is that versus the technical challenges because you kind of just suggested the technical challenges are quite high as well I mean for making a distributed money you could do that on alar end right now I mean so that while ethereum is too slow there's algorand and there's a few other more modern more scalable blockchains it would work fine for a a decentralized global global currency right so I think there were technical bot next to that two years ago and maybe ethereum 2.0 will be as fast as Al I I don't know that's not that's not F fully written yet right so I think the obstacle to currency being put on the blockchain is that is the other currency will be on the blockchain it'll just be on the blockchain in a way that enforces centralized control and government hedge money rather than otherwise like the ER andb will probably be the first Global the first currency on the blockchain the E Ruble maybe next they're already e Ruble yeah yeah yeah I mean the point that's hilarious digital currency you know makes total sense but they would rather do it in the way that Putin and xiin ping have have have access to the the global keys for everything right then so and then the analogy to that in terms of Singularity net I mean there's Echo I I think you've mentioned before that Linux gives you hope and AI is not as heavily regulated as money right not yet right not yet oh that's a lot slipperier than money too right I mean money is is easier to regulate because it it's kind of easier to to Define whereas AI is it's almost everywhere inside everything where's the boundary between Ai and software right I mean if you're going to regulate AI there's no IQ test for Every Hardware device that has a learning algorithm you're going to be putting like honic regulation on all software and I don't rule out that that sof yeah but how do you tell if software is adaptive and what every software is going to be adaptive I mean or maybe they they maybe uh the you know maybe we're living in the Golden Age of Open Source that will not that will not always be open maybe uh it'll become centralized control of software by government it it is entirely possible and part of what I think we're doing with things like Singularity protocol is creating a tool set that can be used to counteract that sort of thing say a similar thing about Mesh networking right plays a minor role now the ability to access Internet like directly phone to phone on the other hand if your government starts trying to control your use of the internet suddenly having mesh working Mesh networking there can be very convenient right and so right now something like a decentralized blockchainbased a AGI framework or or narrow AI framework it's cool it's it's nice to have on the other hand if government start trying to T down on my AI interoperating with someone's AI in in Russia or somewhere right then suddenly having a decentralized protocol that nobody owns or controls becomes an extremely valuable part of the of the tool set and you know we've we've put that out there now it's not perfect but it but it it it operates and you know it's pretty blockchain agnostic so we're talking to algorand about making part of single run run on algorand my good friend TWY CBA has a cool blockchain project called TOA which is a blockchain without a distributed Ledger it's like a whole other architecture so there so there there there's a lot of more advanced things you can do in the blockchain world Singularity net could be ported and to a whole bunch of it could be made multi-chain and port to a whole bunch of different blockchains and there there's a lot of potential and a lot of importance to putting this kind of tool set out there if you compare to opencog what you could see is opencog allows tight integration of a few AI algorithms that share the same knowledge store in real time in in Ram right Singularity net allows loose integration of multiple different AIS they they can share knowledge but they're mostly not going to be sharing knowledge in in Ram in RAM on on on the same machine and I think what we're going to have is a network of network of networks right like I mean you you have the knowledge graph in inside inside the the opencog system and then you you have a network of machines inside a distributed opencog mind but then that opencog will interface with other AIS doing deep neural Nets or or custom biology data analysis or what whatever they're doing in Singularity net which is a looser integration of different AI some of which may be may be their their their own networks right and I think at a very loose analogy you could see that in the human body like the brain has regions like cortex or hippocampus which tightly interconnect like cortical columns with it within the cortex for example then there's looser connection within the different loes of the brain and then the brain interconnects with the endocrine system and different parts of the of the body even even more Loosely then your body interacts even more Loosely with the other other people that you talk to so you often have networks within networks within networks with progressively looser coupling as as as you get get higher up in that hierarchy I mean you have that in biology you have that in in the internet as a just networking medium and I think I think that's what we're going to have in the network of of software processes leading to to AGI that's a beautiful way to see the world uh again the same similar question is with open Cog if somebody wanted to build an AI system and plug into the singularity net what would you recommend like how so that's much easier I mean o Open Cog is still a research system so it takes some expertise to in sometime we have tutorials but it's it's somewhat cognitively labor intensive to get up to speed on on on opencog and I mean what's one of the things we hope to change with the true AGI opencog 2.0 version is just make make the learning curve more similar to tensor flow or torch or something is right now open Cog is amazingly powerful but but not simple to not simple to deal with on the other hand Singularity net you know as a as a open platform was developed a little more with usability in mind although the blockchain is is still kind of a pain so I mean I mean if you're a command line guy there's a command line interface it's quite easy to you know take na AI that has an API and lives in a Docker container and put it online anywhere and then it joins the global Singularity net and Anyone who puts a request for services out into the singularity net the peer-to-peer Discovery mechanism will find your your AI and if it does what what was asked it will it can then start a conversation with your AI about whether it wants to ask your AI to do something for it how much it would cost and so on so that that that's that's fairly simple if you wrote an AI and want it listed on Like official Singularity net Marketplace which is which is on our website then we we have a a publisher portal and then there's a kyc process to go through because then we have some legal liability for what goes on on that website so the in a way that's been in education too there's sort of two layers like there's the open decentralized protocol and there's the market yeah anyone can use the open decentralized protocol so say some developers from Iran and there's brilliant guys in University of isvan and Tran they can put their stuff on Singularity net protocol and just like they can put something on the internet right I don't control it but if we're going to list something on the singularity net Marketplace and put a little picture and a link to it yeah then if I put some Iranian AI genius's code on there then Donald Trump can send a bunch of Jack booted thugs to my house to to arrest me for doing business with Iran right so so I mean we we already see in some ways the value of having a decentralized protocol because what I hope is that someone in Iran will put online an Iranian Singularity net marketplace right which you can pay in a cryptographic token which is not owned by any country and then if you're in like Congo or somewhere that doesn't have any problem with Iran you can subcontract AI services that you find on on on on that marketplace right even though US citizens can't by by US law so right now that's kind of a point you know as as you alluded if if regulations go in the wrong direction it could become more of a major point but I think it also is the case that having these workarounds to regulations in place is a defense mechanism against those regulations being put into place and you could see that in the music industry right I mean Napster just happened and bit torrent just happened and now most people in my kids generation they're baffled by the idea of paying for music right I mean my dad pays for music but I mean yeah but because these decentralized mechanisms happened and then the regulations followed right and the regulations would be very different if they'd been put into place before there was Napster and bit Tor and so forth so in the same way we got to put AI out there in a decentralized vein and Big Data out there in a decentralized vein now so that the most advanced AI in the world is fundamentally decentralized and if that's the case that's just the reality The Regulators have to deal with and then as in the music case they're going to come up with regulations that sort of work with the with the decentralized reality beautiful you were the chief scientist of Hansen robotics uh you're still involved with Hansen robotics uh doing a lot of really interesting stuff there this is for people who don't know the company that created sopia the Robot can you tell me who who Sophia is I'd rather start by telling you who David Hansen is because it's David is the brilliant mind behind this Sophia robot and he remains so far he remains more interesting than his than his creation alth although she may be improving faster than he is actually I mean's yeah so yeah I met a good point I met David maybe 2007 or something at some futurist conference we were both speaking at and I could see we had a great great deal in common I mean we we're both kind of crazy but we also we we both had a passion for AGI and and and the singularity and we were both huge fans of the work of uh Philip KCK the the science fiction writer and I wanted to create benevolent AGI that that would uh you know create massively better life for all humans and all sensient beings including animals plants and superum beings and David he wanted exactly the same thing but he had a different idea of of how to do it he wanted to get computational compassion like he wanted to get machines that that would would love people and empathize with people and he thought the way to do that was to make a machine that could you know look people eye to eye face to face look at look at people and make people love the machine and the Machine loves the people back so I thought that was very different way of looking at it cuz I'm very math oriented and I'm just thinking like what is the abstract cognitive algorithm that will let the system you know internalize the complex patterns of human values blah blah blah whereas he's like look you in the face in the eye and love you right so so I we we we hit it off quite well and we talk to each other off and on then I moved to Hong Kong in 20 20 11 so I'd been I mean I've been I've been living all over the place I've been in Australia and New Zealand in my AC academic career then in in Las Vegas for a while was in New York in the late 90s starting my my entrepreneurial career was in DC for 9 years doing a bunch of US Government consulting stuff then moved to Hong Kong in in in 2011 mostly because I met a Chinese girl who I fell in love with we we we got married she's actually not from Hong Kong she's from mland China but we converge together in Hong Kong still married now have have a 2-year-old baby so went to Hong Kong to see about a girl I guess yeah pretty pretty pretty much yeah and on the other hand I started doing some cool research there with gou at Hong Kong poly Technic University I got involved with a project called idea using machine learning for stock and Futures prediction which was quite interesting and I also got to know something about the consumer electronics and Hardware manufacturer ecosystem in Shenzhen across the border which is like the only place in the world that makes sense to make complex consumer electronics at large scale and low cost it's just it's astounding the hardware ecosystem that you have in in in in South China like you us people here cannot imagine what it what what it's like so David was starting to explore that also I invited him to Hong Kong to give a talk at Hong Kong Pou and I introduced him in Hong Kong to some investors who were interested in his robots and he didn't have Sophia then he had a robot of Philip K dick our favorite science fiction writer he had a robot Einstein he had some little toy robots that looked like his his son Zeno so through the investors I connected him to he managed to get some funding to basically Port Hansen robotics to Hong Kong and when he first moved to Hong Kong I was working on AGI research and also on this uh machine learning trading project so I didn't get that tightly involved with with Hansen robotics but as as I hung out with David more and more as we were both there in the same place I started to get I started to think about what you could do to make his robots smarter than they were and so we started working together and for a few years I was Chief scientist and head of software at at Hansen robotics then when I got deeply into the blockchain side of things I I stepped back from that and co-founded Singularity net David Hansen was also one of the co-founders of of of singularity in that so part of our goal there had been to make the blockchainbased like Cloud mind platform for sopia and the other other other sopia would be just one of the robots in this uh ins Singularity net yeah yeah yeah EXA ex exactly Sophia many copies of the Sophia robot would would would be you know among the user interfaces to the globally distributed singular net Cloud mind and I mean David and I talked about that for quite a while before co-founding s Singularity by the way in his in his vision and your vision was uh was Sophia tightly coupled to a particular AI system or was the idea that you can plug you could just keep plugging in different AI systems within the I think David's David's View was always that sopia would be a platform much like say the pepper robot is is is a platform from SoftBank should be a platform with a set of nicely designed apis that anyone can use to experiment with their different AI algorithms on on on that platform and Singularity net of course fits right into that right because Singularity net it's an API Marketplace so anyone can put their AI on there opencog is a little bit different I mean David likes it but I'd say it's my thing it's not his like David has a little more passion for biologically based approaches to AI than I do which which makes sense I mean he's really into human physiology and and biology's he's a character sculptor right yeah so yeah he he's interested in but he also worked a lot with rule-based and logic based AI systems too so yeah he's interested in not just Sophia but all the H and robots as a powerful social and emotional robotics platform and you know what I saw in sopia was a a way to you know get AI algorithms out there in front of a whole lot of different people in an emotionally compelling way and part of my thought was really kind of abstract connected to AGI ethics and you know many people are concerned AGI is is gonna enslave everybody or turn everybody into into computronium to to make extra hard drives for for for their their cognitive engine or whatever and you know emotionally I'm not driven to that sort of of paranoia I'm I'm really just an optimist by nature but intellectually I have to assign the nonzero probability to those sorts of nasty outcomes because if you're making something 10 times as smart as you how can you know what it's going to do there's an irreducible un uncertainty there just as my dog can't predict what I'm going to do tomorrow so it seemed to me that based on our current state of knowledge the best way to bias the agis we create toward benevolence would be to infuse them with love and compassion the way the way that we do our own children so you want to interact with AIS in the context of doing compassionate loving and benef icial things and in that way as your children will learn by doing compassionate beneficial loving things alongside you in that way the AI will learn in practice what it means to be compassionate beneficial and loving it will get a sort of ingrained intuitive sense of this which it can then abstract in in its own way as it gets more and more intelligent now David saw this the same way that's why he came up with the name Sophia which means which means wisdom so it seemed to me making these like beautiful loving robots to be rolled out for beneficial applications would be the perfect way to roll out early stage AGI systems so they can learn from people and not just learn factual knowledge but learn human human values and ethics from people while being their you know their home service robots their education assistants they're they're nursing robots so that that was the Grand Vision now if you've ever worked with robots the reality is is quite different right like the first principle is the robot is always broken work I mean I worked with robots in the 90s a bunch when you had to solder them together yourself and I'd put neural Nets doing reinforcement learning on like overturn solid Bowl type robots in in the 90s when I was a professor things of course Advanced a lot but but the principle still holds the principle of the robots always broken still holds yeah yeah so faced with reality of making Sophia do stuff many of my Robo AGI aspirations were temporarily cast aside and I mean there's just a practical problem of making this robot interact in a meaningful way because like you know you put nice computer vision on there but there's always glare and then or it you have a dialogue system but at the time I was there like no speech text algorithm could deal with Hong Kong Hong Kong people's English accents yeah so the the speech of text was always bad so the robot always sounded stupid yeah because it wasn't getting the right text right so I started to view that really as what what in software engineering you call a walking skeleton which is maybe the the wrong metaphor to use for Sophia or maybe the right one but I mean what a walking skeleton is in software development is if you're building a complex system how do you get started but one way is to First build part one well then build part two well then build part three well and so on another way is you make like a simple version of the whole system and put something in the place of every part the whole system will need so that you have a whole system that does something and then you work on improving each part in the context of that whole integrated system so that's what we did on a software level in Sophia we made like a walking skeleton Software System where so there's something that sees there's something that hears there's something that moves there's something that there's something that remembers there's something that learns you put a simple version of each thing in there and you connect them all together so that the system will will will do its thing so there's there's a lot of AI in there there's not any AGI in there I mean there's computer vision to recognize people's faces recognize when someone comes in the room and leaves try to recognize whether two people are together or not I mean the dialogue system it's a mix of like hand-coded rules with deep neural Nets that that come up with with with their with their own responses and there's some attempt to have a narrative structure right and sort of try to pull the conversation into something with a be beginning middle and end and this sort of story arc so it's I mean like if you look at the lobner prize and the the systems that beat the touring test currently they're heavily rule-based because uh like you had said narrative structure to create compelling conversations you currently new networks cannot do that well even with Google Mina um when you actually look at fullscale conversations it's just yeah this is the thing so we've been I've actually been running an experiment the last couple weeks taking Sophia's chatbot and then the Facebook's Transformer chatbot which they open the model we've had them chatting to each other for a number of weeks on the server just that's funny gen we're generating training data of what Sophia says in a wide variety of conversations but we can see compared to Sophia's current chatbot the Facebook deep neural chatbot comes up with a wider variety of fluent sounding sentences on the other hand it Rambles like mad the Sophia chatbot it's a little more repetitive in in the sentence structures it uses on the other hand it's able to keep like a conversation Arc over a much longer longer period right so there now you can probably surmount that using reformer and like using various deep neural architectures and to improve the way these Transformer models are trained but in the end neither one of them really understands what's going on and I mean that's the challenge I had with Sophia is if I were doing a robotics project aimed at AGI I would want to make like a robo toddler that was just learning about what it was seeing because then the language is grounded in the experience of the robot but what Sophia needs to do to be Sophia is talk about sports or or the weather or or robotics or the conference she's talking at like yeah she needs to be fluent talking about any damn thing in the world and she doesn't have grounding for for all for all those all those things so there's there's this just like I mean Google Mina and Facebook's chap I don't have grounding for what they're talking about about either so in in a way the need to speak fluently about things where there's no non-linguistic grounding pushes what you can do for Sophia in the short term a bit a bit away from uh from I mean it pushes you towards uh IBM Watson uh situation where you basically have to do heuristic and hardcode stuff and Rule based stuff I have to ask you about this okay so because uh you know in in part Sophia is like an uh is an art creation because it's beautiful uh it's she's beautiful because she inspires through our human nature of uh anthropomorphize things we immediately see an intelligent being there because David is a great sculptor is is great sculptor that's right so uh in fact if Sophia just had nothing inside her head said nothing if she just sat there we already prescribe some intelligence to there's a long selfie line in front of her after every talk that's right so it captivated the imagination of the um of many people I was going to say the world but yeah I mean a lot of people and uh billions of people which is amazing it's amazing right now of course uh many people have prescribed much greater prescribed essentially AGI type of capabilities to Sophia when they see her and of course um friendly French folk like uh Yan laon IM immediately see that of the people from the AI community and get really frustrated because uh it's understandable so what and then they criticize people like you who sit back and don't say anything about like basically allow the imagination of the world allow the world to continue being captivated uh so what what's your what's your sense of that kind of annoyance that the AI Community has well I I I think there's several parts to my reaction there first of all if I weren't involved with Hansen robach and didn't know David Hansen personally I probably would have been very annoyed initially at Sophia as well I mean I can understand the reaction I would have been like wait all these stupid people out there think this is an AGI but it's not an AGI but they're tricking people that this very cool robot is an AGI and now those of us you know trying to raise funding to build AGI you know people will think it's already there and and and already works right so I yeah on the other hand I think even if I weren't directly involved with it once I dug a little deeper into David and the robot and the intentions behind it I think I would have stopped being being pissed off whereas folks like Yan Lun have remained pissed off after their after their after their initial well their initial re his thing that's his thing yeah I think that in particular struck me as somewhat ironic because Yan Lun is working for Facebook which is using machine learning to program the brains of the people in the world toward vapid consumerism and political extremism so if if your ethics allows you to use machine learning in such a blatantly destructive way why would your ethics not allow you to use machine learning to make a lovable theatrical robot that draws some foolish people into it its theatrical illusion like if if if the if the push back had come from Yoshua Benjo I would have felt much more humbled by it because he's he's not using AI for blatant evil right on the other hand he also is a super nice guy and doesn't bother to go out there trashing other other people's work for no good reason right son Sha's fired but I get you I I mean that's I mean if if you're if you're gonna ask I'm I'm gonna answer but no for sure I think we'll go back and forth I'll talk to Yan again I would add on this though I mean David Hansen is an artist and he often speaks off the cuff and I have not agreed with everything that David has said or done regarding Sophia and David also was not agree with everything David has said her done about important point I mean d David David is an artistic uh Wild Man and that's that's that that's part of his charm that's that's part of his genius so certainly there have been conversations within Hansen Robotics and between me and David where I was like let's let's be more open about how this thing is working and I did have some influence in in nudging Hansen robotics to be more open about about how Sophia was working and and David wasn't especially opposed to this and you know he was actually quite right about it what he said was you can tell people exactly how it's working and they won't care they want to be drawn into the illusion and he was 100% 100% correct I'll tell you what yeah this wasn't Sophia this was Philip K dick but we did some actions between humans and Philip KCK robot in Austin Texas a few years back and in this case the Philip KCK was just teleoperated by another human in the other room so during the conversations we didn't tell people the robot was teleoperated we just said here have a conversation with Phil dick we're going to film you right and they had a great conversation with Phil K dick tell operated by my friend Stan buy after the conversation we brought the people in the back room to see Stefan who was controlling the the the the Philip K dick robot but they didn't believe it these people were like well yeah but I know I was talking to Phil like maybe Stefan was typing but the spirit of Phil was animating his mind while he was typing yeah so like even though they knew was a human in the loop even seeing the guy there they still believe that was Phil they were talking to a small part of me believes that they were right actually because our understand well we don't understand the universe right I mean there is a cosmic mind field that we're all embedded in that yields many strange synchronicities in in in in the world which is a topic we don't have time to go into too much here I mean there there's there's some nature there's something to this where uh our imagination about Sophia and people Yan Lon being frustrated about it is all part of this beautiful dance of creating artificial intelligence that's almost essential you see with Boston Dynamics I'm a huge fan of uh as well you know the kind of I mean these robots are very far from intelligent uh I I I played with her last one actually with the spot mini yeah very cool I mean it it reacts quite in a fluid and and flexible way right but we immediately ascribe the kind of intelligence we immediately ascribe AGI to them yeah yeah if you kick it and it falls down and goes ow you feel bad right you can't help it yeah and uh I mean that's that's that's uh part of uh that's going to be part of our journey in creating intelligence systems more and more and more and more like as uh as Sophia starts out with a walking skeleton as you add more and more intelligence I mean we're going to have to deal with this kind of idea absolutely and about Sophia I would say I mean first of all I have nothing against Yan L this is F this is nice guy if he if he wants to play the media media banter game I'm I'm I'm I'm I'm happy to he's a good researcher and and a good human being and I'd happily work with the guy but the other thing I was going to say is I have been explicit about how Sophia works and I've posted online and that what H+ magazine an online web Zine I mean I posted a moderately detailed article explaining like there are three software systems we've used inside sopia there's there's a timeline editor which is like a rule-based authoring system where she's really just being an outlet for what a human scripted there's a chatbot which has some rule-based and some neural aspects and then sometimes we've used opencog behind Sophia where there's more learning learning and reasoning and you know the funny thing is I can't always tell which system is operating here right I mean so when she whether she's really learning yeah or thinking or or just appears to be over half hour I could tell but over like 3 or 4 minutes of interaction I I so even having three systems that's already sufficiently complex where you can't really tell right away yeah the thing is even if you get up on stage and tell people how Sophia is working and then they talk to her they still attribute more agency and Consciousness to her than than is is is really there so I think there's there's a couple levels of ethical issue there one issue is should you be transparent about how Sophia is is working and I think you should and and I think I think we we have been I mean I mean it's there's articles online that there's some TV special that goes through me explaining the three subsystems behind Sophia so the way Sophia works is is out there much more clearly than how Facebook say I works or something right I mean we've been fairly explicit about it the other is given that telling people how it works doesn't cause them to not attribute too much intelligence agency to it anyway then then should you keep fooling them when they want to be fooled and I mean the you know the whole media industry is based on fooling people the way they want to be fooled and we we are fooling people 100% toward a good end I mean I mean we are we are playing on people's of empathy and compassion so that we can give them a good user experience with helpful robots and so that we can we can fill the ai's mind with love and compassion so I've been I've been talking a lot with Hansen robotics lately about collaborations in the area of of medical Robotics and we we haven't quite pulled the trigger on a project in that domain yet but we we may well do so quite soon so we've been we've been talking a lot about you know robots can help with with elder care robots can help with kids David's done a lot of things with h with autism therapy and robots robots before in the co era having a robot that can be a nursing assistant in various senses can be quite valuable the robots don't spread infection and they they can also deliver more attention than human nurses can give right so if you have a robot that's helping a patient with covid if that patient attributes more understanding and compassion and agency into that robot than it really has because it looks like a human I mean is that really bad I mean we can tell them it doesn't fully understand you and and they don't care because they're lying there with a fever and they're sick but they'll react better to that robot with its loving warm facial expression than they would to a pepper robot or or a metallic looking looking robot so it's it's really it's about how you use it right if you made a human looking like doorto door sales robot that used its its human looking appearance to to scam people out of their money yeah then you're using that that connection in in a bad way but you you could also use it in in a in a good way and that but then that's the same the same problem with every technology right beautifully put so like you said uh we're living in uh the era of the covid This Is 2020 one of the craziest years uh in recent history so uh if if we zoom out and look at this pandemic uh the Corona virus pandemic maybe let me ask you this kind of thing in in viruses in general when you look at viruses do you see them as as a kind of intelligence system I think the concept of intelligence is not that natural of a concept in the end I mean I I think human minds and bodies are a kind of complex self organizing adaptive system and viruses certainly are that right they're a very complex self-organizing adaptive system if you want to look at intelligence as as Marcus Hooter defines it as sort of optimizing computable reward functions over computable environments for sure viruses are doing that right and and I mean in in in doing so they're they're causing some some harm to us and that so there there you know the human immune system is a very complex self organizing adaptive system which has a lot of intelligence to it and viruses are also adapting and dividing into new Mutant strains and and and so forth and ultimately the solution is going to be nanotechnology right I mean I mean the solution is going to be making little Nanobots that fight the viruses or well people will use them to make nastier viruses but hopefully we can also use them to just detect combat and and kill the viruses but I think now now we're stuck with the biological uh mechanisms to to combat these these viruses and yeah know we've been AGI is not yet mature enough to use against Co but we've been using machine learning and also some machine reasoning in in opencog to help some doctors to do personalized medicine against covid so the problem there is given a person's genomics and given their clinical medical indicators how do you figure out which combination of antivirals is going to be most effective against covid for for that person and so that that's something where machine learning is interesting but also we're finding the abstraction we get an open Cog with machine reasoning is interesting because it can help with transfer learning when you have not that many different cases to study and qualitative differences between different strains of of a virus or people of different ages who may have Co so there's a a lot of different disparate data to work with and it's small data sets and somehow integrating them you know this is one of the shameful things it's very hard to get that data so I mean we're working with a couple groups doing clinical trials and and they're sharing data with us like under non-disclosure but what should be the case is like every covid clinical trial should be putting data online somewhere like suitably encrypted to protect patient privacy so that anyone with the right AI algorithms should be able to help analyze it and any biologist should be able to analyze it by hand to understand what they can right instead instead that data is like siloed inside whatever hospital is running the clinical trial which is completely asinine and and ridiculous like what why why the world works that way I mean we could all analyze why but it's insane that it does you look at this hyd hydrochloroquine right all these clinical trials were done were reported by surgisphere some little company no one ever heard of and everyone paid attention to this so they were doing more clinical trials based on that then they stopped doing clinical trials based on that then they started again and why isn't that data just out there so everyone can analyze it and and and see what's going on right you hope that uh we'll move uh that data will be out there eventually for future pandemics I mean do do you have hope that our society will move in the direction of not in the immediate future because the US and China frictions are getting very high so it's hard to see us and China as moving in the direction of openly sharing data with each other right it's it's not there's some sharing of data but different groups are keeping their data private till they've mil the best results from it and then they share it right so it's so yeah we're working with some data that we've managed to get our hands on something we're doing to do good for the world and it's a very cool playground for for like putting deep neural it's an open Cog together so we have like a bio adom Space full of all sorts of Knowledge from many different biology experiments about human longevity and from biology knowledge bases online and we can do like graph to Vector type embeddings where we take nodes from the hypergraph embed them into vectors which can then feed into neural nets for different different types of analysis and we were doing this in the context of a project called uh reu that we spun off from Singularity net to do longevity longevity analytics like understand why people live to 105 years or over and other people don't and then we had this spinoff Singularity Studio where we're working with some some healthc care companies on on data analytics but so this bio space we built for these more commercial and Longevity data analysis purposes were're repurposing and feeding Co data into the same same bio atom space and playing around with like graph embeddings from that graph into neural nets for bioinformatics and so it's it's both being a cool testing ground some of our bio AI learning and reasoning and it seems we're able to discover things that people weren't seeing otherwise because the thing in this cases for each combination of antivirals you may have only a few patients who've tried that combination and those few patients may have their particular characteristics like this combination of three was tried only on people age 80 or over this another combination of three which has an overlap with the first combination was tried more on young people so how do you combine those those different pieces of data it's a very dodgy transfer learning problem which is the kind of thing that the probalistic reasoning algorithms we have inside opencog are better at than deep neural networks on the other hand you have gene expression data where you have 25,000 genes and the expression level of each gene in the peripheral blood of each person so that sort of data either deep neural Nets or to like XG boost or cat boost these decision forest trees are better at dealing with than open Cog because it's just these huge huge messy floating Point vectors that that are annoying for a logic engine to to deal with but are are are perfect for a decision forest or a neural net so it's it's a great playground for like hybrid hybrid AI methodology and we can have Singularity net have open Cog and one agent and XG boost in a different agent and they talk to each other but at at the same time it's it's highly practical right because we're working with we're working with for example some Physicians on this project in thep physicians in the group called anopinion based out of uh of Vancouver and Seattle who are these guys are working every day like in the hospital with with patients patients dying of covid so it's it's quite cool to see like neural symbolic AI like where the rubber hit hits the road trying trying to save people's lives I've been I've been doing bio AI die since 2001 but mostly human longevity research and fly longevity research try to understand why some organisms really live a long time this is the first time like race against the clock and try to use the AI to figure out stuff that like if we take two months longer to solve the AI problem some more people will die because we don't know what combination of antivirals to give them yeah at the societal level at the biological level at any level are you hopeful about us as a human species getting out of this pandemic what are your thoughts on any general well the pandemic will be gone in a year or two once there's a vaccine for it so I I mean that's that that's but a lot of pain and suffering can happen in that time so I mean that could be irreversible I mean I I think if you spend much time in subsaharan Africa you can see there's a lot of pain and suffering happening all the time like you walk through the streets of any large city in subsaharan Africa and there are loads of I mean tens of thousands probably hundreds of thousands of people Lying by the side of the road dying mainly of curable diseases without without food or water and either ostracized by their families or they left their family house because they didn't want to infect their family right I mean there's tremendous human suffering on the planet all the time time which most folks in the developed World pay no attention to and Co is is not remotely the worst how many people are dying of malaria all the time I mean So Co is bad it is by no mean the worst thing happening and setting aside diseases I mean there are many places in the world where you're at risk of having like your teenage son kidnapped by armed militias and forced to get killed in someone else's War fighting tribe again tribe I mean so Humanity has a lot of problems which we don't need to have given the state of advancement of our our technology right now and I think Co is one of the easier problems to solve in the sense that there are many brilliant people working on vaccines we have the technology to create vaccines and we're going to we're going to create new vaccines we should be more worried that we haven't managed to defeat malaria after so long and after the Gates Foundation and others putting putting so much so much money in into it I mean I think clearly the whole Global Medical system Global health system and the global political and socioeconomic system are incredibly unethical and unequal and and badly designed and I mean I don't know how to solve that directly I think what we can do indirectly to solve it is to make systems that operate in parallel and off to the side of the of the governments that are are nominally controlling the world with with our armies and and militias and to the extent that you can make compassionate peer-to-peer decentralized Frameworks for doing things these are things that can start out unregulated and then if they get tractioned before The Regulators come in then they've influenced the way the world works right Singularity net aims to do this with with AI ruv which is a spin-off from from Singularity net you can see ru. IO that how do you spell that re eju V ru. iio that aims to do the same thing for medicine so it's like peer-to-peer sharing of medical data so you can share Medical Data into a secure data wallet you can get advice about your Health and Longevity through through through apps that that that Ru will launch within the next couple months and then Singularity AI can analyze all this data but then the benefits from that analysis are spread among all all the members of of of the network but I mean of course I'm going to Hawk my particular projects but I mean whether or not Singularity and and ruv are are the answer I think it's key to create decentralized mechanisms for everything I mean for AI for for human health for politics for for jobs and employment for sharing social information and to the extent decentralized peer-to-peer methods designed with universal compassion at the core can gain traction then these will just decrease the role that government has and I think that's much more likely to do good than trying to like explicitly reform the global government system I mean I'm happy other people are trying to explicitly reform the global government system on the other hand you look at how much good the internet or or Google did or mobile phones did you mean you're making something that's decentralized and throwing it out everywhere and it takes hold then then government has to adapt and I mean that's what we need to do with with AI and with health and in that light I mean the centralization of healthcare and of AI is is certainly not ideal right like most AIP phgs are being sucked in by you know a half dozen dozen big companies most AI processing power is is being bought by a few big companies for their own proprietary good and most medical research is within a few pharmaceutical companies and clinical trials run by pharmaceutical companies will stay solid within those pharmaceutical companies you know these large centralized entities which are intelligences in themselves these corporations but they're mostly malevolent Psychopathic and sociopathic intelligences not saying the people inv vved but the corporations as self-organizing entities on their own which are concerned with maximizing shareholder value as as as a sole objective function I mean Ai and Medicine are being sucked into these pathological corporate organizations with government cooperation and Google cooperating with British and and US Government on this as one among many many different examples 23 and me providing you the nice service of sequencing your your genome and then licensing the genome to glos Smith Klein on an exclusive basis right right now you can take your own DNA and do whatever you want with it but the pulled collection of 23 and me sequence DNA is just to to to gacos Smith Klein someone else could reach out to everyone who who had worked with 23 and me to sequence their DNA and say give us your DNA for our our open and decentralized repository that will make available to everyone but nobody's doing that because it's a pain to get organized and the customer list is proprietary to 23 in me right so yeah I mean this this I think is a greater risk to humanity from AI than Rogue AGI turning the universe into paper clips or or computronium because what you have here is mostly good-hearted and nice people who are sucked into a mode of organization right of large corporations which has evolved just for no individual's fault just because that's the way Society has evolved is not altruistic get self-interested and become Psychopathic like you said Corporation is psychopathic even if the people are not and that exactly that's really the disturbing thing about it because the corporations can do things that are quite bad for society even if nobody has nobody has a has a bad intention right and then no individual member of that Corporation has a bad intention no some probably do but they don't but but it's not necessary that they do for the for the corporation like I mean Google I know lot of people in Google and they're with very few exceptions they're all very nice people who genuinely want what what's good for the world and Facebook I know fewer people but it's probably most it's probably mostly true it's probably like fine young Geeks Who who want to build cool technology I actually tend to believe that even the leaders even Mark Zuckerberg one of the most disliked people in Tech is also wants to do good for the world what do you think about Jamie demon who's Jamie demon oh the heads of the Great may have a different psychology oh boy yeah well I tend to um I tend to be naive about these things and see see the best in uh I I I tend to agree with you that I think the individuals want to do good by the world but the mechanism of the company can sometimes be its own intelligence syst I mean there there's a one my cousin Mario gel has worked for Microsoft since 1985 or something and I can see for him I mean as well as just working on cool projects you're coding stuff that gets used by like billions and billions of of people and you think if I improve this feature that's making billions of people's lives easier right so of course of course that's cool and you know the engineers are not in charge of running the company anyway and of course even if you're Mark Zuckerberg or Larry Page I mean you still have a fiduciary responsibility and I mean you you responsible to the show sh holders your employees who you want to keep paying them and so forth so yeah you're Imes in this system and you know when I worked in DC I worked a bunch with inscom US Army intelligence and I was heavily politically opposed to what the US Army was doing in Iraq at that time like torturing people in Abu graa but everyone I knew in US Army in inscom when I hung out with him was a very nice person they were friendly to me they were nice to my kids and and my dogs right and they really believed that the US was fighting the forces of evil and if you ask them about Abu gayab they're like well but these Arabs will chop us into pieces so how can you say we're wrong to waterboard them a bit right like that's much less than what they would do to us it's just in in their world view what they were doing was really genuinely for the for the good of humanity like none of them woke up in the morning and said like I I want to do harm to good people because I'm I'm just a nasty guy right so yeah most people on the planet setting aside a few genuine Psychopaths and sociopaths I mean most people on the planet have a heavy dose of benevolence and wanting to do good and also a heavy capability to convince themselves whatever they feel like doing or whatever is best for them is is for the good of humankind right and so the more we can decentralize control of decentralization you know the democracy is horrible but this is like when Church Hill said you know it's the worst possible system of government except for all the others right I mean I think the whole mess of humanity has many many very bad aspects to it but so far the track record of elite groups who know what's better for all of humanity is much worse than the track record of the whole teaming Democratic participatory mass of humanity right I mean none of them is perfect by by any means the issue with a small Elite group that knows what's best is even if it starts out as truly benevolent and doing good things in accordance with its initial good intentions you find out you need more resources you need a bigger organization you pull in more people internal politics arises difference of opinions arise and bribery happens like some some opponent organization takes a second and command out to make some the first in command of some some other organization and I mean that's there's a lot of history of of what happens with Elite groups thinking they know what's best for for for the human race so you have if I have to choose I'm going to reluctantly put my faith in the vast Democratic decentralized mass and I think corporations have a track record of being ethically worse than their constituent human parts and you know democratic governments have a more mixed track record but there are at least but it's the best we got yeah I mean you can you can there's Iceland very nice country right I mean very de Democratic for 800 plus years very very benevolent beneficial government and the I think yeah there are track records of democratic Modes of organization Linux for example some of the people in charge of Linux are overtly complete right and trying to reform themselves in in many cases in in other cases not but the organization as a whole I think it it it's it's done a good job over overall it's been very welcoming in in in in the third world for for example and it's it's allowed advanced technology to roll out on all sorts of different embedded devices and Platforms in places where people couldn't afford to pay for for proprietary software so I'd say the internet Linux and many democratic nations are examples of how sort an open decentralized Democratic methodology can be ethically better than than the sum of the parts rather than worse and corporations that has happened only for a brief period and and then and then then it go it goes sour right I mean I'd say a similar thing about universities like University is a horrible way to organize research and get things done yet it's better than anything else we've come up with right a company can be much better but for a brief period of time and then then it stops stops being so good right so then I I think if you believe that AI is going to emerge sort of incrementally out of AI doing practical stuff in the world like controlling humanoid robots or or driving cars or diagnosing diseases or operating killer drones or spying on people and Reporting on the government then then what kind of organization creates more and more advanced narrow AI verging toward AGI may be quite important because it will guide like what's what's in the mind of the early stage AGI as it first gains the ability to rewrite its own code base and project itself toward toward super intelligence and if you believe that AI may move toward AGI out of the sort of synergetic activity of many agents cooperating together rather than just have one person's project then who owns and controls that platform for AI cooperation becomes also very very important right and is that platform AWS is it Google cloud is it Alibaba or is it something more like the internet or Singularity net which is open and de open and decentralized so if if all of my weird machinations come to pass right I mean we have we have the Hansen robots being a beautiful user interface you know gathering information on on on human values and being loving and compassionate to people in medical home service robow about office applications you have Singularity net in the back end networking together many different AIS toward Cooperative intelligence fueling the robots among many other things you have opencog 2.0 and true AGI as one of the sources of AI inside this decentralized network powering the robot and medical AIS helping us live a long time and cure diseases am among among other things and this whole thing is operating in a in a democratic and and decentralized way right I think if if anyone can pull something like this off you know whether using the specific Technologies I've mentioned or or or something else I mean then I think we have a higher odds of moving toward a beneficial technological singularity rather than one in which the first Super AGI is indifferent to humans and just considers us an INE efficient use of molecules that was a beautifully articulated vision for the world so thank you for that but let's talk a little bit about life and death I'm I'm pro-life and anti death speak well you for for most people there's few exceptions that I won't mention here I'm I'm I'm glad just like your dad you're taking a stand against uh death uh you have uh by the way you have a bunch of Awesome music where you play Piano online well one of the songs that I believe you've written uh the lyrics go by the way I like the way it sounds people should listen to it's awesome I was I considered I probably will cover it it's a good song uh tell me why do you think it is a good thing that we all get old and die is one of the songs I love the way it sounds but let me ask you about death first do you think there's an element to death That's essential to give our life meaning like the fact that this thing ends the say I'm I'm uh pleased and a little embarrassed you've been listening to that music I put online that's awesome one of my regrets in life recently is I would love to get time to really produce music well like I I I haven't touched my sequencer software in like five years like I I would love to like rehearse and produce and and edit and but the with a two-year-old baby and and trying to create the singularity there's no time so I I just made the decision to when I'm playing random in an off moment just record it just just record it put it out there like like whatever maybe if I'm unfortunate enough to die maybe that can be input to the AGI when it tries to make an accurate mind upload of me right death is bad I mean that's very simple it's battling we should have to say that I mean of course people can make meaning out of out of death and if if someone is tortured maybe they can make beautiful meaning out of that torture and write a beautiful poem about what it was like to be tortured right I mean we we're very creative we can we can milk Beauty and positivity out of even the most horrible and and and shitty things but just because if I was tortured I could write a good song about what it was like to be tortured doesn't make torture good and just because people are able to derive meaning and value from Death doesn't mean they wouldn't derive even better meaning and value from ongoing life W without death which I very definite yeah yeah so if you could live forever would you live forever forever I my my goal with longevity research is to abolish the plague of involuntary death I don't think people should die unless they choose to die if I had to choose forced immortality versus dying I would choose forced immortality on the other hand if I chose if I had the choice of immortality with the choice of suicide whenever I felt like it of course I would take that instead and that's the more realistic choice I mean there there's no reason you should have forced immortality you should be able to live until you get until you get sick of living right I mean that's and that will seem insanely obvious to everyone 50 years from now and they will be so I mean people who thought death gives meaning to life so we should all die they will look at that 50 years from now the way we now look at the anabaptists in the year 1000 who gave away all their positions went on top of the mountain for Jesus for Jesus to come and bring them to the to the Ascension I mean it's it's ridiculous that that people think death is is is good because because you gain more wisdom as you approach dying mean of of of of course it's true I mean I'm 53 and you know the fact that I might have only a few more decades left it does make me reflect on on things differently it it it it it does give me a deeper understanding of many things but I mean so what you could get a deep understanding in in a lot of different ways pain is the same way like we're going to abolish pain and that that that's even more amazing than abolishing death right I mean once we get a little better at Neuroscience we'll be able to go in and adjust the brain so that pain doesn't hurt anymore right and that you know people will say that's bad because there's so much Beauty in overcoming pain and suffering well sure and there's Beauty in overcoming torture too but and some people like to cut themselves but not not many right I mean that's an interesting so but to push I mean to push back again this the Russian side of me I do romanticize suffering it's non obvious I mean the way you put it it's seems very logical it's almost absurd to to romanticize suffering or pain or death but to me a world without suffering without pain without death it's non obvious what you can stay in the people's Zoo the people torturing each other right no but that what I'm saying is I I don't well that's I guess what I'm trying to say I don't know if I was presented with that choice what I would choose because it to me no this this is this is a subtler it's a subtler matter and I've posed it in this conversation in an unnecessarily extreme way so I I think I think the way you should think about it is what if there's a little dial on the side of your head and you could turn how much pain hurt turn it down to zero turn up to 11 like in spinal tap if it wants maybe through an actual spinal typ right so I mean would you opt to have that dial there or not that that that's the question the question isn't whether you would turn the pain down to zero all all all the time would you opt to have the dial or not my my guess is that in some dark moment of your life you would choose to have the dial implanted and then it would be there just to confess a small thing I'm uh don't ask me why but I'm I'm doing this physical challenge currently where I'm doing 680 push-ups and pull-ups a day and and my shoulder is currently as we sit here in a lot of pain and uh I I don't know I would certainly right now if you gave me a dial I would turn that sucker to zero as quickly as possible but I don't I think the whole point of this journey is I don't know well because you're you're a twisted human being I'm a twisted so the question is if am I somehow Twi am I is Twisted because I have I I created some kind of narrative for myself so that I can deal with the with with the Injustice and the suffering in the world uh or is this actually going to be a source of happiness for me well this is this is a to an extent is a research question that Humanity will undertake right so I mean human human beings do have a particular biological makeup which sort of implies a certain probability distribution over motivational systems right so I mean we we we and that that is there well put that is there now the the the question is how flexibly can that morph as society and Technology change right so if if we're given that dial and we're given a society in which say we don't have to we don't have to work for a living and in which there's an ambient decentralized benevolent AI Network that will warn us when we're about to hurt oursel you know if we're in a different context can we consistently with being genuinely and fully human can we consistently get into a state of consciousness where we just want to keep the pain dial turned all the way down and yet we're leading very rewarding and fulfilling lives right now I suspect the answer is yes we can do that but I I don't I don't know that a research I don't know that for certain yeah now I'm more confident that we could create a nonhuman AGI system which just didn't need an analogue of feeling pain and I think that AGI system will be fundamentally healthier and more benevolent than than human beings so I think it might or might not be true that humans need a certain element of suffering to be satisfied humans consistent with the human physiology if it is true that's one of the things that makes us and disqualified to be the be the Su the super AGI right I mean this is a the nature of the human motivational system is that we we seem to gravitate towards situations where the best thing in the large scale is not the best thing in in in the small scale according to our subjective value system so we gravitate towards subjective value judgments where to gratify ourselves in the large we have to UNG gratify ourselves in the in the small and we do that in you see that in in music there's a theory of Music which says the key to musical Aesthetics is the surprising fulfillment of expectations like you you want something that will fulfill the expectations are listed in the prior part of the music but in a way with a bit of a Twist that that that surprises you and that I mean that's true not only an outdoor music like my own or that of Zappa or or Steve V or or Buckethead or Kristoff pendi or something it's even there in in Mozart or something it's not there in elevator music too much but that that's that's that's why that's why it's boring right but wrapped up in there is you know we want to hurt a little bit so that we can we can feel the we can feel the pain go away like We want to be a little a little a little confused by what coming next so then when the thing that comes next actually makes sense it's so satisfying right and it's the surprising fulfillment of expectations is that what you said yeah yeah so beautifully put is there um we've been skirting around a little bit but if I were to ask you the most ridiculous big question of what is the meaning of life uh what would your answer be three values Joy growth and choice I I I think you you need you need Joy I mean that that's the basis of everything if you want the number one value on the other hand I'm unsatisfied with a a static joy that doesn't progress perhaps because of some Elemental element of human perversity but the idea of something that grows and becomes more and more and better and better in some sense appeals to me but I also sort of like the idea of individuality that as a distinct system I have some agencies there's some Nexus of causality within within this system rather than the causality being wholly evenly distributed over the joyous growing Mass so I you start with joy growth and and choice as three basic values that's and those three things could continue indefinitely that's not that's something yeah that can last forever is there is there some aspect of something you called which I like super longevity that you find exciting that what is there you research-wise is there ideas in that space that I mean I I think yeah in terms of the meaning of life this really ties into that because for us as humans probably the way to get the most Joy growth and choice is transhumanism and to go beyond the human form that that that that we have right now right I mean I think human body is great and by no means to any of us maximize the potential for Joy growth and choice imminent in our human bodies on the other hand it's clear that other configurations of matter could manifest even greater amounts of Joy growth and choice than that than than humans do maybe even finding ways to go beyond the realm of matter that as as we understand it right now so I think in a practical sense much of the meaning I see in human life is to create something better than humans and and and and go beyond life but certainly that's not all of it for me in a practical sense right like I have four kids and and and a granddaughter and uh many friends and parents and family and just enjoying everyday human Human Social existence well we can do even better yeah yeah and I mean I I love I've always when I could live live near nature I spend a bunch of time out in nature in the forest and on the water every day and so forth so I mean enjoying the pleasant moment is is part of it but the you know the growth and choice aspect are severely limited by our human biology in particular dying seems to inhibit your potential for personal growth considerably as as as far as we know I mean there's some element of life after death perhaps but even if there is why not also continue going in in in in in this in this biological realm right in in in super longevity I mean you know we we haven't yet cured aging we haven't yet cured death certainly there's very interesting progress all around I mean crisper and and Gene editing can be can be an an incredible tool and I mean right now stem stem cells could potentially prolong life a lot like if if you got stem cell injections of of just stem cells for every tissue of your body injected into every tissue and you can just have replacement of your old cells with new cells produced by those stem cells I mean that that could be highly impactful at prolong life now we just need slightly better technology for for having them grow right so you using machine learning to guide procedures for stem cell differentiation and trans trans differentiation it's kind of nitty-gritty but I mean that that's that that that's quite interesting so I think there's there's a lot of different things being done to help with with prolongation of of human life but we could do a do a lot better so for example The extracellular Matrix which is a bunch of proteins in between the cells in your body they get stiffer and stiffer as you get older and the The extracellular Matrix trans transmits information both electrically mechanically and to some extent biop photonically so there's all this transmission through the parts of the body but the stiffer The extracellular Matrix gets the less the transmission happens which makes your body get get worse coordinated between the different organs as you get older so my friend Christian schafmeister at my alumnus organization the great my alma mother the great Temple University Christian schafmeister has a potential solution to this where he has these novel molecules called spirro ligers which are like polymers that are not organic they're specially specially designed polymers so that you can algorithmically predict exactly how they'll fold very simply so he designed a molecular scissors that have spirro ligers that you could eat and would then would then cut through all the glucosa pain and other cross-link proteins in your extracellular Matrix right but to make that technology really work and be mature is several years of work as far as I know no no one's funding it at the moment but there so there's so many different ways that technology could be used to prolong longevity what what we really need we need an integrated database of all biological knowledge about human beings and model organisms like Bas hopefully a m distribute opencog bio atom space but it can exist in other forms too we need that data to be opened up and a suitably privacy protecting way we need massive funding into machine learning AGI Proto AI statistical research aimed at solving biology both molecular biology and human biology based on this massive massive data set right and and and then we need Regulators not to stop people from trying radical therapies on on on themselves if if they so so wish to as as well as better cloud-based platforms for like automated experimentation on microorganisms flies and mice and so forth and we could do all this you look after the last financial crisis Obama who I generally like pretty well but he gave $4 trillion do to large Banks and insurance companies you know now in this covid crisis trillions are being spent to help Everyday People in small businesses in the end we probably will find many more trillion to being given to large Banks and insurance companies anyway like could the world put1 trillion into making a massive holistic bio Ai and bio simulation and experimental biology infrastructure we could we could put 10 trillion dollars into that without even screwing us up too badly just as in the end Co and the last financial crisis won't screw up the world economy so badly we're not putting 10 trillion into that instead all the resurch is siloed inside a few big companies and and and and government agencies and most of the data that comes from our individual bodies personally that could feed this AI to solve aging and death most of that data is sitting in some some hospitals database doing nothing right I got a uh two more quick questions for you uh one I know a lot of people are going to ask me you on the Joe Rogan podcast wearing that same amazing hat um do you have a origin story for the hat is there does the Hat have its own story that you're uh able to share uh the Hat story has not been told yet so we're going to have to come back and you can you can interview the Hat the Hat we'll leave that for the Hat Zone interview all it's too much it's too much to pack into is there a book is a hat gonna write a book okay we'll uh it may transmit the information through direct neural transmission okay so it's it actually there might be some neuralink competition there uh beautiful we'll leave it as a mystery uh maybe one last question if uh you uh build an AGI system uh you're successful at building the A A system that could lead us to The Singularity and you get to talk to her and ask her one question what would that question be we're not allowed to ask what is the question I should be asking yeah that would be cheating but I guess that's a good question I'm thinking of a I wrote a story with Stefan bugy once where these AI developers they created a super smart AI aimed at answering all the philosophical questions that have been worrying them like what what what is the meaning of life is there free will what is consciousness and so forth so they got the super AGI built and it uh it turned a while it said those are really stupid questions and then it puts off on the spaceship and and and and left the Earth right see be afraid of scaring it scaring it off that that's it yeah I mean honestly there's there there there there is no one question that that rises among among all all the all the all the others really I mean what interests me more is upgrading my Mo my own intelligence so that I I can absorb the whole the whole world world view of the of the super AGI but I mean of course if if the if the answer could be like what's the what is the chemical formula for the immortality pill like then I would do that or emit emit a bit string which uh will be the the code for a super AGI on the Intel i7 processor right so those would be good questions so if you're on mind was expanded to become super intelligent like you're describing I mean there's a you know there there's kind of a notion that with intell intelligence is a burden that it's possible that with greater and greater intelligence the that other metric of joy that you mentioned becomes more and more difficult what's your pretty stupid idea so you think if you're super intelligent you can also be super joyful I think getting root access to your own brain will enable new forms of joy that we don't have now and I I think as I've said before what I aim at is really make multiple versions of myself so I would like to keep one version which is basically human like I am now but you know keep the dial to turn P pain up and down and get rid of death right and make another version which fuses its mind with superhuman AGI and then will become massively transhuman and what whether it will send some messages back to the human me or not is will be interesting to find out the thing is once you're super super AGI like one subjective second to a human might be like a million subjective years to that super AGI right so it would be on a whole different basis I mean at very least those two copies will be good to have but it could could could be interesting to put your mind into into a dolphin or a space amoeba or all sorts of other things or you can imagine one version that doubled its intelligence every year and another version that just became a super AGI as fast as possible right so I mean now we're sort of constrained to think one mind one self one body right but but I think we actually we don't need to be that constrained in in in thinking about future intelligence after we've mastered AGI and nanotchnology ology and Longevity biology I mean then each of our minds is a certain pattern of organization right and I I know we haven't talked about Consciousness but I I sort of I'm pan psychist I sort of view the universe as as conscious and so you know a light bulb or a a quark or an ant or a worm or a monkey have their own manifestations of Consciousness and the human manifestation of Consciousness it's partly tied to the particular meat that that we're manifested by but it's largely tied to the pattern of organization in in in the brain right so if you upload yourself into a computer or a robot or or what whatever else it is some element of a human consciousness may not be there because it's just tied to the biological embodiment but I think most of it will be there and these will be incarnations of your Consciousness in a slightly different flavor and you know creating these different versions will be amazing and each of them will discover meanings of life that have some overlap but probably not total overlap with with the human Bend's meaning meaning of life the the thing is to get to that future where we can explore different varieties of of Joy different variations of human experience and values and transhuman experiences and values to get to that future we need to na navigate through a whole lot of human  of companies and and governments and and killer drones and making and losing losing money and and so and so forth right and that's that that's the challenge we're facing now is if we do things right we can get to a benevolent Singularity which is levels of Joy growth and choice that are literally unimaginable to to human beings if if we do things wrong we could either annihilate all life on the planet or we could lead to a scenario where say all humans are are annihilated and there's some super AGI that goes on and does it does its own thing unrelated to us except via our our role in in originating it and we may well be at a bifurcation point now right where where what we do now has significant causal impact on what comes about and yet most people on the planet aren't thinking that way whatsoever they're thinking only about their own narrow a narrow aims and Asim aims and goals right now of course I'm thinking about my own narrow aims and goals to some extent also but I'm I'm trying to use as much of my energy and mind as I can to push toward this more benevolent alternative which will be better for me but Al but also for also for everybody else and that's a it's weird that so few people understand what's going on I know you interviewed Elon Musk and he understands a lot of what's going on but he's much more paranoid than I am right because because Elon gets that AGI is going to be way way Smarter Than People yeah and he gets that an AGI does not necessarily have to give a about people because we're very Elementary mode of organization of matter compared to many a many agis but I don't think he has a Clear Vision of how infusing early stage agis with compassion and human warmth can lead to an AGI that loves and helps people rather than viewing us as uh as you know a historical artifact and and a a waste of ma a waste of mass energy but but on the other hand while I have some disagreements with him like he understands way way more of the story than almost anyone else in such a large scale corporate leadership position right it's it's terrible how little understanding of these fundamental issues exists out there now that may be different five or 10 years from now though because I I can see understanding of AGI and Longevity and other such issues is certainly much stronger and more prevalent now than than 10 or 15 years ago right so I mean humanity is as a whole can be slow Learners relative to what what what what I would like but on a historical Sense on the other hand you could say the progress is astoundingly fast but Elon also said I think on The Joe Rogan podcast that love is the answer so uh maybe in that way you and him are both on the same page of how we should proceed with AI I think there's no better place to end it I hope we get to talk uh again about the hat and about Consciousness and about a million topics we didn't cover Ben it's a huge honor to talk to you thank you for making it out thank you for talking today NOK thanks for having me this was this was uh was really really really good fun and uh we dug deep into some very important things so thank thanks for doing this thanks very much awesome thanks for listening to this conversation with Ben geril and thank you to our sponsors the Jordan Harbinger show and Master Class please consider supporting the podcast by going to Jordan Harbinger dcom Lex and signing up to masterclass and masterclass.com Lex click the links buy the stuff it's the best way to support this podcast and the journey I'm on in my research and startup if you enjoy this thing subscribe on YouTube review it with five stars on Apple podcast support on patreon or connect with me on Twitter Alex fredman spelled without the e just f r i d m an I'm sure eventually you will figure it out and now let me leave you with some words from Ben gzel our language for describing emotions is very crude that's what music is for for thank you for listening and hope to see you next time