Polyworld: Using Evolution to Design Artificial Intelligence
Articles Blog

Polyworld: Using Evolution to Design Artificial Intelligence

>>STRICKLAND: Hey there. I’m Henry Strickland,
our speaker is Virgil Griffith he’s talking about Polyworld using evolution to design
artificial intelligence and having had to take artificial intelligence classes in–in
college, I’d be very happy to let evolution do it instead of me debugging all those list
programs they gave me. So, Virgil, as a young lad read a little too much of Douglas Hofstadter
and he therefore dedicated his life to cognitive science and causing trouble. After some under
graduate at University of Alabama, he went to Indiana, where he teamed up with Larry
Jaeger. Some of the older Googlers might know Larry Jaeger form Apple Computer. He had a
project called Polyworld long time ago and it still leaves on and Virgil’s been working
on it and adding features and things to it. Virgil has done internships at the Santa Fe
Institute and at the Keck Institute and now is his first year as a grad student at Caltech.
All right.>>GRIFFITH: Thank you Stu. Hi. I’m Virgil,
I’m a–I’m first year grad student at Caltech. You can reach me, that’s my–that’s my web
site for those of you wondering, the .gr stands for Griffith, people get confused about that.
I’m not Greek. And that’s my email address. So–so, in short, yes, I’ll be talking to
you about basically trying to use evolutionary algorithms as a shortcut to creating artificial
intelligence. Simply because artificial intelligence is well, hard and–and evolution is fairly
easy–well this was easy to set up. And the hope is that–that us–we can take advantage
having lots of the CPU cycles and we let evolution to do a lot of the designing for us. So–so,
that’s the–that’s the general gist and well–well, let’s move on with it. So, there we go. So,
what I interested in–feel like asking, what is artificial life anyway? They just go, you
know, this is ill-defined. Well, in short, artificial life is–is–artificial life is
like a super set of biology. So, all biology is artificial life but to be more precise,
artificial life is all as it is today. So, as it says in the circle, and also what like
potentially could be. So, all these, all their possible evolutionary paths that are–that
evolution could have taken will also ultimately create artificial life and will be [INDISTINCT]
with these areas because we’ll be hoping to explore AI. Please I’ll say it once. So, I
was–so, just to begin, let’s show real quick. So, this is a brief intro to evolution. Evolution
is an algorithm. It’s really straight forward actually. Here’s how it goes. You had a population
and you have–and some things stick around more than others. So, and–but some, yeah,
that must–must be the case. So, that’s for selection. And then, you had these things–there
are some heredity. And then, you rinse-repeat. And regardless of substrate, you always get
evolution with this. Very straight forward. You have a population of things, you–you
only have only one that you have hill climbing and that–that’s crap, you got to have a bunch
and some reproduce more than others straight forward and then there’s heredity. And with–with
occasional errors. Done. It’s all you got to do. So, no matter–just–yeah, it’s great.
So, okay, get that on the table. So moving on, I’m showing you a nice–nice–good example
of using evolution to design body plans so, this is–before we get to AI. And this was
not my work, this is by Carl Sims in 1994, it’s very [INDISTINCT] so I’m showing it to
you. So, basically in this case–so, he’s doing–using evolution to design bodies–design
body–body morphologies to do a different task in the world. In this case, the population
is a–do we have a laser pointer or anything like that? I can just point. Well, anyway,
okay so, the population is a whole bunch of these nodes and connections joining them.
And you can mix and match nodes so, it’s like they say, “Hey. I’m going to put this [INDISTINCT]
over here and vice versa.” And you kind of see, can–how they make these morphologies
you know, about how–you know, how this makes a tree and vice versa. It’s actually kind
of cute when you look at it. So, they’re actually worth understanding so. All right, sweet.
Okay. So, in this case the–yeah, usually there’s joints between parts. So, yeah. So,
this population is a graph of nodes and edges and the–and the selection is to go different
with certain tasks so, walking, jumping, something like that. And the–and the mutation is grafting
nodes here and there. And we’re just going to let it go and see what happens and here
we go. So–no, okay.>>This demonstration shows.
>>GRIFFITH: Trying to…>>Virtual creatures that were evolved to
perform specific tasks in simulated physical.>>GRIFFITH: And that one. All right, start
it again.>>This demonstration shows virtual creatures
that were evolved to perform specific tasks in simulated physical environments. Swimming
speed was used to determine survival. Most of the creatures are results from independent
evolutions. Some developed strategies–is their evolved. Multiple–these creatures in
simulated together–friction. Some simple solutions was just two parts were found. Some
seemed like they could use some assistance while others were fairly efficient such as
this rowing like behavior. Here is an odd cousin of the previous. A mutation caused
him to tumble. Some creatures evolve to incorporate contact sensors in their control systems.
Here is another inch worm like creature that tends to go in circles. This was actually
a creature first evolved for its ability to swim in water then later put on land and evolved
further. A successful side winding ability resulted. Here is one with a hopping style.
The protrusions on its arms seem to help prevent it from tipping over. This was the fastest
with a successful galloping like stride. This group was evolved for their jumping ability.
This group was evolved for their ability to adaptively follow a red light source. The
resulting creatures are now being interacted with. A user is moving the light source around
as the creature behaves. This one seems to flail randomly but somehow still manages to
approach the light. Perhaps it is mean to move the goal away just it is arrives. Here
is one that has propeller like fins which are tilted depending on the direction of the
light. It can adaptively swim up or down very well.
>>Just a pause. This is one is especially nice because it looks like something that
a human would design. Some kind of motor thing and if it weren’t for this little part just
hanging off here, you’d swear it was design and this case is a case where evolution has–has
toned across–they’re are very good designs, extremely efficient and it looks, you know,
very much something that we would build ourselves. So, like basic seeing designs like this should
like–should comfort so yes, this–this can work. Sure, is there a question?
>>[INDISTINCT] recently this [INDISTINCT]>>GRIFFITH: You mean the cross network? X,
this work was recently redone for the Artificial Life Ten Conference. I know that I used to,
to evolve catapult designs. So I don’t–I don’t know if they’ve actual recreated all
of this but–but I do know at least large sections of this have been recreated and I
know that for a fact because I worked in the lab. So, so that’s all I got.
>>I’d like to read this book.>>GRIFFITH: Okay. All right, now I still-what’s
the next one we got here? Oh, so I’ve set some before where they–where they’re moving
kind–kind of weirdly specially the one where this–have like the big hanging mass. The
sole fitting this function in this case was to–was to move your center of mass forward
or just–just move it period. So in this case like–like evolution is very–like it loves
to cheat all the time to–to find some way to do this. So in this case what I was doing
is was have this big long tentacle thing and it was just moving its tentacle thing around.
Thus its–thus its center of mass was moving. So just another thing to keep in mind is that–is
that if you ever have any–you have to–when you design your evolutionary simulations you
have to always know all the weird ways it could cheat and we’ll get back to that later.
So here’s some more.>>This final group of creatures was evolved
through for their ability to compete for control of a green cube. The creature closest to the
cube at the end of the simulation is the winner. Here a strategy first arose for simply tumbling
towards the cube. Then one learned to block out his opponent. But then later one learned
to overcome the obstacle by climbing over it. Some pinned down their opponents. Some
covered the cube with protective arms. Others simply unfolded onto the cube. The success
of this strategy is often highly dependent on the opponent. Here’s a Hockey playing creature,
which takes the cube away and wins by a large margin. Here are two similar Hockey strategies
battling it out with the appropriate gestures. This crab like creature walks well but often
continues past the cube and instead seems to prefer beating up on his opponent. Against
the arm, the crab seems to simply walk away. A successful strategy is this two armed technique
that swipes quickly in from the side and moves the cube over to his second arm. These are
the final rounds of competition amongst the overall best. Finally, the seeker arm goes
against the sideswiper but the cube is just out of reach.
>>GRIFFITH: Okay, so this is a fun movie that I would like to show. Number one, it’s
pretty and the second is because, you know, designing body types–well, that’s kind of
hard like doing those solutions yet I kind of think about them for a little bit. Now
this is not designing AI but it does show ho–how like–how, how evolution can sow across
very inventive solutions. And so this is meant to be like inspiring and say, “Oh, you know,
maybe you can do something else better with this.” So that’s what we have next. So next
is using artificial life to evolve artificial intelligence. So here’s a–well, hear this–this
idea. So the first question is how we do a population for–like, like what, what’s–what
thing do we mutate and tinker with to–force it to be intelligent and there’s a lot of
answers to this question. So Marionettes had–the Greeks had Marionettes and so–yeah, they–they
strings so they’re all deeply connected in this clearly the way you think about intelligence.
And then Descartes says it’s Hydraulics, so the mind it’s like the Sewer system, here
we have little compartments here and then lots of pretty art from that time of all about
it. And Pulleys and Gears such Industrial Revolution–yeah, we have done this before.
Telephone switchboard–yeah, we, we–we’ve heard–we’ve even heard this analogies. But
Boolean logic–yeah, that didn’t go so well. But I’m pleased that we finally solved it.
And the answer is, not digital computers but it’s neural networks. Praise the Lord. So,
so I guess–I mean given the history should partake neural networks was kind of a grain–a
grain of salt. But, you know, definitely there’s some reason to think–think neural networks
are a reasonable way for representing intelligence. I mean, after all, we, we, we really are–like
we’re modeling the brain much, much closer than say digital computer or Boolean logic.
So, so even though there had been many–been many attempts, it’s like what is the proper
frame to–to capture intelligence. You know, the hi–history is not really on our side.
But I still think there’s–there’s a good reason for it. So just–just go with me on
this one. So now, the nervous systems–now, this ends very nice is that, if you look at
the neuron–see a human neuron, like an individual one versus–versus say–say some other mammalian
creature–even reptiles, you often can’ t tell the difference between them. It takes
like a real expert to do it. So like the–like an individual neuron level, we’re all pretty
much the same. It’s all in–it shows in the connections. And evolution and it–like from
us all the way down to like sea slugs. You see–you still see nervous systems that are
roughly the same. So this very nice because roughly this says, “Because hey, if we could
just get our basic model right. You know–you know–was say a sea slug, it could perhaps
ride this model all the way up to the top.” And if evolution did it once, why couldn’t
it do it again? So yeah–so now, we’ll talk about sort of the way–so now we have our
nervous system, the important parts about it. So in this case–so this case, we, we
do not–do know some behaviors are innate. There must be–must be some things that are–that
are, I mean, inherited. We also have many things that are learned. So the–so the nervous
systems must change with the organism’s lifetime. This just–this is just sort of basic principles,
seems reasonable, we’re going to go with that–so not too hard. And with all this in mind–I’m
[INDISTINCT] to you, Polyworld. Tad-dah! This is the simulator. Not to be confused with
Polyworld, we–we–we got a thread about this. So just so you know, this is not us, we’re
the other one. It’s with two L’s, we’re with one. And we do–we do pre-date them but not
that really matters. Okay, so what is Polyworld? Poly–Polyworld is an attempt to–well, before–since
we evolved, artificial intelligence the same way natural [INDISTINCT], which is simply
put the evolution of neuro systems in–in a complex, rich ecology and they compete with
one another. So and we’re–yeah, so the, the hope is that, we, we view with the model to
make very simple and then through competition and through making the world, world richer.
It can gradually like get better and better and better. Sure.
>>[INDISTINCT] what causes the natural world very rapidly if they happen with enormous
perils. How are you going to beat their time schedule?
>>GRIFFITH: That’s hard. I mean, I–let’s see. Well, how would you do that? I guess,
in short, the, the answer would be number one, we can place the ideas that create, or
even an intelligent designer. We, we can help it along. And, and the hope is that, you know,
we can say. “Oh, that’s good. We want to like really help you.” And it’s not something natural
evolution had, had the benefit of. And furthermore, Moore’s law is really nice. And so I agree
with you that is a problem but, but both of those two, two factors help. But it’s, it’s,
it’s, it’s, it’s a legitimate concern. So yeah–and in short–but Polyworld is a new
software, it’s open source. I’ll give you the link at the end. And, and, you know, there’s
a kind of a girls–but most recently, people are using it for doing behavioral ecology
experiments and like–and like–we experience very simple neural networks. So if you’re
side is to use for that to. So now we know what Polyworld is, what Polyworld is not.
So Polyworld is not fully open ended. It’s currently just–just designed to be a flat
world. Well, it’s like–yeah, let’s have your fight–where–where critic interacts. It’s
not an accurate model of really anything. But it could be done. There’s–I mean, there’s–it’s
a–there’s, there’s no real problem with it. The–the only reason we hadn’t made an accurate
model of especiallly anything is because it’s computationally expensive. And we don’t believe
it’s, it’s specially important. So if–so like right now, we’re still using a simple
summing and squashing neurons. If you wanted to, you could like–you could render all the
way down to actual biochemistry if you’re in to that kind of thing. I’m personally not.
But, you know, you could. And if you’re into ecology, you can do that to. So yeah, that’s
what I got. So we want some more. So until we uphold more–so here’s usually what evolves
in Polyworld. So organisms have evolving genes and mate sexually, straight forward. They,
they do have a body but the most important thing about them is the neural network brains.
Now, the connections in the net–in the brains are genetic. But at birth, all the weights
are random. And–and Hebbian learning, which is the learning mechanism and, and, and the
primary that makes the human brain. But simply put well, and that sets all the weights. And
Hale learning, it’s a very simple algorithm. It works like this. If two neurons that are
connected together, fire at about the same time. The connection between them gets stronger.
And then–so that’s step one. And then step two is, all connections decrease in strength
slightly so–and that’s it. It’s, it’s kind of surprising. It’s kind of surprising that–that
it’s this one learning mechanism that accounts most of our intelligence. But you know, so
it goes. And their–and their vision of the world is quite–it inhabits flatland so they
see a little–a little strip of pixels in front of them. And so basically, it’s evolving–it,
it is evolving a neural network to take their one dimensional vision and turn it into behaviors
that help them survive. So–and just to let you know, there’s no cheating in any of this
on as you often see in evolution stimulations. There’s no fitness function. This is like
pure natural selection. This is as raw as it gets. If something survive–like, like
the only criterion is really to survive any way you can. And this includes exploiting
bugs in the code. And we’ll show an example of that. So okay–yeah, so–yeah, I’ll show
you that in a second. So too-too-doo. Okay–go back. Okay, so here’s a nice, pretty picture
of Polyworld. Here’s how it goes. So–where is my thing? Here we go. So, these round things
are barriers, they can’t cross those. These moving things here are the critters. And these
green things here are food. So you see when a critter dies, they become food. Now, this
is kind of an early stage–stage of the stimulator. And so they aren’t very smart. They like going
along the edge a lot. But they get smarter. I promise. So–so–so, basically, merely existing
in this world cause you to lose energy. And if you–and if you–and if your energy gets
to zero, well, you–you seize to exist. So, so thus like for anything to stick around,
it must go out and find food or go out and kill something and–or it must mate with other
organisms as well. If it doesn’t, it just not going to stick around long. It’s, it’s,
it’s pure Darwinian. So–and you can kind of see how it looks here. So here’s the–there’s
a top down view. And in between these little–well, in between these little squares here you saw,
this was the world rendered from one critter’s perception. But it’s a stretched out slightly
for our convenience. But to know exactly, they see–they see the middle strip of pixels
in that. So okay–so that’s Polyworld. So now listen to the, the Genetic model because
I always get asked about that. You have to pay a lot of attention–this is mostly for
reference, for those of you who are into this kind of thing. So these are–so the–I think
before, there are body genes, there are brain genes. And this is the body ones. So here’s
usually how it works. A critter can be big, but when–but when it’s big, it doesn’t move
really fast, but it can hold more energy in it. So, you know, it’s kind of a trade off.
And if a critter wants to be a predator, it can be really strong, so we can do that. And
it can also determine its maximum lifespan. This come–this actually–this actually form
from the evolution literature. They–they said that it–it’s good that we have like
a hard limit that we can’t that–see. It’s good we have a hard limit that if you just–eventually
die of age. Because even though it’s extremely unlikely for something unfit to live a long
time, it’s so utterly bad if something unfit lives a long time and mates a lot that you–that
you want a really hard limit on the–on how long you can live. So this is also kind of
motivated, it’s kind of [INDISTINCT] so like for example, you want to have tons of kids,
but give the most no energy. So it’s the parent can decide how much energy they want to give
them. Or if you want to have a few kids, and gives them lots of your energy. So this is,
you know, whichever you want to use. So we’ll go back to the colors in a little bit, but–yeah.
So the green–how green a particular critter is–is determined at birth. So you could have
like the light green critter and dark green critters, and stuff like that. And also their
mutationry is also specified genetically. So–yeah. No counter points of genetic grade.
Okay. So this exciting part, so this is the brain genes. This is like 95 percent of the
genome. So here’s how it works. So the genetic models specifies which colors you want to
attention to in your environment. So if you think red is really important in your environment,
you can spend a lot of neurons to go see it. Yeah. Also there are internal groups and these
internal neural groups which were like this, and supposed how they’re connected. So the
genetic model only specifies roughly how many connections are between each neural group.
It does not specify at the pure neuron level. And this is motivated from biology. So you
–so if you see–yeah. Like–well, it just is, and stuff we were getting into. So for
those who are neural network buffs, you can be with all that. But the main thing that
takes home from this is that the genes loosely specify–loosely specify the brain, and it
does that in sort of the neural groups level. That’s really the main thing to take from
this. So, to make this clearer, so here is how a typical brain looks. So you have one
neural group here, you have excitatory neurons and inhibitory neutron. We distinct–we distinct–many
neural networks have been the inhibitory neurons and excitatory neurons. They can like, like
a single a single neuron have both excitatory and inhibitory connections. But when you do
that, some biologist puts up their hands and says that brains don’t work like that. And
you say, “Well, fine.” So there, for you biologists in the room, they’re different, be happy.
All right. So you have multiple of these things and they can have different numbers of excitatory–inhibitory
nodes. And they cling to each other. So straightforward. And they connect back, it’s nice. And then
you can have multiple neural groups. And they can all connect to each other however else
they want. Now, these internal neural groups connect to some output neurons or behavior
neurons. And here they are. Now, these are–these are the seven behavior neurons, and they’re
defined in the simulation. And in short, there are things like move forward, turn left, turn
right, eat, mate, fight, blink–I’ll show that in one second, and focus. So basically
every critter has this little light in front of it. That it can sort of, they can–that
they can blink with. The idea is they could use some primitive signaling mechanism. As
far as I know, they haven’t fully–they haven’t taken advantage to this fore signaling. But
you know, you can give room to grow. They obviously can’t evolve from doing it if you
don’t give it to m in the the first place. So it’s in there. And we also weren’t sure
what kind of eye they wanted. So this–so depending on the activity of this neuron,
they can have sort of a fish eye lens where they can have like, you know, really straight.
So, and that’s just only because we weren’t sure what kind of eye they might want. So,
you know, evolution can decide. Sure.>>[INDISTINCT]
>>GRIFFITH: Oh, no. This comes next. Oh, sorry [INDISTINCT]. Okay. So here–so here
are the inputs. Okay. So genetically–so if you’re going to pay attention–so this critter
wants to pay attention a lot to green, a little bit to red and not so much to blue . And so
these are basically the inputs. And these inputs can connect to any of these internal
groups that they want. And it also has an energy level. So this tells you roughly how
healthy the critter is how healthy it is. And it also has sort of a random firing. Just
because, you know, might want it this is the free will of the critter. You can think of
it like that. And I’m surprised that they actually use the random. You wouldn’t really
think so. But they like random. I’m not entirely sure why they like random. But–you know,
well, regardless. We put in there because they might–they might like it, and behold
they do. So…>>[INDISTINCT] networks, how does a [INDISTINCT]
networks?>>GRIFFITH: So like these internal groups
could connect to each other however they want.>>[INDISTINCT] convergence?
>>GRIFFITH: Yes.>>Okay.
>>GRIFFITH: Oh, okay. Yeah, we’ll deal with this later, so this thing’s the input units
and processing units. Not so important. Sure.>>Have you assigned energy cost to neurons?
>>GRIFFITH: Yes. And roughly, the reason we did the…
>>Repeat the questions.>>GRIFFITH: Huh?
>>Repeat the question.>>GRIFFITH: Oh, I’m’ sorry, I was asked whether
or not there’s an energy penalty for–for having a large number of neurons or for neurons
being activated, the answer is yes to both, that problem was you didn’t do this, they
grew huge brains that like 99% did nothing, so you’re just like well, like computation
is just silly, so if you’re going to have a big brain, it better well do something.
So yes they get a cost for having–for just having a size–certain sized brain, or for
neurons being activated, so like doing anything, cost you something. Okay, so good question,
we didn’t do that initially, and that’s what happened. So this is rough–roughly the same
picture I showed you before, and this is made using–using dot, it’s really nice. Oh sorry,
graph this, so this just shows your Polyworld brain map, saying no, really, I’m not joshing
you, that’s how they work, and these are the inputs here, they connect to excitatory neurons
and inhibitory neurons, and these are sort of the behavior neurons, up here, you know,
there’s fight, turn, light, blink, et cetera. So it just kind of shows you what their brains
typically look like, when they are not idealized, so, that’s all you get from that. So, okay,
so as far as the previous concern, everything is about getting energy so they get energy,
they die, and that’s bad. So, here’s how they get energy, they can eat food pellets or they
can eat other critters, straightforward. And they lose energy by doing anything, like merely
existing loses energy, so if they don’t do one of these things, they’re gone. These especially,
like mating cost energy, and being big and strong costs energy and just for having a
brain costs energy, so, mention that. Okay, so now I’m going to show you some behavioral
samples, of how the output neurons, well this is what it looks like, when they turn these
things on. So here’s eating, this neuron right here, and you see it slurps it up, Ta-dah.
So, I’m going to show you some more of these, more into the emergent stuff. So, what’s going
to happen here is that, one critter, so okay–oh I’m sorry, I should mention this, the color
of every critter, is an Archibee triplet, so, the redder a critter is, is how aggressive
it is at this moment, the bluer a critter is, is how much it wants to mate with–mate
with–just mate, at this moment, and the green is genetic, as specified before. The reason
we decided this is because, well you know, you want to know when someone wants to kill
you, you want to know someone wants to mate with you, very straight forward. And for green,
the idea is that you might want to do kin selection. It’s like for example, say hey,
I’m in light green now and I want to be nice to you because you’re a light green. Sure.
So it’s–we’ve seen a few cases where they have done some tribalism based on the green,
but usually you have to kind of like trick it into doing it, but it does happen. So,
based–the important thing is here is that these are both kind of red so they’re going
to do battle, so let’s watch this one. So here we go, he runs into it, and it gets eaten
and it turned into a food pellet and this one slurped up the body. That’s how eating
works. Oh, in this case, so the bigness of a critter is proportional to its strength,
so basically, even though this critter was stronger, it just had like a lower amount
of energy, and it got eaten by the weaker one. Okay, so here’s how mating works, so
this goes–I’m going to come in here, and mate with this one, and a little child will
pop-out. Okay, so now we see what happened here, okay so, they made a child but they
were so, they expended so much energy given the child–they put so much energy into the
child that they immediately died afterward and the child ate their carcasses. So here,
we can see that again, for those of you with kids. What is he going to–with the loop?
Let’s do it, there we go, nope, okay, mating, let’s see it again. Okay, so their coming
on, make the child, and they–they both die, and the child doesn’t really care, slurp,
okay. Next we have is the lighting, this is the blinky, I’ll just show you this. This
is–because it’s me coming here and he’s going to blink at you, so here it comes in oh, I’m
sorry, it turns his blinky off, so right now the blinky is on cause you see that’s his
normal color and that’s the blinky and now it’s turned it off. So, so they could shine
lights at each other. Okay, so, now I’ll show you some–I’ll show you some of the emergent
behaviors. So this is one of the–so we call these things species just because it’s kind
of natural, technically they can still mate with each other but behaviorally they’re so
different that, it’s seemed trees would call them that, so these are joggers, and all they
do, they just go forward all the time. This case the world is–is–is–is-is a toroidal
world so you can’t go off the edge. We have other worlds where you can go off the edge.
And they move in circles a lot. So, by this case, usually the first thing you see in a
simulation, just always go straight. It’s very easy to code and the food is–is–is–is–is
randomly distributed. Why not? I mean you know, you’re–it’s–it’s quick and simple.
So that works. Okay, so this is a really nice one. I talked to you before about how evolution
takes advantage of absolutely anything, like including your bugs. So, this is a very nice
bug. Now, what this was–this was initially done, it had not occurred to me that–that
having a child cost energy. You know, because you–you just do it, it’s pretty easy. So,
you know, that’s my male bias. But, well, I’m sure it happens. When we initially–there
initially was no cost for having children. Guess what happens so you’ll see them, I think
they’re over there and we will zoom in a little bit. So, you see, they’re all in a cluster
over there. And we’re going to zoom. There we go, okay. So, you see that–that they have
this whole orgy going on here and they are–they are popping out kids, like–like looked like,
and then immediately eating them. And with this is–this is because–because eating–eating
the children becomes a free source of energy. So, you have two so as far as the critters
are concerned, you have two choices: you go out and get food or you can mate and have
a piece of food appear right next to you. The solution is clear, and–and this was like
really boggling, like why are they doing that? Because, this would be immensely successful,
we take over everything. And I could–took a lot to figure that out. But yeah, so we–now,
we cost–so, now like it costs energy to have kids. So, now we don’t eat them. It’s not
as–not as–not as not as prevalently. So, okay. So, just–just to let you know that
evolution will take advantage of your bugs. That’s a really good way to test. So, okay.
So, moving on from the indolent cannibals. Okay. So, now I’m going to show you some–so
now I’m going to show you some actually intelligent behavior, at least well, primitive intelligent
behavior, that has emerged form this. So, this is just to show you that yes, this is
actually doing something, all right. Okay. So, we’re going to actually get to see them
act–they actually use their visions. So, [INDISTINCT] come by and this–and the critter
lurched forward. And see that–okay. Here, well’s–okay, there’s more of them. Yeah,
see–see, it jump forward. So, really all this was saying is that hey, they actually
are using their eyes for something and they’re using their eyes to control their behavior.
So, simple enough, not–not very big claim. But you’ll see is that we’re actually getting
something right, like keep in mind when these critters start, they have completely random
brains. And I assure you, they’re crap. They don’t do that. So, I’ll show you examples
if you’d like. Okay, so now I’ll show you some more ones. Here’s fleeing attack. Or
in short running away from red things. So, usually like the first thing–usually the
first things the critters learn is number one, move that helps to find food. Number
two, move towards green things because green–because food is the only green things. Well, solely
green things and that internal getaway–turn toward the blue things because they want to
mate with you. And get away from red things because they want to kill you. So, here’s
them wanting to get away from red things. So, we see a red thing coming up here and
it’s going to run away form it. And, run away. So, this is very nice. So, they–they–and
this came out completely naturally. No–no–no–no supervision at all. Just–just–just playing
do as the creator and letting it go. So, here’s some more. So here are some foraging patterns.
So, usually they–they–they like to kind of act out on their own, become a lone forager.
But some of them they swarm, so you’ll find like a whole bunch of very weak critters and
they mostly just go in–just go in circles all the time. And they–and so, like they
say hey, like, say there will be dark greens, okay, I want to turn–I want to follow dark
green things and I want to turn in circles a lot. And if you do that, the swarm just
sort of gradually moves, because the ones that are near food, they live. And so the
swarm just kind of gradually moves towards the direction where food is. And that’s–and
that works. Slowly but it does work. Okay, well that’s–let’s see, [INDISTINCT] this
one. Oh, this is kind of fun, you can see–actually you can–can’t see them engaging in a purposeful
behavior. Like you saw at the very begin of the stimulation, they all just kind of sat
there. We just [INDISTINCT] no, they’re actually moving around, actually turning towards green
things, actually displaying kind of you know, pseudo purposeful behavior. So, that’s a–steps
in the right direction. All right, so here is what we’ve seen so far. First of all they
make a lot of different kinds of brains. They’re actually–they are using their eyes for something,
that’s good. And they’re actually doing useful things with them, also good. So, all right.
So now I’ll show you some–show you some more science-y things we’ve tried to look it–we’ve
tried to analyze the behavior to determine if we’re actually getting anywhere and trying
to quantify it. So this is a nice one from the animal foraging literature. So this is
actually pretty straight forward. This is what you do, you have a world. You have a
food patch on one end and a food patch on the other. And you say “Okay, well how are
the critters going to allocate themselves?” So the very beginning they kind of uniformly
dispersed, middle some like “Oh, like you know some hang out in here, some hang out
in there, some in no man’s land. And then the late they go “Oh being in no man’s land
is bad” I don’t want to go there, so they hang out in the two food patches. So, so,
so this–they’re foraging that’s good and they are doing it correctly. And even better
if you actually look at–they actually form their optimal foraging pattern. So there’s
this distribution you commonly see in the foraging literature called the ideal free
distribution and lo and behold, they hit it perfectly. So, all right, good for the critters
in optimal foraging. So now I’ll show you some Predator-Prey Cycles, these are kind
of neat. So the colors don’t come out that gray but it will be okay. So in this case
we’re looking at predator-Prey Cycles between the critters and the food. So in this case
the red is the critters. This is for a particular food patch, the ones you saw before. So the
red is numbers of critters in that food–is the percent of critters in that food patch.
And the green is the percent of food in that food patch. So in short, what you see, let’s
pick I’ll say this one here okay. There you see that the–that the critters lag the food.
So first the food grew up high and then shortly afterward the critter said “Oh I want to go
in this food patch” and then they over harvested and the food goes down. And the critters leave
and go to the other food patch. And then the food was back up again and moved up and go
back to the food patch. And this oscillates forever. Yes?
>>[INDISTINCT] distribution there is no food growing in the middle…
>>GRIFFITH: Right.>>Does the food in this graph strictly other
critters?>>GRIFFITH: The food in this patch? No, no
and this–this case this was two food patches close to each other and they would just go
back and forth between the two food patches is what they would do. And depending on where
the food–where more food was at that time. And they would oscillate always following
the food. So, yeah. And this is nice because this is–this is a very similar pattern to
what we see in like–in Predator-Prey cycles. You know the standard [INDISTINCT] thing so,
also nice. And this is again like we didn’t program any of this. Like we just simply designed
a simple world with food and neural nets and said go. And we get all this–it just comes
right out. So, okay, so now we look at the brains cause that’s what we’re really concerned
about. So the main thing to keep in mind here is really kind of the connection matrix. This
other stuff here being a scientist, like that. So anyway, this is a random brain from the
very beginning–at the very beginning of evolution. All things are randomly wired together. And
so there–there’s one connection matrix. And this is one from the vision cortex of the
cat. Now and it should be random slides of it. And actually one from a Polyworld critter
after evolution. Ta-daa! Now let me take away from this. It’s not a cat but it’s certainly
not random. And so they seem that evolution has gone from this to this with doing nothing
but just sitting there and letting see a few cycles turn on it. So, again I’m not claiming
the poly organisms are cats but I am saying that evolution is doing something very useful
and it’s putting tons of structure in there that you do not put in so all right. And this
is kind of inspiring and you would go wow and maybe we actually could get a cat with
this. So here we go. So now I’m going to show you some more quantitative plot, more than
just looking at pictures. Oh sorry, so I always get this question a lot from philosophers
in the room. They always say, “Oh is not alive” well okay, fortunately there’s [INDISTINCT]
that a really good definition of life. It’s the Farmer Belin–the artificial revolution,
published from the Santa Fe Institute. And basically it says it has these measured criteria
to determine if something is alive. And not so coincidentally Polyworld explicitly designed
to satisfy all these criterions. So in short yet kind of space-time, it does reproduce,
it does have creature storage, it does eat and it has interactive environment and it
does evolve. So in short, to that–well it fits the definition of life that most people
used. So in your face. Okay. So then you will you say, I’m not sure if it’s intelligent.
Well it’s a–sure?>>[INDISTINCT] it certainly has metabolism
and it has functional interaction.>>GRIFFITH: Right.
>>[INDISTINCT]>>GRIFFITH: Yeah. No, no, no, I’m saying
here is quite satisfies all these.>>[INDISTINCT]
>>GRIFFITH: It doesn’t have information storage, it doesn’t have that.
>>Well if you have a coal left over from a fire you can initiate another fire. Would–is
that information?>>GRIFFITH: I suspect–I mean, I don’t really
care if fire’s alive or not. Fire probably can satisfy three or four of these. I mean,
I’m not really attached–I’m indifferent to fire. But I suspect if you look at the kind
of structure of coal or something. You probably wouldn’t find–it might be I wouldn’t have
much information there. I’m not sure exactly how you’d like at it, I’m sure it’s something
you could do but even if fire is alive, okay sure why not. Okay Belin would say, well it
is it really intelligent cause we just see them just moving around. Well there’s no real
way to quantify intelligence unfortunately. And I even [INDISTINCT] can do this. But however
we see this on simulation means we have access to a lot more things that biologists don’t.
And sure we can use information theory and complexity theory to try and analyze the critters
behaviors and their brains. And this is most of our research right now. So yes we analyze
their brains over time. So, so here’s a nice one so there are like three or four measures
of Neural complexity out there. And so far I’ve implemented two of them and the critters
all kind of follow this pattern. Oh sorry for this kind of complexity this is the [INDISTINCT]
complexity. I’ll get you the paper on it. In short this metric of Neuro complexity and
Schwartz says, if all neurons fire independently that’s not complex. And so yeah and if they
all fire in unison, that’s not complex either. So in short you want this kind of middle ground
between everything behaving randomly and everything behaving uniformly that’s what Neuro complexity
is. But in short, if you look at any of these they encourage all on. They go up for a little
bit and then they kind of plateau. And they’re like “hmm” And both metrics do that. So well
that’s what I got. And right now we’re trying to figure out how to make that go up more
and try to explain why it plateaus. So I’m changing some other stuff now. So now that
we know that neuro complexity does indeed go up. We want to know if evolution is actually
helping this–helping the complexity go up or if it was just kind of going up accidentally.
So there are two kinds of views of the evolution of complexity. The first one is this one,
this is a more natural one. And it says that “Hey, you know evolution actually favors more
complex from bacteria.” You know just big bacteria and then eventually to us. And evolution
really wants that. And the other one kind of says, you know what evolution really doesn’t
give a crap about complexity. Some things just kind of increase by accident on complexity
and some doesn’t really care. And the idea of this one is that if this is just mirrored
if this diffuses outwards. You know on the spectrum of complexity you know just doesn’t
care about it at all, you know you will eventually get complex things and it’s ready to start
with this and you could get to that. And so this is evolution actually favoring complexity
versus evolution not giving a rip. This is actually a debated question and we can use
part one to answer this.>>[INDISTINCT]
>>GRIFFITH: Right.>>[INDISTINCT] simple environment.
>>GRIFFITH: Yes I do. The question is whether or not the complexity of organisms is predominantly
a product of their environment. And the reason that we’re not seeing a big increase in complexity
is because the environment is so simple. And I think that’s exactly it. So–and–so what
we are looking at that now for ways we can make the environment more complicated to encourage
more interactions and things like that. But that’s about four or five slides from now
so we’ll get to it. This is the two ones this kind of experiment. Here’s what you see so
basically [INDISTINCT] polyworld to make all matings random. So in short even if you mate
with someone, you don’t actually get their gene. You get some random persons genes. It’s
sneaky so–and this is the dash line. This is where evolution turned off and oh sorry.
This is complexity here, and this is time and the dark line here is with evolution on.
Now this is very depressing, because you’re like oh well with evolution turned off you
get a higher complexity. You’re like, well you’re doing nothing. And I was very sad when
I first saw this graph. But I always look at this thing here. This always appears like
I’ve run this thing–I don’t know, I believe it’s ten times now. In short, there’s always
this hump here and I’m sorry and this is also a T test right here will get that in a second.
But in short–the idea I came out with is that there’s always this hump here and this–and
the solution that I came was, well evolution does fairly increase in complexity but only
up to a point. After you solve the world, we don’t care if you’re complicated anymore.
In fact it actually costs you something to be complicated. And so as to the result we’re
going to keep you roughly right there. While the diffusive one just kind of goes up on
its own. It’s completely–it doesn’t give out complexity at all. And it continues to
go on up. Sure.>>Yeah. Isn’t this [INDISTINCT] evolution
just where the fitness function is how long will you survive instead of how much you made
because if you randomly select a creature, creatures who live a long time are going to
be around more to get selected at random? So, if you just survive a long time and you’re
alive when other people are mating then your genes will get passed on more?
>>Let–let me think about this.>>[INDISTINCT] what did you do to select
this–selection–the selection of the random genes from all the creatures who are alive
at that time, that’s my question.>>Yeah. I’m thinking, how was it done? I
think it was–I think it was from all the critters who were alive at that time. So,
the idea was–no, I’m sorry. No, actually–no, this is [INDISTINCT] that a very good–that’s
very good question but that was controlled for. So, in short, I’ll–well, I’ll give the
more of the detail. Basically, this was that we ran this black line first and then, we
said, okay, you know, I–and then we–then we, and then we said okay like critter–critter
one lived exactly as many times of creature two, live exactly this number of time steps.
So, we did random–so, we did random mating combined with enforcing that–that each creature
lived exactly the same amount of time. So, but–but good question, clever.
>>[INDISTINCT] for several thousand years sort of pruning out the dead code?
>>Sorry, what does that mean, I don’t quite understand.
>>The complexity goes down because some of it is discovered to be unnecessary?
>>Yes. I think–yes, correct. And that basically fits with my–with my current belief. I’m
not exactly sure–sure why it plateaus and why it gets–why it stays there while the
past has go up. But I think–I think it’s pretty reasonable. So the idea is that, I
mean, because you always see that like in the complexity is useful at the beginning
but you want to be more complex than your environment makes you be so–so, the idea
is that we don’t make that more complicated and we’ll see that if it goes up more. But
yeah that’s–I agree exactly. If you want to see this here, this is a T test Pleistocene
to what extent based on the degree of confidence to which the dash line and the solid line
are thought to come from the same population and they say, if it’s above this critical
here, which basically says, “Yes, we’re pretty sure that humans have different populations.”
So, we see that–okay, right here, we’re sure they came from different populations now,
but actually [INDISTINCT] kind of crosses about right here. It just–it just–it just
kind of–it mostly kind of sits there. So, there’s a–so there’s some math to make us
think that as well. Okay. That’s just what I got. So, now it’s a Neural complexity–another
one for genetic complexity and this came from my professor at Caltech, Professor Adame and
it’s really nice that you correlate math over quite well. So, it [INDISTINCT] complexity
of the genes. [INDISTINCT] actually was it–it was 7,000 when they cross before ? Yeah, about
7,000. Okay, how about this one? 7,000 we see is roughly similar. Okay. So, the way
this one works the dash lines again are the passive runs and the solid lines are the–are
the–are the–with the evolution turned on. And so, in–we basically, see that on the
passive runs the genetic–genetic complexity basically went down to crap while on the–on
the active ronds the [INDISTINCT] did not go to crap, and in fact it stays quite high.
So, roughly what this says–roughly what this–what this measures look for, it looks like the
amount are not of disorder in the genome so, basically, if every gene was equally probable
or–sorry, if every gene is equally present in the population then–then it goes to here.
But if there some genes that are more favored than others then–then, I get this measure
gets higher. I can see the equation for it but that’s roughly how it goes, roughly it
measures the amount of disorder in the population of genes and roughly this says, okay with
evolution turned on, there’s less disorder in the genes, so. That’s good and nice. It’s
also can be that we see, the genetic complexity and the Neural complexity being roughly correlated,
yes?>>[INDISTINCT] when you say evolution is
off, your [INDISTINCT] turned off the sharing of genetic information for mating.
>>Yeah.>>[INDISTINCT] for mating, where do you [INDISTINCT]
>>Okay. When I say evolution is off, I say that the matings are random. And–yeah, I
just say, the matings are random and critters are forced to live the same amount of time.
So, the idea–so, there’s controls and the matings are random and so.
>>[INDISTINCT] made the results is one, it is in fact [INDISTINCT] Okay. Whenever evolution
is off–when evolution is on, when two creatures make, there genes get match together and they
make a child, so, completely normal. When evolution is off, when two creatures mate,
it takes a completely–it takes a two random genes from things currently alive so, and
then, it pops out that child.>>[INDISTINCT] made a copy of one of the
parents or something. I don’t understand the motivation for getting a random gene from
some other creature.>>I’d like to think…
>>[INDISTINCT] main copy [INDISTINCT]>>If it’s completely random, its random–I
mean I–I think–I know I have to–there you may be able to do this if you just make–make
a copy of one of the–of the parents. You may be able to–I’ll have think about it that’s
why that one would work two but-but I know if–if that–if–if very creature is equally
favored, no matter what its genes are, evolution doesn’t move like that–that’s the rule. Like–like–like
that–that has–that has selection with everything being equal–equally selected for. So that–that’s
what motivated it. Sure?>>Random selection on the [INDISTINCT] any
have plan of population? You will have a genetic group, is that right?
>>I think you should see here. This–this up and down genetic drift due to decline.
>>I mean sometimes some ideals will be lost in the population just because of they.
>>Right.>>Regular see. There would be–there will
be this pair of mixing of possibilities but its slowly go to a fixed point, right?
>>Um.>>Do you–do you see this?
>>Well, you–you certainly all right. Like I mean because of funny population you–you
will see variations in the pop–in the population. And I think its–is what you’re seeing here.
So in this case like this is two has completely random mating and it’s moving up and down
a little bit. And I–and this is–this is due to drift but as you increase population
size this gets less and less and less as exactly as you’d expect. So–so yes, you’re right.
And–and in we’re seeing it. So it’s good. Okay.
>>[INDISTINCT]>>Oh, okay we have to quick to them. Al l
right, so it’s the next time do really quick and to pass through this. So there’s a real
question of, so for this passive complexity it could be just be this passing complete–like
why is this leveling out at all? So it could be that–that sort of–sort of upper bound
in simulation because simulation cant support something–something of higher neural complexity
plus we’ll–so we journey with Polyworld to say, okay we will sole–we put through a fitness
function mode. There’s no longer natural selection of. We were working solely for having a complex
brain and that that’s the red one here. So in short this says, hey, you know the simulation
can support much higher complexity if you–if you like really forced it do it. So, in short
this phase says, hey there–there’s room to grow for–for evolution. So, all right. So
basically we have so the next pencil be making more complex environment and trying to move–move
this curves closer up to the red. So, okay its making a draw from there. So, these are
the few directions will take Polyworld into but predominantly making the world more–more
complex and then come in with more measures of complexity for studying it. So in short
more exercise in complexity there’s–there’s still like–there’s still four or five that
we haven’t looked into yet, more complex environment. So the first thing at right now I want to
add all like day and night cycles. So–so in this is really easy to do because it’s
all an open GL and you could just tweak the ambient lighting up and down. And the idea
is that these would force them to–to have a sort of an internal clock, saying, hey it’s
dark now. I–I can’t see anything probably shouldn’t go foraging. Notice having different
kind of food types. So you could have different colors of food and–and one will give you
more energy that the other. So it’s kind of having specialization. And the others giving
them more–more senses, right now they only see and if you give like smell or touch it
is they could have more interaction with the environment and that would be good. Yeah,
so were done the actual forging we did that by recently. Yeah, and we held this to answer
question about evolutionary theory as we did. Answer more questions of evolutionary theory
like we did before. And did eventually we can skip up to casual–casual conditioning
experiments. So this is kind of like–like the direction you want to go for the next
few years. And I think you have ideas especially for here. Let me know or you want to get to
decode. So this is mostly it. The source codes available, you can get it now. It runs on
Linux and Mac via Qt, its just works. And then we can download it. Yeah–and at the
very end I always get the questions, oh, you’re making Frankenstein this is a terrible idea.
And I–I was like this snide respond to them. So and yeah, I have no problem with that responsibility.
It’s a–its–its–if the polygon kill us all well–well it happens. Okay and I’m done.
>>Questions.>>So just an idea about directions for–to
test theories and evolution. Have you thought of a sex selection to see if their specialization
have been given very little or a lot of contribution to the offspring, as if their two initials,
two genders developed?>>Well, currently there’s no gender. You
could certainly do it. My–right now there is no gender right now it could be the one
cut-cut the population like the mating pool in half. So like right now, these critters
currently run with about 300 agents in a simulation, I’m sorry, the answer is yes you could do
that. That’d be really cool but right now we don’t do it because we are concerned about,
it might be hard to find a mate.>>But I mean, I’m pretty ignorant to this,
under some theories would say that the origin of the division of genders is that there was
a specialization to niches, the males contribute very little, they tried to mate a lot, the
females contribute a lot more and so maybe you can look for, you see if this two niches
develop, even in the absence of explicit gender, I don’t know, that was just an idea.
>>GRIFFITH: You could certainly–it’s certainly possible like if you had two different kinds
of behaviors, and one was favorable one time and the other was favorable the other times,
you could get that to come out naturally, but when they can always mate sort of all
the time, it’s going to be tricky for that two not to be enforced over the long term,
but yeah, it’s certainly possible and if you wanted to do gender differences, it’s actually
really neat, I mean if you start enforcing it see if they were starting to use each other,
things like that, so that would be cool.>>This networks, at least, as I took it to
mean, don’t have any state on our cursive networks?
>>GRIFFITH: Ah, no, they are recurrent networks so they can connect back if they want, we
actually have a new kind, this recur to something like squashing neurons, we have a brand new
model that has spiking neurons, I don’t know much about it yet, but I haven’t use it much
yet but we do have more fancier models.>>And do you save state in between cycles?
>>GRIFFITH: Well see, no we don’t save state between cycles, but we do update their vision.
>>Right, right, that seems to me to be necessary in order to maintain a mental model of where
you are in the world, as opposed to just a single state, here I am, what am I going to
do, it seems like that’s uh…>>GRIFFITH: That’s a fundamental part, yeah,
let’s see, I don’t think we’re saving–we’re saving the state of the network from [INDISTINCT]
to the next, like of the internal nodes. I’d have to think, well I can answer the question
empirically, and like 10 minutes I went to the code, so I’ll answer it a little bit.
>>I think this is a really good, interesting presentation but I guess I have a little difficulty
because I’m not that familiar with the area to have some context for it, could you say
just a few words about sugar world and Tierra and Neuro Darwinisms so I have some sense
on how this ..>>GRIFFITH: Oh, yeah, I’ve heard of sugar
world, but I haven’t–I have never, I’ve heard of sugar world, I know that–I’m sorry, I
should back-up, so there are previous simulations, Tom Ray’s Tierra was basically–was the first
thing of evolving code, and it was really awesome, but there were a few problems with
it is that they see things always got smaller and smaller and smaller, so that was kind
of a problem in Tom Ray’s Tierra, so like it always became better, if you’re genome
got smaller because that way you can reproduce faster because they were penalized. They only
get a certain number of cycles to reproduce themselves and if you’re very small you reproduce
yourself a lot. I don’t actually know if, as far as I know Tierra has not been extended
to account for this original defects, but certainly Tierra is like really great, as
far as sugar scape, I’ve heard of it, I don’t know much about it, so, but if you send me
a paper on it, I’ll certainly read it, and I can go and come and tell you then. Sorry,
what was the other–oh, neuro Darwinism, okay, so neuro Darwinism is a theory of neuro science.
It’s probably even true, in short it says that the way connections are formed in the
brain, is kind of like evolution , it’s not exactly, but roughly it says that neurons
initially kept to a whole bunch of things and most of them suck and the ones that sucked,
get pruned and they go away. So, roughly neuro Darwinism is like expand, prune, expand, prune.
And it says this is how connectivity in the brain comes about, and it’s probably true.
>>So on your final slide, I think it was the final slide, you said that one of your
goal is to make the environment more complex?>>GRIFFITH: Yes.
>>And experiment with more features [INDISTINCT] and so I think it’s, maybe a little bit of
problem because your current system is already very complex and the thousand thing that affect
the way evolution goes in your kind of system and you know how you construct their production
procedure and so on and so on, so you’re not afraid that if you make the environment more
complex, you will be, possibly you will be able to see very fancy simulations but, it
maybe more difficult to understand why actually evolution went this part, not the other way.
>>GRIFFITH: Your bachelors isn’t in physics by any chance?
>>I’m sorry?>>GRIFFITH: Your bachelors isn’t in physics
by any chance? I mean, physicists always say that. So I’m wondering what your background
is.>>No, no. my background is actually, I solve
evolutionary computation for my…>>GRIFFITH: Oh, okay, all right. Well, yes,
okay. Well–the concern is roughly, well if you make it more complex, you always have
parameter help. You already have parameter help but it could be even worse. Like ninth
layer parameter help. And the answer is yeas. That–that can happen. And I guess the response
is, well, it seems like a lot of these things don’t depend on the parameter very sensitively.
So be like very, a bunch of parameters we have right now, you roughly see a lot of the
same stuff. And the hope is if you choose this even remotely reasonable values, the
good stuff will come out. And so, the point is valid. But we don’t think–but we think
like the benefit of having a more complex world far exceeds the concern of parameter
help.>>I’ve been thinking about this for a while,
I mean, you showed it to me earlier today, but I’ve also been thinking about this general
problem, and I think that we can state without being too contentious that there are better
strategies in the worlds that you’re presenting. Like if we’re really careful and designed
one, we could probably clean the clock of a number of these evolved systems. And I think
part of that’s going to be not a product of the structures of the brains but the kind
of input that they have available to them when they drive their behavior. Put another
way, I don’t think you should be adding complexity to your simulated world in terms of adding
lighting effects or for or the things like that. I think there need to be more signals
that have to do with kin selection and not–not just green. Like, in the natural world, even
at the very cellular level, you–just is a natural by-product of the way evolution’s
is going to effect what kind of presentation you throw up on your cell walls. Like you
can do kin selection in the environment pretty easily, like that’s assumed. And so you can–a
lot of the complexity we see in natural systems and how central systems are and how predation
systems interact, seem to be driven by really complicated gradients that end up working
out down the kin similarity. Like, I don’t want to mate with someone who’s exactly like
me and I don’t want to mate with someone who’s really, really different from me either because,
if I mate with someone who’s exactly like me, it’s not worth the energy because there’s
not going to be much variation. If I mate with someone who’s too different, the child’s
not going to be viable. And like, the complexity in your environment should flow out of the
behavior of the features that you’re competing with. And you should see speciation resulting
from preferences. And alternate patterns in like–that it doesn’t seem like there’s enough
input for the neural networks that you’re evolving which seem to be really cool to exploit
that gradient. So I think maybe finding some way to allow them to sense the presence–and
go ahead and cheat. You know, like look aside–do similarity scores and provide like, a sense
that–similarity sense. You know, not based on light at all I mean, you’re looking directly
at the genes, because in any natural evolving system, you’d end up having pheromones and
various other markers that you would learn to exploit. But they don’t really have that.
All they have is what they present directly and it would take a very, very long time for
that to evolve.>>GRIFFITH: I think I get your–so your point
seems to be roughly that the critter should have more complex interactions with each other
rather than more complex interactions with the environment.
>>Well, not even necessarily more–I mean, the actions that they can take are fine. I
just don’t think that they can observe the other critters well enough.
>>GRIFFITH: Okay, yeah. Well, and so I guess the answer is “I agree.” And if someone’s
to write it. If you’re a writer, I would gladly put the patch in. So, now as to whether that
would be–that–as to whether or not more complexity between–more complexity between
critters would be more valuable than interaction with the environment, I guess you could try
it and find out. I mean, I think those would be great. So, yeah, so there’s no contention.
Okay.>>All right, let’s take one more question
in Mountain View and then we’ll let the video tapers go, unless there’s a remote office
that had a question that I wasn’t fair to.>>GRIFFITH: Yes, so as a biological creature
myself, I kind of hope that death is not inevitable and I was curious of what you were noticing
in your simulations if you had turned off the limited lifespan of a creature.
>>Oh, let’s see. I guess you could just clamp it. You could do that. I don’t know. The reason
I did that is just because, like, I saw a paper at a conference that just had these
mating populations. And it said that–that having a fixed lifespan or at least a max
lifespan was a good thing. So I said, “Oh, well, just put it in a gene, done.” So I have
no–I’ve never actually clamped it and compare the differences. But you can certainly do
it. I mean it’s just a parameter. So…>>GRIFFITH: So what I’ve been thinking is
that if you didn’t have a limited lifespan, what would the results of your simulations
be? That’s what I’m curious about.>>Well, most critters don’t get to their
max lifespan. Most of them die of energy. So in this case, like, I think, like, the
average critter lifespan is something like 300, 400 time steps and the maximum lifespan
is something like 700, 800, something like that. So most, so very, very few get killed
by that. So, I guess I don’t think the maximum lifespan has much impact on it. And I just
put it in there because I saw a paper that said this was good. So, and it was–and I
was writing that piece of code at that time, so.
>>GRIFFITH: Okay. We’ll still be around after the talk is over if anyone wants to chat more.
>>Okay.>>GRIFFITH: Thank you.

100 thoughts on “Polyworld: Using Evolution to Design Artificial Intelligence

  1. It has been 4 years. 2011.
    What are the most impressive achievements of this brilliant program?
    is there a new software?

  2. If it is possible to re program Polyworld with gpu cycles instead of cpu cycles the evolution can go a lot faster because gpu are a lot faster in linear processing than cpu are. End when you look at rainbow calculations a gpu is really 200 times faster than a cpu: a passport like "23P#d3" can be hacked in 7 minutes with gpu processing and a cpu needs 23 hours to do the same linear calculation. I hope someone who can code gpu programs wants to look at it. Greets Pieter Schreurs

  3. This might sound pseudo scientific but I would still like to get the point across and it might be more generalistic than just AI. The critters are information feeding on information to producte other pieces of information. It is a copy paste of the Big Bang (intial information) and the after effects. Not following on those lines can there be a system where there is no competition but co-existence if the initial information (you) is manipulated? Play God and create a competition less environment.

  4. Since the digital world presented here is information being fed over and over again, let's assume the physical world too is the same for a higher intelligent programmer. Now the two systems follow the same Darwinian principle. But can there be an exactly opposite system? Neural network not according to our understanding of biology but assuming we are elements too of a larger organism unlike the parallel reality concept. It'll b hard to define what has life and what doesn't. Ths may be pseudo sc.

  5. We need to push this research as far as possible…we'll never have real AI I'll predict, but some useful things will come of it…calling these thing "alive" because they follow an inadequate definition of life is not being very honest, they are cool toys though…that's all i got.

  6. i think defenition of "alive" is being self aware. knowing that you are you. i know that doesnt really apply to plants but if anything artificial was to be considered alive it would have to be selfaware.

  7. : : : : THIS IS PRE-TERMINATOR ALGORITHMS!!! ————— mEXACTLY what I said, Check my Videos out on LIFE's Challenges

  8. I LOVE YOU GUYS!!! I made a Reply video, I am AFRAID of the TERMINATOR idea coming into affect DUE to Error.. in short, viruses extort these errors and can over produce and completely wipe us out if WE DO NOT add a LIMITER (music terms) to the overall limits of these mechanics we make…. I made a video in Response, but you guys aren't allow replies. so here it is



    Google :

    youtube theealdeen Re: Polyworld

  9. People interested in AI are welcome to apply to our new AI project called Virtual Biosphere.

    Check us out on Google Groups. The groups is still very new (small, but growing).

  10. True. The internet has turned too many people into wanna be critics. I think that's why YouTube got rid of the star system in favor of thumbs up thumbs down. People love giving 1 star for some reason even when it's not deserved. Like if he messed up one thing or didn't address one thing they wanted, 1 star. Harsh.

  11. Looking through the comments it becomes immediately clear to me that the commenters here doesn't know about aspergers or autism.

    I believe the reason he is jittery and seem so stressed is because that's just how he is in his head. When he is explaining something his brain is going in overdrive so to speak and every time he says "so" it's to summarize in his head, giving a step by step breakdown of something the only way he knows how.
    The time is too short and he is too smart to explain it well.

  12. Now there are a number of other reasons that leads me to believe he has Aspergers or something along the autistic spectrum and that is the constant jumping back and forth in the topic, always fiddling with his hand or items in his hands. Clapping his hands together when he shows the first couple of slides etc.

    I have a friend that has been diagnosed with Aspergers since childhood and he does the exact same things. He also does drum solos all over his body when he gets a small "break".

  13. The question of the complexity of the environment limiting the development was interesting as was that point that a designer could implement a constructed agent that could easily out compete the evolved entities. Perhaps the element that should increase environmental complexity should be a constructed agent that scales forcing evolution to keep up with it. In essence that is the true goal, not to just evolve a moderately efficient system but to evolve a system that could not be designed.

  14. The answer to the parallelism question he botches may saying "we can help them along" is actually to get this thing running on multiple computers. (15:20)

    I'm sure that if someone wrote a game, with nice graphics, cute fighting critters etc. that evolved on the computer (i.e. Spore done right) which enabled people to evolve things on their computer, then upload them to servers which enabled them to compete the "wider world" that this would actually have quite a few people playing with it.

  15. How can anyone not see this video is a spoof after the announcer pronounces "polyworld" like Will Ferrell is doing a Bush junior impression?

  16. I imagine this kid saying (As Daniel Tosh once said):

    “Because I'm not in the 7th grade and I've got things to do."

  17. I tried to skip to the part where they were explaining the neural part. *skip a few seconds* narrator: uh… *skip a few seconds* narrator: uh… *skip a few seconds* narrator: uh…


  18. 0:13:22 "Evolution stuck with nervous systems on all levels of complexity"
    Well, not really, microbes, fungus and plants don't got nervous systems…

  19. Larry Yaeger should be concerned whether fire satisfies the criteria of life, or not. Since fire obviously is not life, then his definition needs revision if it cannot refute the assertion that fire is life.

    In defense of Yaeger, I don't believe fire has the ability to evolve. Not in a Darwinian sense at least.

  20. Pretty simple stuff really. Been done years ago. I think it's unfortunate for the field (of AI) that we are just going back to old things because no one can think of any new ideas. Also, on a larger level, there is no need to get evolution to design an intelligent system in the computer because we all already have one from millions of years of work. We just need to study our brains a bit more. The kid is not thinking on a stratospheric level over everyone else… he's just nervous, relax people.

  21. Turning to evolution to design an AI in a digital system is actually a pretty new idea. Studying the brain and attempting to redesign it by hand is not.

  22. Yes, if you allow interaction between agents, agents become a part of the environment (i.e, coevolution). My understanding of polyworld is that agents *are* allowed to interact.

  23. Well, I've seen numerous sample codes and projects that specifically "wire up" neural nets based on evolutionary environments since 15-20 years ago. Just saying from experience.

  24. Having reviewed the literature, neuroevolution was born in the early 1990's for simple optimization problems like pole balancing. Karl Sims made a huge breakthrough with his work in 1994, but his experiments didn't allow for much coevolution between agents (other than simple 1v1 competitions). Polyworld extended this by allowing coevolution between multiple agents, which is huge for evolution of species diversity (see Cambrian explosion). Sadly, seems Polyworld kinda died out for now.

  25. This is a common counter-argument against evolutionary computation in general. But even non-evolutionary techniques (like Deep Learning) are having to turn to high-performance computing centers to get any useful results. Fact of the matter is: it took billions of years of evolution to create us, so it's going to take a correspondingly large number of computing hours to create us again in the computer. Ultimately, how long it takes to create AI doesn't matter; what DOES matter is if makes an AI.

  26. a little confused about the experiment where he turns off evolution – aren't the genes that are randomly used still a result of selection? That is, you can only randomly select from creatures still alive – thus selected. There may be a scale difference in complexity but it's still evolution. Perhaps simply cloning with no cross-over or mutations is evolutionless?

  27. did he say that? perhaps he meant once a nervous system evolved? But i guess how would he prove that… still interesting stuff…

  28. these things should have family trees. some traits skip a generation. some things are too closely related to breed. sometimes a defect can change up the game. also family probably cultivates familiarity. if the blinking function had a code that could be inherited and particular families blinked certain codes, like birds sing certain songs, then they might start to use that thing. I wouldn't only take the genes of the parents. I wish there was a family tree aspect to it. probably too late >.>

  29. The failure of these models will itself be the key to intelligence and the death of the concept of artificiality.

  30. shit it's after the simulation… i love this idea.  what would it be to devise an animilistic world?  rabbits eat grass, deer, stags, bears and breeding and such.  no chance anyone would be willing to work with me using the unity… can be debated… i prefer open-source

  31. My question would be:  At what point can you make them viable as characters in a game?  will they eventually answer my mail?

  32. I think what you are doing is amazing…I did not expect the cannabalizm of children.   But it is logical Just strange to see.

  33. the real world is pretty complex. why not just make that its environment? robots are getting pretty cheap these days. they couldnt reproduce physical traits. but parents could reproduce and pass on their brain structure to robots that "died".

  34. I would like to say this is a great video but i couldn't understand anything with this speaker presenting it… Not to bag on him or anything, but he's quite horrible and confusing.

  35. When is anyone gonna try to experiment to see if a neural net can predict things before they happen. (predict randomness i guess) So they get rewarded when they get a prediction right and don't when they get it wrong

  36. If somebody is interested in testing out a neuro-evolutionary algorithm: I created a game where you can train your own drivers (their neural networks) by a genetic algorithm, it is still in beta, but you can have a look at http://www.kongregate.com/games/sem23/neurodriver to play the game.

  37. Hello everybody, Could anyone pleasee explain to me how the connection matrix ( 38:30 ) works? I have a simple program with similar neural networks how do i build such a connection matrix for them

  38. Evolution does favor complexity, but I'm not sure the system described could actually see this. If they made it so that creatures produced waste, it would go a long way towards helping. The drive towards complexity naturally is due to the fact that every living thing makes changes to its environment. For instance, with bacteria, they metabolize something and turn it into a waste product. So even if their food source regenerates, the buildup of waste creates a pressure. It creates a situation that greatly favors the survival of an organism which could thrive both in the world the original bacteria thrives in AND in the waste. Even development of a symbiotic organism which could survive solely from the waste, which might be seen as equivalent in complexity since it's just surviving on something else, would be inferior to that more complex organism because its survival would be restricted by its food source – the original bacteria.

  39. None of you will listen because you're all too drugged up on your "genius" but you are going to kill us all by building skynet. Please stop it.

  40. Watching this while letting my virtual creatures evolve 🙂 They are way more simple though. Great talk! This is just perfect!

  41. "Um" "some" "then" "yeah" "so" "and" "but" "it's" "you know" "well" "like" "alright" "uh"

    This person is the WORST public speaker I've ever heard. It's like his brain is skipping, he'll start saying something and then he'll backpedal to before he said the word and repeat it or change what he's saying.
    He's constantly doing this "tut" thing that's like lip smacking, really annoying.

  42. Great to watch – thank you.

    Firstly is there a modern update to this anywhere with updates ? I.e. Where we are now.

    I know this is a very old video so my comments won't really be relevant – but my ideas for development

    * make maximum life span dependent on size , as well as energy storage and maximum brain development , although have a longer gestation period period for spawning – if it's bigger it should live longer, have more capacity for neural links. This would hopefully generate more complex 'critters'.
    * being ten years on the sandbox should be able to be a lot bigger with the option to run two or more separate species – evolution may then create a carnivore species and less likely a cannibal.
    *the only real update to the environment I think that's needed is a day / night cycle – promoting possible dormant night time evolution with food only being spawned during the day. Using this a dark permanent 'cave' environment could be created to see if a subspecies(s) develop in or around that area . (Simple organisms in a half covered Petrie dish).

  43. I have built my own simulation with much more complex adaptive nueral networks, and I would like to visualize the brains the way he visualized them using the connection matrix at 38:10. an explanation on how this type of visualization works would be highly appriciated

  44. Speaking of Nervous Systems… It's way too difficult to understand what the presenter is saying, due to the stuttering and mumbling.

  45. Something I would love to see in one of these evolving critters projects people work on is this:

    Food that isn't just worth X amount of energy, but food that has X amount of Stuff A, Y amount of Stuff B, and Z amount of Stuff C… and so on. Then the creature would have organs don't don't require energy to function but require a specific Stuff to work right. The specific Stuff any given organ would need to work right would be based on it's genetics, and what the organs do would again be based on the genetics. Each creature would have also a max amount of stuff it could hold in it's body at any given time.

    So you could then start having organs that develop that do things like, using a bit of Stuff A as energy convert any Stuff B you have in your body into Stuff C. Or another organ that says, using a bit of Stuff C as energy expel from your body some of the Stuff C you are currently holding.

    Depending on how it was done you might get complex things emerging like: When I'm ready to breed start expelling Stuff A and B from my body to attract a mate, and look for Stuff C while mating because it just so happens that there is another type of creature that evolved along side me that when it's ready to mate expels C from its body when it is ready to mate.

    You might also have creatures that develop "toxins" as well. What I mean is that your world might have only Stuff A, B, and C in it naturally but this creature might develop an organ that converts a bit of stuff C into stuff D. Nothing has yet developed an organ though that can do anything with stuff D though, so when some random creature comes over and eats the creature that has built up a bit of stuff D… now this random creature has stuff D filling up it's stomach and has no organ to convert D into a useful Stuff and nor does it have an organ that can expel it, so this random creature effectively now has a smaller stomach for eating the foods it wants to eat.

  46. God, you could do so much with this system he has created. Another point I would make is this… If the virtual creatures discovered an exploit inside their environment, then who is to say that actually biology hasn't found exceptions to physics that we are not aware of? Such as ghosts? Could consciousness/biology even found a way to survive death of the body?

  47. 5:03 Compters are hard. Especially Macs.
    31:34 It took a lot to figure out that reproduction is naturally not for free and that individuals might misuse a system where they get free stuff? Like "Communism doesn't work" was hard to figure out? That's what you get when you hire a millenial, I guess.
    54:04 At least he's taking responsibility. So when the day comes and AI eradicates humanity: this guy will take the blame. Just sue him.

Leave a Reply

Your email address will not be published. Required fields are marked *

Back To Top