AI Prototyping in Unity (Part 2)Sat Dec 23 2017
In my first AI-post I started prototyping and learning artificial intelligence and self learning neural networks in Unity, but the first tries were kind of disapponting. Let's try to make it better.
So, what we are missing. In my first tries our creature had only 3 inputs for it's body angle, but it doesn't have any idea of it's legs angles or it doesn't "feel" if it's body is on the ground, so we could make it aware of it's leg positions and something that makes it "feel" when it's hitting the ground.
To make our creature more aware of itself and environment we need to add lot more inputs and also update our fitness function to improve our scoring when our creature is doing something right.
What, I have legs?
So I added 3 (X, Y, Z) new inputs for each leg parts, so each leg has two parts and it makes 2x3x4=24 new inputs. As you can see from the picture, I created simple input components in Unity, so it makes it easy to add new inputs and we can easily collect all the inputs from the whole scene on start. As you can see, I have also flag there do I want input to return local or world angle. In my previous post I added 3 body angle inputs to our creature body in world coordinates, but for my legs I want to use local coords, since they are rotating with the body and I want our creature to be aware of the leg angle in relation to itself.
Add some feeling
To make our creature aware of the environment I added some raycast inputs and I also use the raycast data in my fitness function. As you can see, I added some raycasts in a different directions of body and also to the feet, so now our creature can sense when body or feet are hitting something and also sense when then are close to hit something. I could have used colliders for this, but at least for body it's good to have some idea which side of the body is hitting something and collider would also give us only true or false value, so with raycast we get little bit sense if we are moving further away or closer to some object.
Add sense of movement
I want also my creature to be aware of it's movement, so I added creature body velocity and angular velocity inputs, so now it can sense if it's moving or rotating and hopefully learn how to react on that.
Quick recap what we have now
Now we have added lots of new things, so let's see what we have.
|8||Body raycasts up|
|8||Body raycasts sides|
|4||Body raycasts down|
|3||Body Angular Velocity|
What about our fitness function now. In my code I collect all the different fitness-components which normalize their output between 0 - 1 and then in my fitness function I just compute average of all the different fitness scores, so fitness score will also be between 0 - 1.
|1||Body difference from zero angle|
|1||Body raycast up, 1 = no raycast hit, we return the minimum value for fitness|
|1||Body raycast sides, 1 = no raycast hit, we return the minimum value for fitness|
|1||Body raycast down, 1 = no raycast hit, we return the minimum value for fitness|
|4||Feet raycasts, 1 = when feet are hitting the ground|
|1||Body velocity difference from zero|
|1||Body angular velocity difference from zero|
Now that our creature knows much more of it's surroundings, velocities, leg angles it should be much better. I added some statistics also to Unity about the AI iterations, fitness and current score at the moment, so it's easier to follow what's happening. This is something I need to look at closer and add some more helpers for debugging the AI.
Is it better now?
This starts to look more like what I was trying to achieve. There's still a lot of room for improvement, but as my first task was to make it stand up, now it seems to end up standing in a quite steady position after practicing for a while. There's still many things that we can do to optimize the learning, but I guess this is pretty good start. One thing I noticed that Unity joints seem to be quite "soft", so sometimes it's legs seem to start to twist when it tries to stand up, even if my other axis are locked or limited. That's something that I need to take a look at, but this is the first step to get our creature do some more advanced tricks.
After watching this guy struggling for so many hours I'm really happy to see him succeed, I'm starting to like this guy.
To be continued...