State space in games

This is a small detour from the L’oaf AI training write-up from a few days ago. I was thinking about if you could represent the game as a set of game-states in a larger, high-dimensional state space. Then we can see what the agent is actually doing in this state-space. I figured I wouldn't learn too much from this, except for getting some nice visuals. To make that map, I generated a big pile of real positions by simulating games between my PPO model and the distilled tree. For each decision point I stored the model’s internal representation of the state, plus metadata like whose outcome it ended up being and which turn it came from. Then I compressed the representation down to 3 dimensions with PCA so it became something I could look at in a 3D graph. On top of this, I showed two trajectories, one in which the AI wins and one in which it loses, to see how the games actually move through the state space. I put the 3D representation below so you can appreciate what it looks like.