I have been so busy writing my book on Julia that I could not dedicate time to finalizing Space Invaders.
Previously I have managed to train the model to reach 200, but after implementing a few changes, it dropped back to random policy and stalled on around 165.
Continue reading “SpaceInvader story continues”
Few updates before I move on to Space Invaders. I have updated my GitHub repo and updated DQN to support multiple layers and also managed to fix some bugs.
I am a very active Oslo Data Science meetup Hacking group member where we actively discuss Reinforcement Learning. Lately, we have been thinking about solving Space Invaders, and some of the members were struggling with implementing the solution. I decided to join their army and see if I can handle it in Julia.
I have started with reviewing my existing code and created an additional solver based on a standard DQN and made it dependent on a previous frame. So now to predict our next action, we will first find a difference between the last two frames.
My current score varies very much with an average of around 210 and maximum 630 points. Most probably I need to train the model a little bit longer so it can explore more of the field.
I am also planning to introduce a Replay Buffer, which should “help” the model to remember “old” frames and keep gradients alive.
I haven’t shared any code yet, and it is to come as soon as working solution is available.
Finally! I did it! I’ve been struggling for some time trying to make DQN work and could not succeed.
Today I have managed to make it work and solve CartPole from OpenAI gym using DQN. You know what the problem was? Size of my neural network!
Continue reading “Playing CartPole using Julia and MXNet implementation of DQN”