Yesterday I spent my night setting up MXNet for solving Space Invaders. Initially, I thought it would not take me too long as I have been previously configuring data providers for MXNet in Julia, but it resulted in the opposite.
Few updates before I move on to Space Invaders. I have updated my GitHub repo and updated DQN to support multiple layers and also managed to fix some bugs.
I am a very active Oslo Data Science meetup Hacking group member where we actively discuss Reinforcement Learning. Lately, we have been thinking about solving Space Invaders, and some of the members were struggling with implementing the solution. I decided to join their army and see if I can handle it in Julia.
I have started with reviewing my existing code and created an additional solver based on a standard DQN and made it dependent on a previous frame. So now to predict our next action, we will first find a difference between the last two frames.
My current score varies very much with an average of around 210 and maximum 630 points. Most probably I need to train the model a little bit longer so it can explore more of the field.
I am also planning to introduce a Replay Buffer, which should “help” the model to remember “old” frames and keep gradients alive.
I haven’t shared any code yet, and it is to come as soon as working solution is available.