Squadrick Whatever works

Neural Network Compiler (NNC)

This was the documentation for a project I completed as part of my compilers class. It’s a language to define neural networks, which is then optimized and compiled into efficient executable C++ code. The code of the compiler needs some refining, and will be released shortly. Also, note that the entire project was completed in around 48 hours, so... Read more

Efficient Multiple Gym Environments

In Reinforcement learning, most of the training time is spent in collecting experience from your environments(real life robots, simulators), and very little time is spent on the actual learning part on the GPU. So if we want fast RL training, we need to build fast environments. Due to its ubiquitous usage in RL and easy-to-use API, I’ll be stic... Read more

Natural Gradients in Tensorflow

So I recently started learning deep reinforcement learning, and decided to make an open source Deep RL framework called ReiLS. So I went ahead and implemented a couple of popular actor-critic methods like DDPG, A3C and the more recent PPO, and soon turned my attention to TRPO. The difficulty with TRPO is that it uses natural gradients, as oppo... Read more