June 19th,2019
For the last month or so, I had been working on strengthening my understanding of all the concepts that power deep learning. I was getting frustrated by just using high-level libraries such as Tensorflow and not always being able to fully envision all the concepts that were powering these libraries. So, I went back to the drawing board to refresh and understand all the basic concepts in a better way than I already do.
I have been redoing all the concepts from scratch: forward propagation, cost functions, activation functions just to name a few. I had to bounce around a lot while doing this to get a full picture of what I was doing. It was here that I discovered the notes that were to serve as supplemental material for Stanford’s class on Convolutional Neural Networks (CS231n). The notes can be found here.
I have read these notes only once so far and I really liked the thoroughness of the material covered. I thought that these notes really helped me develop a better appreciation and understanding of backpropagation and the equations at play. I have been trying to develop my own analogy for backpropagation which doesn’t rely on hardcore math to convey its meaning and I think I have got something, though still half cooked. I will continue working on it.
The notes for CS231n are the best resource I have seen so far. It is a well arranged sequential approach to taking you from the basics of deep learning to CNN in a relatively short amount of time. The notes explain a lot of the things that are taken for granted like activation functions and the intuitive need for them. For me, it served best to fully understand backpropagation. I will be reading it at least a few more times to completely understand all the details being discussed here and develop my backpropagation analogy to fully visualize the process down to its details.
The lectures for CS231n are also available online here. I am currently making my way through them and will be using notes in concurrence with them. I am also reading Deep Learning in Python by Francis Chollet and Neural Networks and Deep Learning by Michael Nielsen.
deepjourney 