Episode 3 - Training Deep Neural Networks on a GPU

Video: https://events.genndi.com/channel/DeepNeuralNetworks
Code: https://jovian.ai/aakashns/04-feedforward-nn

“Deep Learning with PyTorch: Online Workshop Series” is a collection of webinars covering various topics like the basics of Deep Learning, building neural networks with PyTorch, CNNs, RNNs, NLP, GANs. The workshop series consists of 5 episodes. This episode will focus on the topic “Deep Neural Networks & Training on GPUs”.

The workshop takes a hands-on coding-focused approach, and everything will be taught live using interactive Jupyter notebooks. Theoretical concepts will be explained in simple terms and using code.

Prerequisites:

  • Basic coding skills (preferably Python)
  • Basic understanding of linear algebra (matrices, derivatives, etc.)
  • Completed Beginner level workshop (conducted the week before)

Use this thread to ask questions.

Give this a like if you’re here.

7 Likes

Is this subsampler is equivalent to bootstrapping sampling?

How ReLU is working to give the required output?
Why are we eliminating negatives?

Relu is a just a activation function consider it as a filter which only lets positive values through it.

Relu alone does not give the required output, backpropagation updates the values in such a way that the required values are in positive for a certain decision to be made. (This is based on its information of truth value and loss).

Eliminating negatives is a characteristic of relu, relu says it considers all the negative values as same(0 in value). There are other activation which gives slight importance to the magnitude of the negative value like LeakyRelu.

1 Like

Hey @aakashns I am having trouble starting jupyter notebook from virtual enviroment. Jupyter notebook is working fine out of virtual enviroment but not inside it.

i guess this is the problem

Hi, Great tutorial.
One question, when I change the batch_size to 1000, the accuracy actually decreases in different epochs, and it decrease even more for 10000. That is counter-intuitive, as I figured more batch in each data sample should make it more accurate. Why that happened?