About the fit function

In the first lecture video on https://www.youtube.com/watch?v=5ioMqzMRFgM&t=4861s at 1:21:01 while applying the fit fuction if we get the loss vaalue least at some poit consider as 70th epoch the loss value at 100th that is the last epoch is considerebly high compared to the minimumI have found so for my model to be good I need the model of the 70th epoch so how can I get it? as the final model will be of the 100th epoch…!!

You can’t turn back your model as far as I know, a better way will be to decrease the learning rate and train the model for a few more epoch untill it decreases.

Oo :slightly_frowning_face: btw thanks for your reply…!! :smile:

@vsvgupta8 @birajde
You can’t revert the state from 100th epoch to back to 70th epoch. But in the future, you can look at saving checkpoints (say after every 10 epochs, or each time the loss decreased)so that you will have this option to load back any previously saved state. It’s a good convention to follow, esp. moving into Deep Learning you may save a lot of time with this convention.

Check this out to know how to save and load models https://pytorch.org/tutorials/beginner/saving_loading_models.html

1 Like

Thanks for this useful information @PrajwalPrashanth .

1 Like

Oo wow it looks cool thanks a lot @PrajwalPrashanth for sharing the info

1 Like

If I wanted to use the model with the best performance -whatever the last performance is on the last training epoch- I would add a performance checker in the fit function, such as:

#performance comparison between epochs
def model_performance(epoch,performance,loss):
    if loss.item() <= performance[list(performance.keys())[-1]]['loss']:
        performance[epoch] = {'loss': loss.item(), 'weights': model.weight, 'bias': model.bias,} 

# Utility function to train the model
def fit(num_epochs, model, loss_fn, opt, train_dl):
    performance = dict()
    # Repeat for given number of epochs
    for epoch in range(num_epochs):
    
        # Train with batches of data
        for xb,yb in train_dl:
        
            # 1. Generate predictions
            pred = model(xb)
        
            # 2. Calculate loss
            loss = loss_fn(pred, yb)
        
            # 3. Compute gradients
            loss.backward()
        
            # 4. Update parameters using gradients
            opt.step()
        
            # 5. Reset the gradients to zero
            opt.zero_grad()

      # storing performance for the first epoch
      if epoch == 0:
            performance[epoch] = {'loss': loss.item(), 'weights': model.weight, 'bias': model.bias} 
      if epoch > 0:
            model_performance(epoch,performance, loss)
  
  # Print the progress
        if (epoch+1) % 10 == 0:
            print('Epoch [{}/{}], Loss: {:.4f}'.format(epoch+1, num_epochs, loss.item()))

    return performance

As a result fit function returns a performance dictionary of the training which you can choose the model you like the best.

Here is the notebook.

1 Like