6 Best Technique To Improve The Accuracy Of Neural Network W/O Overfitting
We often face a situation while trying to improve the accuracy of the neural network we end up overfitting the model on the training data. This leads to a poor prediction when we run the model of the test data. Hence I take a dataset and apply these techniques that not only improve the accuracy but also handles the overfitting issues.
In this notebook, we'll use the following techniques to train a state-of-the-art model in less than 5 minutes to achieve over 95% accuracy in classifying images from the Fruit 360 dataset :
- Data augmentation
Data augmentation in data analysis are techniques used to increase the amount of data by adding slightly modified copies of already existing data or newly created synthetic data from existing data. It acts as a regularizer and helps reduce overfitting when training a machine learning model.Read more
- Batch normalization
Batch normalization is a technique for training very deep neural networks that standardizes the inputs to a layer for each mini-batch. This has the effect of stabilizing the learning process and dramatically reducing the number of training epochs required to train deep networks.Read more
- Learning rate scheduling
Learning rate schedules seek to adjust the learning rate during training by reducing the learning rate according to a pre-defined schedule. Common learning rate schedules include time-based decay, step decay and exponential decay. Read more
- Weight Decay
We use weight decay to keep the weights small and avoid exploding gradient. Because the L2 norm of the weights are added to the loss, each iteration of your network will try to optimize/minimize the model weights in addition to the loss. This will help keep the weights as small as possible, preventing the weights to grow out of control, and thus avoid exploding gradient.Learn more
- Gradient clipping
Using gradient clipping you can prevent exploding gradients in neural networks.Gradient clipping limits the magnitude of the gradient.There are many ways to compute gradient clipping, but a common one is to rescale gradients so that their norm is at most a particular value. Learn more
- Adaptive optimizer
This family of optimizers has been introduced to solve the issues of the gradient descent’s algorithms. Their most important feature is that they don’t require a tuning of the learning rate value. Actually some libraries — i.e. Keras — still let you the possibility to manually tune it for more advanced trials.Learn more
Total number of images: 90483.
Training set size: 67692 images (one fruit or vegetable per image).
Test set size: 22688 images (one fruit or vegetable per image).
Number of classes: 131 (fruits and vegetables).
Image size: 100x100 pixels.
Filename format: imageindex100.jpg (e.g. 32100.jpg) or rimageindex100.jpg (e.g. r32100.jpg) or r2imageindex100.jpg or r3imageindex100.jpg. "r" stands for rotated fruit. "r2" means that the fruit was rotated around the 3rd axis. "100" comes from image size (100x100 pixels).
Different varieties of the same fruit (apple for instance) are stored as belonging to different classes.