High accuracy and incorrect predictions

I tried apply the ResNet9 model to another dataset, avaliable here.
I’ve made some changes so the model could run, and it all seemed to be working fine.
When training, I could reach a very high accuracy, above 0.99. But when I tell the model do predict a single image, it ALWAYS fails.
Can someone help me find out what’s going on? I can share my notebook if necessary.

Thanks in advance

I am facing the same problem, however, maybe your data, not enough or it may be that your validation data not representative of your testing data. Or your data has many false positives/negatives, biased data or your problem is too complex for the model and high performing models are often highly sensitive. Maybe your model is overfitted and not generalised enough? I tried my best and waiting for a better answer.

If the problem still persists do share a link to your notebook here.

Can you please share the Jovian Notebook link, The Google Colab link is a temporary link, and also it asks for user’s permission.

Ok!

Can anyone please help me with this? I can’t find the error in my code

Hey, Extremely sorry for the late reply, I see you are getting a wrong prediction but a very good accuracy because you have not applied normalization. Due to this when you try to apply torch.max(yb, dim = 1), it takes the maximum value from an unnormalized set of prediction and gets the result according to that. So the class with the highest value is the predicted value.
As far as I know the solution to this problem is applying normalization in the train and test data.
The below image explains how I came to know about the error.
I printed out the prediction you were getting and checked what is the maximum value, (In the below image I have already applied normalization). When normalization was not applied the value was not present in a particular range rather it was having a high range and was biased. After I applied normalization I was getting correct results from your code.


Results after normalization.

Thank you very much for your help!
I did not apply normalization to the pictures because finding the means and standard deviations of the dataset turned out to be way more difficult than I thought.
Do you have any suggestion? Should I keep trying to do that? Or should I search for another type of normalization?

1 Like

I think you should come up with some way to find the mean and standard deviations of the dataset, Anyways even if you are unable to find the exact mean and standard deviation, You can give something like (0.5, 0.5, 0.5), this is just a shortcut way, as the data will normalize with this value. The better way would be to find the mean and standard deviations for the data. If you give some effort and time I think you can do that.
Currently, I don’t have any other idea for this. I will let you know if I get a better one.

It worked! Thank you very much for your help!

1 Like