I am new to Neural Networks and currently doing a project for university. I am trying to train a CNN using frames that portray me shooting a ball through a basket. And my aim is for the network to be able to classify the result( hit or miss) correctly. When I train the network, the training accuracy increases slowly until it reaches 100%, while the validation accuracy remains around 65% (It is important to mention here that 65% is the percentage of shots that have a Miss label. So the network gives the highest Validation accuracy when it predicts all frames are a miss) Does anyone have experience with a similar problem?
I am using PyTorch and Resnet18 ( have tried other architectures as well but they all gave the same result). My frames are jpg images of sie 224.
As an optimizer, both Adam and SGD gave the same result
Thank you in advance