Hi Mike!



 mkmitchell:

I do augment my training data.  I’ll attempt removing that.  My accuracy shoots up to 90% in the first epoch.  I believe I may be overfitting.


I’m still not completely clear on what you’re are doing and what
your problem is.
Just to make sure that we’re on the same page regarding context
and terminology, here are a few comments:
Training, validation, and test data:  You train on your training data.
That is you fit your network to get good results on your training
data.  You then run your network on your test data to see if you
get similar results.  It’s okay if your test results are a little worse
than your training results – after all, you did fit your training data.
But if your test results are significantly worse, then you probably
optimized too far, and overfit your training data.
If you want to get a little fancier, while you’re training, you can
run your partially-trained network on your so-called validation
data.  You might then stop training when your validation results
get good enough, or when they start to get worse, or when they
start to diverge too much from your training results.
Your validation and test results ought to be quite similar and in
many cases you don’t really need separate validation and test
data.  But if you use your validation results as a more substantive
part of your model building, for example to tweak your learning
rate or explore modified network architectures, then it would
be prudent to run your final model on some “clean” test data
that hadn’t been used at all when training your model.
You said in your original post that when you reran your model
on some training data, your accuracy fell from 90% to 60%.
This would be very surprising and suggest an outright error.
Even if you overfit your training data, you should still get
good results on your training data.  Of course, you might
get significantly worse results on your test data, but that’s not
what you said in your original post.
The point is that whether or not you overfit, or whether or not
you augment your training data, if you rerun your model on
training data, you should get the same results as when you
ran it on the training data before.  If you get significantly
different results (i.e., 60% accuracy instead of 90%) something
looks very fishy.
So I only see two choices:  You have an error in your calculation
somewhere, or your 60% accuracy came from running your model
on what turned out to be an unusual subsample of your training
data.  This latter seems quite unlikely, but not impossible.
If you are still having this issue, perhaps you could clarify what
you did, and add a few more details about which, and which kinds
of data give you which accuracies.
Best.
K. Frank