Solved by oras in post #2 
              
              
                I found the solution. It was an indexing issue with my labels. 
My labels were starting from 1 to 8, I changed them to 0…7 and that fixed the issue for me. 
Credit to this answer on Stackoverflow. 
Hope this will help someone in the future.