Question: Am I doing modulus wrong? Because in Java `-13 % 64` evaluates to `-13` but I want to get `51`.

Answer: 
Are you sure you are working in Java? 'cause Java gives -13 % 64 = -13 as expected. The sign of dividend!
In my version of Java JDK 1.8.0\_05 -13%64=-13

you could try -13-(int(-13/64))
in other words do division cast to an integer to get rid of the fraction part
then subtract from numerator
So numerator-(int(numerator/denominator)) should give the correct remainder & sign