Hi, we’ve had an implementation up for a while now:

  
      
      GitHub
  
  
    

rdevon/DIM 43

Learning Deep Representations by Mutual Information Estimation and Maximization - rdevon/DIM


  
  
    
    
  
  


Though the NCE version with the “deep bilinear” model isn’t up yet. We are in the process of updating the repo.
That said, 60% is quite low and even with JSD and the concat-style scoring function should get at least 64-65% with the global vector (iirc) and over 71% for the local feature maps. Give our current repo a try (and the updated version once its finished), and let me know if you are encountering the same problems.