Recommendations, Machine Learning, Data Science, Recommendation System, Engineering.
optimizing for offline performance on rank placement, such as Precision and Recall @ K, NDCG, Hit Rate, etc… were not very robust over consecutive day runs. This was due to the high variability of host regularity on the platform and streaming schedules. Offline data quickly becomes less comparable