Pinterest, Engineering, Monetization.
small fixed dimension. More Feature Interaction: A general practice when using Transformer on modeling user sequence is to first embed the entire sequence into a vector, then use this vector to interact with other features. However, early stage feature interaction is essential for ranking models.