Rep-MTL: Unleashing the Power of Representation-level Task Saliency for
  Multi-Task Learning
Despite the promise of Multi-Task Learning in leveraging complementary
knowledge across tasks, existing multi-task optimization (MTO) techniques
remain fixated on resolving conflicts via optimizer-centric loss scaling and
gradient manipulation strategies, yet fail to deliver consistent gains. In this
paper, we argue that the shared representation space, where task interactions
naturally occur, offers rich information and potential for operations
complementary to existing optimizers, especially for facilitating the
inter-task complementarity, which is rarely explored in MTO. This intuition
leads to Rep-MTL, which exploits the representation-level task saliency to
quantify interactions between task-specific optimization and shared
representation learning. By steering these saliencies through entropy-based
penalization and sample-wise cross-task alignment, Rep-MTL aims to mitigate
negative transfer by maintaining the effective training of individual tasks
instead pure conflict-solving, while explicitly promoting complementary
information sharing. Experiments are conducted on four challenging MTL
benchmarks covering both task-shift and domain-shift scenarios. The results
show that Rep-MTL, even paired with the basic equal weighting policy, achieves
competitive performance gains with favorable efficiency. Beyond standard
performance metrics, Power Law exponent analysis demonstrates Rep-MTL's
efficacy in balancing task-specific learning and cross-task sharing. The
project page is available at HERE.
68884427af872d625c10c66b
Zedong Wang
claimed_verified
2025-07-29T07:07:39.228Z
6594d390674349122ce6f368
https://cdn-avatars.huggingface.co/v1/production/uploads/6594d390674349122ce6f368/KdWz6lZyGYQpjAgBDeiC1.jpeg
Zedong Wang (Jacky)
user
ZedongWangAI
68884427af872d625c10c66c
Siyuan Li
claimed_verified
2025-07-29T07:07:42.285Z
640f7083208821a59b74c757
https://cdn-avatars.huggingface.co/v1/production/uploads/1678735253848-640f7083208821a59b74c757.jpeg
Siyuan Li
user
Lupin1998
68884427af872d625c10c66d
Dan Xu
Rep-MTL: Unleashing the Power of Representation-level Task Saliency for
  Multi-Task Learning
Despite the promise of Multi-Task Learning in leveraging complementary
knowledge across tasks, existing multi-task optimization (MTO) techniques
remain fixated on resolving conflicts via optimizer-centric loss scaling and
gradient manipulation strategies, yet fail to deliver consistent gains. In this
paper, we argue that the shared representation space, where task interactions
naturally occur, offers rich information and potential for operations
complementary to existing optimizers, especially for facilitating the
inter-task complementarity, which is rarely explored in MTO. This intuition
leads to Rep-MTL, which exploits the representation-level task saliency to
quantify interactions between task-specific optimization and shared
representation learning. By steering these saliencies through entropy-based
penalization and sample-wise cross-task alignment, Rep-MTL aims to mitigate
negative transfer by maintaining the effective training of individual tasks
instead pure conflict-solving, while explicitly promoting complementary
information sharing. Experiments are conducted on four challenging MTL
benchmarks covering both task-shift and domain-shift scenarios. The results
show that Rep-MTL, even paired with the basic equal weighting policy, achieves
competitive performance gains with favorable efficiency. Beyond standard
performance metrics, Power Law exponent analysis demonstrates Rep-MTL's
efficacy in balancing task-specific learning and cross-task sharing. The
project page is available at HERE.
Rep-MTL enhances multi-task learning by leveraging representation-level saliency to promote task complementarity and mitigate negative transfer.
Multi-Task Learning MTO shared representation space task saliency entropy-based penalization sample-wise cross-task alignment task-shift domain-shift Power Law exponent analysis