Efficient Transformer Encoders for Mask2Former-style models
Paper
•
2404.15244
•
Published
•
1
Encoder - Weighted Stochastic Depth
Note Our motivation stems from the observation that layers within the transformer encoder of M2F exhibit non-uniform contributions to Panoptic Quality (PQ) [19], as discussed in Sec. 1. This prompts us to question the necessity of all K = 6 layers for every image and target minimizing layer usage... Components: 1. Model suitability for early exiting 2. Efficient and effective gating network for optimal exit decision making 3. Dynamic control mechanism for cost-performance trade-off