File size: 1,363 Bytes
94e735e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
ο»ΏMany cutting-edge computer vision models consist of multiple stages:  
➰ backbone extracts the features,  
➰ neck refines the features,  
➰ head makes the detection for the task.  
Implementing this is cumbersome, so πŸ€— transformers has an API for this: Backbone!  

![image_1](image_1.jpg)

Let's see an example of such model. 
Assuming we would like to initialize a multi-stage instance segmentation model with ResNet backbone and MaskFormer neck and a head, you can use the backbone API like following (left comments for clarity) πŸ‘‡ 

![image_2](image_2.jpg)

One can also use a backbone just to get features from any stage. You can initialize any backbone with `AutoBackbone` class. 
See below how to initialize a backbone and getting the feature maps at any stage πŸ‘‡ 

![image_3](image_3.jpg)

Backbone API also supports any timm backbone of your choice! Check out a variation of timm backbones [here](https://t.co/Voiv0QCPB3).

![image_4](image_4.jpg)

Leaving some links πŸ”—:  
πŸ“– I've created a [notebook](https://t.co/PNfmBvdrtt) for you to play with it  
πŸ“’ [Backbone API docs](https://t.co/Yi9F8qAigO)
πŸ““ [AutoBackbone docs](https://t.co/PGo9oILHDw) πŸ’œ  
(all written with love by me!)  
 

> [!NOTE]
[Orignial tweet](https://twitter.com/mervenoyann/status/1749841426177810502) (January 23, 2024)