<p>I want to train my vision transformer model on a benchmark for an image segmentation task: (<a href="https://arxiv.org/abs/2110.08733" rel="nofollow noreferrer">LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation</a>) (<a href="https://github.com/Junjue-Wang/LoveDA" rel="nofollow noreferrer">GitHub</a>), but I don't get any acceptable result from my model.</p>
<p>I don't have a good result either in evaluation nor (obviously) in test.</p>
<p>I'm frustrated since I have trained my model with different parameters: learning rate : [0.001,0001, 6e-e].</p>
<p>The training datasets include about 3000 images, the number of epochs is 1500.</p>
<p>I have used different loss functions: Cross Entropy, Dice loss, Focal loss, and a combination of those losses.</p>
<p>So, I'm wondering if anyone has any experience that could help me find a solution and improve my model's performance?</p>

CC BY-SA 4.0
2022-06-08T14:48:08.070Z
How does the loss function behave during training?
Lee Reeves
CC BY-SA 4.0
2022-06-08T16:35:59.890Z
thanks Lee,
well cross entropy works better than any of them it decreases gradually, but after about 1200 epochs it will stop decreasing
Also I have checked the model with more general dataset, it was working fine 
but still have difficulty in compareto other deep learning like deeplab
sara yaghoobi
CC BY-SA 4.0
2022-06-08T16:55:46.990Z
@sarayaghoobi Edit your post to include the plots of the loss and other metrics, if possible. Said that, a training dataset of 3000 images seems to be very small. How many training parameters does your model have?
nbro
CC BY-SA 4.0
2022-06-08T17:06:12.080Z
Have you tried a learning rate schedule? A lower learning rate might help the model fine tune after converging at a poor solution.
Lee Reeves
<training><transformer><image-segmentation><testing>
What considerations should I take to train my transformer model?
sara yaghoobi
# What considerations should I take to train my transformer model?

(Asked by: username_0 on 2022-06-08)

I want to train my vision transformer model on a benchmark for an image segmentation task: ([LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation](https://arxiv.org/abs/2110.08733)) ([GitHub](https://github.com/Junjue-Wang/LoveDA)), but I don't get any acceptable result from my model.

I don't have a good result either in evaluation nor (obviously) in test.

I'm frustrated since I have trained my model with different parameters: learning rate : [0.001,0001, 6e-e].

The training datasets include about 3000 images, the number of epochs is 1500.

I have used different loss functions: Cross Entropy, Dice loss, Focal loss, and a combination of those losses.

So, I'm wondering if anyone has any experience that could help me find a solution and improve my model's performance?

**Tags:** training, transformer, image-segmentation, testing

> **Comments:**
> **username_1**: How does the loss function behave during training?  
> **username_0**: thanks Lee, well cross entropy works better than any of them it decreases gradually, but after about 1200 epochs it will stop decreasing Also I have checked the model with more general dataset, it was working fine  but still have difficulty in compareto other deep learning like deeplab  
> **username_2**: @sarayaghoobi Edit your post to include the plots of the loss and other metrics, if possible. Said that, a training dataset of 3000 images seems to be very small. How many training parameters does your model have?  
> **username_1**: Have you tried a learning rate schedule? A lower learning rate might help the model fine tune after converging at a poor solution.  

---

## 0 answers

