<p>I have a dataset with an input size of 155x155, with the output being 155 x 1 with a 3-4 layer neural net being used for regression. With such a small sample size, should I use full batch gradient descent (so all 155 samples) or use mini batch/stochastic gradient descent. I have read that using smaller mini batch sizes allows better generalisation, but as the batch size is very very small computationally it shouldn't be a burden to use BGD.</p>

CC BY-SA 4.0
2021-02-19T15:15:52.363Z
Input size of 155x155 is rather ambiguous i feel, it could mean a input with dimension 155x155 and not the size of the training set. If the number of samples is small as in your case, I believe that a BGD should converge faster than a SGD. However, in your case where you have so few samples, the time to converge shouldn't be that much significant. FYI: Some Mini batch sizes range up to 256 samples
calveeen
CC BY-SA 4.0
2021-02-19T15:17:10.577Z
Thanks for your answer! To clarify, the training set size is 155x155 not the dimension.
Max
CC BY-SA 4.0
2021-02-19T15:18:25.397Z
You mean (n_samples, n_features) = (155, 155) right?
calveeen
CC BY-SA 4.0
2021-02-19T15:24:21.390Z
Yes, exactly right!
Max
<neural-networks><gradient-descent>
Should I use batch gradient descent when I have a small sample size?
Max
# Should I use batch gradient descent when I have a small sample size?

(Asked by: username_0 on 2021-02-19)

I have a dataset with an input size of 155x155, with the output being 155 x 1 with a 3-4 layer neural net being used for regression. With such a small sample size, should I use full batch gradient descent (so all 155 samples) or use mini batch/stochastic gradient descent. I have read that using smaller mini batch sizes allows better generalisation, but as the batch size is very very small computationally it shouldn't be a burden to use BGD.

**Tags:** neural-networks, gradient-descent

> **Comments:**
> **username_1**: Input size of 155x155 is rather ambiguous i feel, it could mean a input with dimension 155x155 and not the size of the training set. If the number of samples is small as in your case, I believe that a BGD should converge faster than a SGD. However, in your case where you have so few samples, the time to converge shouldn't be that much significant. FYI: Some Mini batch sizes range up to 256 samples  
> **username_0**: Thanks for your answer! To clarify, the training set size is 155x155 not the dimension.  
> **username_1**: You mean (n_samples, n_features) = (155, 155) right?  
> **username_0**: Yes, exactly right!  

---

## 0 answers

