<ol>
<li>You’ll want a reasonable GPU (probably 8GB+), but otherwise no special hardware needed.* You may need to tune down sequence length and batch size to fit your GPU; RAM will be the limiting factor. Don’t try it on a CPU. It will “work” but you’re gonna have a bad time.</li>
<li>Try the <a href="https://huggingface.co/transformers/tokenizer_summary.html" rel="nofollow noreferrer">Huggingface Transformers library</a> as your implementation. It’s well documented and straightforward and includes both models.</li>
</ol>
<p>*assuming an Nvida GPU or something compatible with CUDA. Things are rather hairier on Apple hardware. But you can always grab a cloud VM for a few hours</p>

CC BY-SA 4.0
2021-08-11T06:37:10.423Z
2021-08-11T06:37:10.423Z
G__
<p>I am thinking about a project and have a few questions before I accept it. Would be grateful I anyone experienced of you could give me some advice.</p>
<p>In the project, I have been given a data set with (rather small) 30.000 text documents, which are labeled with 0 and 1. I want to train and evaluate (with respect to accuracy) a BERT and XLNet model.</p>
<p>Can you give me some rough estimates for the following questions?:</p>
<ol>
<li>How much computing power do I need for this task, i.e. can I simply use my private laptop for this or do I need a special CPU/GPU for it?</li>
<li>So far, I just worked with classical machine learning models (e.g. random forests, SVMs, etc.). I am not experienced deep learning architectures yet. How difficult would it be to implement a BERT oder XLNet model with my own data set, having no experience with BERT oder XLNet yet? I.e. how much code would it be that I have to develop by myself? And would I need a deep understanding for it or would be sufficient to follow an online tutorial and basically copy the code from there?
Many thanks.</li>
</ol>

<deep-learning><natural-language-processing><bert><text-classification>
Training and Evaluating BERT and XLNET
MiFischer22
# Training and Evaluating BERT and XLNET

(Asked by: username_0 on 2021-08-06)

I am thinking about a project and have a few questions before I accept it. Would be grateful I anyone experienced of you could give me some advice.

In the project, I have been given a data set with (rather small) 30.000 text documents, which are labeled with 0 and 1. I want to train and evaluate (with respect to accuracy) a BERT and XLNet model.

Can you give me some rough estimates for the following questions?:

1. How much computing power do I need for this task, i.e. can I simply use my private laptop for this or do I need a special CPU/GPU for it?
2. So far, I just worked with classical machine learning models (e.g. random forests, SVMs, etc.). I am not experienced deep learning architectures yet. How difficult would it be to implement a BERT oder XLNet model with my own data set, having no experience with BERT oder XLNet yet? I.e. how much code would it be that I have to develop by myself? And would I need a deep understanding for it or would be sufficient to follow an online tutorial and basically copy the code from there?
   Many thanks.

**Tags:** deep-learning, natural-language-processing, bert, text-classification


---

## 1 answers

### Answer by username_1 on 2021-08-11. Score: 1

1. You’ll want a reasonable GPU (probably 8GB+), but otherwise no special hardware needed.\* You may need to tune down sequence length and batch size to fit your GPU; RAM will be the limiting factor. Don’t try it on a CPU. It will “work” but you’re gonna have a bad time.
2. Try the [Huggingface Transformers library](https://huggingface.co/transformers/tokenizer_summary.html) as your implementation. It’s well documented and straightforward and includes both models.

\*assuming an Nvida GPU or something compatible with CUDA. Things are rather hairier on Apple hardware. But you can always grab a cloud VM for a few hours


---

