Answers.Body: <ol>
  <li>You’ll want a reasonable GPU (probably 8GB+), but otherwise no special hardware needed.* You may need to tune down sequence length and batch size to fit your GPU; RAM will be the limiting factor. Don’t try it on a CPU. It will “work” but you’re gonna have a bad time.</li>
  <li>Try the <a href="https://huggingface.co/transformers/tokenizer_summary.html" rel="nofollow noreferrer">Huggingface Transformers library</a> as your implementation. It’s well documented and straightforward and includes both models.</li>
  </ol>
  <p>*assuming an Nvida GPU or something compatible with CUDA. Things are rather hairier on Apple hardware. But you can always grab a cloud VM for a few hours</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-11T06:37:10.423Z
Answers.LastActivityDate: 2021-08-11T06:37:10.423Z
Answers.UserDisplayName: G__
Body: <p>I am thinking about a project and have a few questions before I accept it. Would be grateful I anyone experienced of you could give me some advice.</p>
  <p>In the project, I have been given a data set with (rather small) 30.000 text documents, which are labeled with 0 and 1. I want to train and evaluate (with respect to accuracy) a BERT and XLNet model.</p>
  <p>Can you give me some rough estimates for the following questions?:</p>
  <ol>
  <li>How much computing power do I need for this task, i.e. can I simply use my private laptop for this or do I need a special CPU/GPU for it?</li>
  <li>So far, I just worked with classical machine learning models (e.g. random forests, SVMs, etc.). I am not experienced deep learning architectures yet. How difficult would it be to implement a BERT oder XLNet model with my own data set, having no experience with BERT oder XLNet yet? I.e. how much code would it be that I have to develop by myself? And would I need a deep understanding for it or would be sufficient to follow an online tutorial and basically copy the code from there?
  Many thanks.</li>
  </ol>
  
Tags: <deep-learning><natural-language-processing><bert><text-classification>
Title: Training and Evaluating BERT and XLNET
UserDisplayName: MiFischer22
ThreadText: # Training and Evaluating BERT and XLNET
  
  (Asked by: username_0 on 2021-08-06)
  
  I am thinking about a project and have a few questions before I accept it. Would be grateful I anyone experienced of you could give me some advice.
  
  In the project, I have been given a data set with (rather small) 30.000 text documents, which are labeled with 0 and 1. I want to train and evaluate (with respect to accuracy) a BERT and XLNet model.
  
  Can you give me some rough estimates for the following questions?:
  
  1. How much computing power do I need for this task, i.e. can I simply use my private laptop for this or do I need a special CPU/GPU for it?
  2. So far, I just worked with classical machine learning models (e.g. random forests, SVMs, etc.). I am not experienced deep learning architectures yet. How difficult would it be to implement a BERT oder XLNet model with my own data set, having no experience with BERT oder XLNet yet? I.e. how much code would it be that I have to develop by myself? And would I need a deep understanding for it or would be sufficient to follow an online tutorial and basically copy the code from there?
     Many thanks.
  
  **Tags:** deep-learning, natural-language-processing, bert, text-classification
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-08-11. Score: 1
  
  1. You’ll want a reasonable GPU (probably 8GB+), but otherwise no special hardware needed.\* You may need to tune down sequence length and batch size to fit your GPU; RAM will be the limiting factor. Don’t try it on a CPU. It will “work” but you’re gonna have a bad time.
  2. Try the [Huggingface Transformers library](https://huggingface.co/transformers/tokenizer_summary.html) as your implementation. It’s well documented and straightforward and includes both models.
  
  \*assuming an Nvida GPU or something compatible with CUDA. Things are rather hairier on Apple hardware. But you can always grab a cloud VM for a few hours
  
  
  ---
  
  