SinclairWang commited on
Commit
3b2aa31
·
1 Parent(s): 5fb751d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -25,7 +25,7 @@ The details are available at [Github:FS-ABSA](https://github.com/nustm/fs-absa)
25
  To bridge the domain gap between general pre-training and the task of interest in a specific domain (i.e., `laptop` in this repo), we conducted *domain-adaptive pre-training*,
26
  i.e., continuing pre-training the language model (i.e., T5) on the unlabeled corpus of the domain of interest (i.e., `laptop`) with the *text-infilling objective*
27
  (corruption rate of 15% and average span length of 1). We collect relevant 100k unlabeled reviews from Amazon Electronics for the laptop domain, respectively.
28
- For pre-training, we employ the [Adafactor](https://arxiv.org/abs/1804.04235) optimizer with a batch size of 80 and a constant learning rate of 1e-4.
29
 
30
  Our model can be seen as an enhanced T5 model in the laptop domain, which can be used for various NLP tasks related to the laptop domain,
31
  including but not limited to fine-grained sentiment analysis (ABSA), product-relevant Question Answering (PrQA), text style transfer, etc.
 
25
  To bridge the domain gap between general pre-training and the task of interest in a specific domain (i.e., `laptop` in this repo), we conducted *domain-adaptive pre-training*,
26
  i.e., continuing pre-training the language model (i.e., T5) on the unlabeled corpus of the domain of interest (i.e., `laptop`) with the *text-infilling objective*
27
  (corruption rate of 15% and average span length of 1). We collect relevant 100k unlabeled reviews from Amazon Electronics for the laptop domain, respectively.
28
+ For pre-training, we employ the [Adafactor](https://arxiv.org/abs/1804.04235) optimizer with a batch size of 84 and a constant learning rate of 1e-4.
29
 
30
  Our model can be seen as an enhanced T5 model in the laptop domain, which can be used for various NLP tasks related to the laptop domain,
31
  including but not limited to fine-grained sentiment analysis (ABSA), product-relevant Question Answering (PrQA), text style transfer, etc.