Commit
·
3b2aa31
1
Parent(s):
5fb751d
Update README.md
Browse files
README.md
CHANGED
@@ -25,7 +25,7 @@ The details are available at [Github:FS-ABSA](https://github.com/nustm/fs-absa)
|
|
25 |
To bridge the domain gap between general pre-training and the task of interest in a specific domain (i.e., `laptop` in this repo), we conducted *domain-adaptive pre-training*,
|
26 |
i.e., continuing pre-training the language model (i.e., T5) on the unlabeled corpus of the domain of interest (i.e., `laptop`) with the *text-infilling objective*
|
27 |
(corruption rate of 15% and average span length of 1). We collect relevant 100k unlabeled reviews from Amazon Electronics for the laptop domain, respectively.
|
28 |
-
For pre-training, we employ the [Adafactor](https://arxiv.org/abs/1804.04235) optimizer with a batch size of
|
29 |
|
30 |
Our model can be seen as an enhanced T5 model in the laptop domain, which can be used for various NLP tasks related to the laptop domain,
|
31 |
including but not limited to fine-grained sentiment analysis (ABSA), product-relevant Question Answering (PrQA), text style transfer, etc.
|
|
|
25 |
To bridge the domain gap between general pre-training and the task of interest in a specific domain (i.e., `laptop` in this repo), we conducted *domain-adaptive pre-training*,
|
26 |
i.e., continuing pre-training the language model (i.e., T5) on the unlabeled corpus of the domain of interest (i.e., `laptop`) with the *text-infilling objective*
|
27 |
(corruption rate of 15% and average span length of 1). We collect relevant 100k unlabeled reviews from Amazon Electronics for the laptop domain, respectively.
|
28 |
+
For pre-training, we employ the [Adafactor](https://arxiv.org/abs/1804.04235) optimizer with a batch size of 84 and a constant learning rate of 1e-4.
|
29 |
|
30 |
Our model can be seen as an enhanced T5 model in the laptop domain, which can be used for various NLP tasks related to the laptop domain,
|
31 |
including but not limited to fine-grained sentiment analysis (ABSA), product-relevant Question Answering (PrQA), text style transfer, etc.
|