chaojiang06/medreadme_medical_sentence_readability_prediction_CWI

Checkpoint for paper MedReadMe: A Systematic Study for Fine-grained Sentence Readability in Medical Domain

This is the best medical sentence readability model trained on our dataset. This checkpoint uses a standard huggingface sentence prediction model.

roberta-large+cwi.py+512+8+1e-5+1

This model is a fine-tuned version of roberta-large on the cwi dataset. It achieves the following results on the evaluation set:

• Loss: 0.2137
• Pearsonr: 0.8429
• Addition Pearsonr: 0.8429
• Addition Pearsonr Pvalue: 0.0000
• Addition Spearmanr: 0.8297
• Addition Spearmanr Pvalue: 0.0000

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 1
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Pearsonr	Addition Pearsonr	Addition Pearsonr Pvalue	Addition Spearmanr	Addition Spearmanr Pvalue
0.3654	1.0	323	0.6906	0.8163	0.8163	0.0000	0.8076	0.0000
0.1951	2.0	646	0.2740	0.8335	0.8335	0.0000	0.8310	0.0000
0.2671	3.0	969	0.2137	0.8429	0.8429	0.0000	0.8297	0.0000
0.1559	4.0	1292	0.2473	0.8238	0.8238	0.0000	0.8263	0.0000
0.1334	5.0	1615	0.2358	0.8427	0.8427	0.0000	0.8289	0.0000
0.07	6.0	1938	0.2452	0.8385	0.8385	0.0000	0.8311	0.0000
0.1171	7.0	2261	0.2491	0.8371	0.8371	0.0000	0.8274	0.0000
0.086	8.0	2584	0.2442	0.8358	0.8358	0.0000	0.8310	0.0000
0.0696	9.0	2907	0.2403	0.8323	0.8323	0.0000	0.8261	0.0000
0.0737	10.0	3230	0.2646	0.8295	0.8295	0.0000	0.8259	0.0000

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.1+cu121
• Datasets 2.15.0
• Tokenizers 0.14.1