pytorch roberta masked-lm transformers license:mit fill-mask alloys metallurgy
.gitattributes README.md added_tokens.json config.json merges.txt pytorch_model.bin special_tokens_map.json tokenizer.json tokenizer_config.json vocab.json
---
license: mit
tags : 
- fill-mask
- alloys
- metallurgy
widget:
- text: "O 6 3 3 ,<mask> 6 7 , Ge 3 0 0 , Na2O 1 0 0 , GeO2 9 0 0"
---
# GlassBERTa
## Language Modelling as Unsupervised Pre-Training for Glass Alloys
### Abstract:
Alloy Property Prediction is a task under the sub field of Alloy Material Science wherein Machine Learning has been applied rigorously.  This is modeled as a Supervised Task wherein Alloy Composition is provided for the Model to predict a desired property. Efficiency of tasks such as *Alloy Property Prediction*, Alloy Synthesis can be modeled additionally with an Unsupervised Pre-training Task. We describe the idea of Pre-training using Language Modelling kind of approach in terms of Alloy Compositions.We specifically inspect that random masking proposed in is not suitable for modelling Alloys. We further go on proposing two types of masking strategies that are used to train GlassBERTa to encompass the properties of an Alloy Composition. The results suggest that Pre-training is an important field of direction in this field of research for further improvement.
### Authors:
Reshinth Adithyan, Aditya TS, Roakesh, Jothikrishna, Kalaiselvan Baskaran
### Footnote:
Work done via [MLDMM Lab](https://sites.google.com/view/mldmm-lab/home)
![alt text](https://lh4.googleusercontent.com/4L1C4_7ZBScAs9TIlkbyfjlotpnlnA4w22PLJXDWrYzh434Cu8RBhExvfBNdV8roOSb_k3WsM6MQHxv0zErcUhg=w16383 "Machine Learning for Design of Mechanical Materials Lab")
