  Educational technologies nowadays increasingly use data and Machine Learning
(ML) models. This gives the students, instructors, and administrators support
and insights for the optimum policy. However, it is well acknowledged that ML
models are subject to bias, which raises concerns about the fairness, bias, and
discrimination of using these automated ML algorithms in education and its
unintended and unforeseen negative consequences. The contribution of bias
during the decision-making comes from datasets used for training ML models and
the model architecture. This paper presents a preliminary investigation of the
fairness of transformer neural networks on the two tabular datasets: Law School
and Student-Mathematics. In contrast to classical ML models, the
transformer-based models transform these tabular datasets into a richer
representation while solving the classification task. We use different fairness
metrics for evaluation and check the trade-off between fairness and accuracy of
the transformer-based models over the tabular datasets. Empirically, our
approach shows impressive results regarding the trade-off between fairness and
performance on the Law School dataset.

Machine Learning;Artificial Intelligence