Geo-Sign 🌐✋ → 📝

Hyperbolic Contrastive Regularisation for Geometrically-Aware Sign-Language Translation

Paper: Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign-Language Translation
Edward Fish, Richard Bowden, CVSSP – University of Surrey (arXiv:2506.00129, May 2025)
Code: https://github.com/ed-fish/geo-sign Paper https://arxiv.org/pdf/2506.00129v1

Code Use

Download the weights and data labels from the files section of this repo and add them to the github repository https://github.com/ed-fish/geo-sign.

You will also need the base-mt5 model from https://huggingface.co/google/mt5-base and put it in the pretrained_weight folder.

Data -> ./Data

best.pth -> ./checkpoints/best.pth

pretraining.pth -> ./checkpoints/pretraining.pth

<https://huggingface.co/google/mt5-base> -> ./pretrained_weight

TL;DR

Geo-Sign projects pose-based sign-language features into a learnable Poincaré ball and aligns them with text embeddings via a geometric contrastive loss.
Compared with the strong Uni-Sign pose baseline, Geo-Sign boosts BLEU-4 by +1.81 and ROUGE-L by +3.03 on the CSL-Daily benchmark while keeping privacy-friendly skeletal inputs only.

Intended Uses & Scope

Primary – Sign-language-to-text translation research, especially for resource-constrained or privacy-sensitive settings where RGB video is unavailable.
Out-of-scope – Real-time production deployments without reliable pose estimation, medical or legal interpretations, or languages beyond datasets the model was trained on.

Evaluation

Dataset	Modality	BLEU-4 ↑	ROUGE-L ↑
CSL-Daily (test)	Pose-only	27.42	57.95

Geo-Sign outperforms all previous gloss-free pose-only methods and rivals many RGB- or gloss-based systems.

Limitations & Ethical Considerations

Pose-estimation dependency – Errors in upstream key-points propagate to the translation.
Training latency – Hyperbolic operations slow training (~4–6 ×) but add no cost at inference.
Generalisation – Evaluated only on Chinese Sign Language; other sign languages are not guaranteed.
Mis-translation risk – Automatic SLT can mis-communicate; keep a human in the loop for critical use cases.
Biases – CSL-Daily is domain-specific (news/TV); outputs may reflect that linguistic style.

Citation

@article{fish2025geo,
  title={Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translation},
  author={Fish, Edward and Bowden, Richard},
  journal={arXiv preprint arXiv:2506.00129},
  year={2025}
}```