metadata

license: mit
base_model:
  - entropy/roberta_zinc_480m
tags:
  - chemistry

Roberta Zinc Compression Head

This model is trained to compress embeddings generated by the roberta_zinc_480m model from the native size of 768 to compressed sizes 512, 256, 128, 64, and 32.

The compressed embeddings are trained to maintain cosine similarity computed using the native embeddings.