TerraTorch
Earth Observation
TerraMind
IBM
ESA
blumenstiel commited on
Commit
bed35d5
·
verified ·
1 Parent(s): 1a7c11a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +61 -1
README.md CHANGED
@@ -1,5 +1,65 @@
1
  ---
 
2
  library_name: terratorch
 
 
 
 
 
 
 
3
  ---
4
 
5
- Model weights for the TerraMind 1.0 Tokenizer for LULC classes (9 classes from ESRI WorldCover).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: apache-2.0
3
  library_name: terratorch
4
+ datasets:
5
+ - ibm-esa-geospatial/TerraMesh
6
+ tags:
7
+ - Earth Observation
8
+ - TerraMind
9
+ - IBM
10
+ - ESA
11
  ---
12
 
13
+ # TerraMind 1.0 LULC Tokenizer
14
+
15
+ TerraMind is the first multimodal any-to-any generative foundation model for Earth Observation jointly developed by IBM, ESA, and Forschungszentrum Jülich.
16
+ The model is pre-trained using FSQ-VAE tokens as targets. This tokenizer encodes and decodes land-use land-cover (LULC) maps for the TerraMind model.
17
+
18
+ ![lulc_tokenizer.png](assets%2Flulc_tokenizer.png)
19
+
20
+ The tokenizer uses FSQ with five dimensions and a codebook size of 4'375 tokens.
21
+ The model was pre-trained for 20 epochs on nine million LULC images from the TerraMesh dataset which are sourced from [ESRI](https://planetarycomputer.microsoft.com/dataset/io-lulc-annual-v02).
22
+ The maps include nine classes and a 10th no-data class: No data, water, trees, flooded vegetation, crops, built area, bare ground, snow/ice, clouds, rangeland.
23
+
24
+ ## Usage
25
+
26
+ The tokenizer is fully integrated into the fine-tuning toolkit [TerraTorch](https://ibm.github.io/terratorch/).
27
+ You can initialize the pre-trained tokenizer with:
28
+
29
+ ```python
30
+ from terratorch.registry import FULL_MODEL_REGISTRY
31
+ model = FULL_MODEL_REGISTRY.build('terramind_v1_tokenizer_lulc', pretrained=True)
32
+ ```
33
+
34
+ Once the model is build, it can be used to encode image and decode tokens.
35
+
36
+ ```python
37
+ # Encode image
38
+ _, _, tokens = model.encode(lulc_tensor)
39
+ # Decode tokens
40
+ reconstruction = model.decode_tokens(tokens)
41
+ # Encode & decode
42
+ reconstruction = model(lulc_tensor)
43
+ ```
44
+
45
+ This tokenizer is automatically loaded with TerraMind generation models like `terramind_v1_base_generate`, see [here](https://huggingface.co/ibm-esa-geospatial/TerraMind-1.0-base#generations) for details.
46
+
47
+ We provide example code for the tokenizer at https://github.com/IBM/terramind.
48
+
49
+
50
+ ## Feedback
51
+
52
+ If you have feedback or any questions, please start a discussion in this HF repository or submitting an issue to [TerraMind](https://github.com/IBM/terramind) on GitHub.
53
+
54
+ ## Citation
55
+
56
+ If you use TerraMind in your research, please cite our [TerraMind](https://arxiv.org/abs/2504.11171) pre-print.
57
+
58
+ ```text
59
+ @article{jakubik2025terramind,
60
+ title={TerraMind: Large-Scale Generative Multimodality for Earth Observation},
61
+ author={Jakubik, Johannes and Yang, Felix and Blumenstiel, Benedikt and Scheurer, Erik and Sedona, Rocco and Maurogiovanni, Stefano and Bosmans, Jente and Dionelis, Nikolaos and Marsocci, Valerio and Kopp, Niklas and others},
62
+ journal={arXiv preprint arXiv:2504.11171},
63
+ year={2025}
64
+ }
65
+ ```