readme: ---
  tags:
  - generated_from_keras_callback
  model-index:
  - name: GeoBERT
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information Keras had access to. You should
  probably proofread and complete it, then remove this comment. -->
  
  # GeoBERT
  
  GeoBERT is a NER model that was fine-tuned from SciBERT on the Geoscientific Corpus dataset.
  The model was trained on the Labeled Geoscientific Corpus dataset (~1 million sentences).
  
  
  ## Intended uses
  
  The NER product in this model has a goal to identify four main semantic types or categories related to Geosciences. 
  
  1. GeoPetro for any entities that belong to all terms in Geosciences
  2. GeoMeth for all tools or methods associated with Geosciences
  3. GeoLoc to identify geological locations
  4. GeoTime for identifying the geological time scale entities
  
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 14000, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
  - training_precision: mixed_float16
  
  
  ### Framework versions
  
  - Transformers 4.22.1
  - TensorFlow 2.10.0
  - Datasets 2.4.0
  - Tokenizers 0.12.1
  
  ## Model performances (metric: seqeval)
  
  entity|precision|recall|f1
  -|-|-|-
  GeoLoc  |0.9727|0.9591|0.9658 
  GeoMeth |0.9433|0.9447|0.9445
  GeoPetro|0.9767|0.9745|0.9756
  GeoTime |0.9695|0.9666|0.9680
  
  ## How to use GeoBERT with HuggingFace
  
  ##### Load GeoBERT and its sub-word tokenizer :
  
  ```python
  from transformers import AutoTokenizer, AutoModelForTokenClassification
  
  tokenizer = AutoTokenizer.from_pretrained("botryan96/GeoBERT")
  model = AutoModelForTokenClassification.from_pretrained("botryan96/GeoBERT")
  
  #Define the pipeline
  from transformers import pipeline
  ner_machine = pipeline('ner',model = models,tokenizer=tokenizer, aggregation_strategy="simple")
  
  #Define the sentence
  sentence = 'In North America, the water storage in the seepage face model is higher than the base case because positive pore pressure is requisite for drainage through a seepage face boundary condition. The result from the resistivity data supports the notion, especially in the northern part of the Sandstone Sediment formation. The active formation of America has a big potential for Oil and Gas based on the seismic section, has been activated since the Paleozoic'
  
  #Deploy the NER Machine
  ner_machine(sentence)
  ```