mrprimenotes commited on
Commit
53a0ea3
·
verified ·
1 Parent(s): af47a58

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -4
README.md CHANGED
@@ -29,8 +29,6 @@ base_model:
29
  ### Summary
30
  Whisper is a powerful speech recognition platform developed by OpenAI. This model has been specially optimized for converting sign language input features into german text.
31
 
32
-
33
-
34
  ### Applications
35
  The model is based on 'primeline/whisper-large-v3-german' and used (in combination with google mediapipe) to translate a video of german sign language into text. This model decodes a sequence of input features, where each input feature represents keypoints extracted from a video (body hands, upper body and face), into text.
36
 
@@ -46,13 +44,13 @@ TBD
46
  ```python
47
  import torch
48
  from transformers import WhisperForConditionalGeneration, AutoProcessor, AutoTokenizer, AutoConfig
 
49
 
50
  device = "cuda:0" if torch.cuda.is_available() else "cpu"
51
  torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
52
 
53
- # See custom config in model.py for configuration options.
54
-
55
  # First load the config using AutoConfig
 
56
  config = AutoConfig.from_pretrained(
57
  "mrprimenotes/sign-whisper-german",
58
  trust_remote_code=True,
@@ -101,6 +99,7 @@ model = AutoModel.from_pretrained(
101
  # output.loss
102
  # output.shape --> b x sq
103
 
 
104
  train_dataset = YourSignDataset(...)
105
  val_dataset = YourSignDataset(...)
106
 
 
29
  ### Summary
30
  Whisper is a powerful speech recognition platform developed by OpenAI. This model has been specially optimized for converting sign language input features into german text.
31
 
 
 
32
  ### Applications
33
  The model is based on 'primeline/whisper-large-v3-german' and used (in combination with google mediapipe) to translate a video of german sign language into text. This model decodes a sequence of input features, where each input feature represents keypoints extracted from a video (body hands, upper body and face), into text.
34
 
 
44
  ```python
45
  import torch
46
  from transformers import WhisperForConditionalGeneration, AutoProcessor, AutoTokenizer, AutoConfig
47
+ from datasets import load_dataset
48
 
49
  device = "cuda:0" if torch.cuda.is_available() else "cpu"
50
  torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
51
 
 
 
52
  # First load the config using AutoConfig
53
+ # See custom config in model.py for configuration options.
54
  config = AutoConfig.from_pretrained(
55
  "mrprimenotes/sign-whisper-german",
56
  trust_remote_code=True,
 
99
  # output.loss
100
  # output.shape --> b x sq
101
 
102
+ # Load your dataset (e.g. mrprimenotes/sign-whisper-german-example)
103
  train_dataset = YourSignDataset(...)
104
  val_dataset = YourSignDataset(...)
105