mrprimenotes
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -29,8 +29,6 @@ base_model:
|
|
29 |
### Summary
|
30 |
Whisper is a powerful speech recognition platform developed by OpenAI. This model has been specially optimized for converting sign language input features into german text.
|
31 |
|
32 |
-
|
33 |
-
|
34 |
### Applications
|
35 |
The model is based on 'primeline/whisper-large-v3-german' and used (in combination with google mediapipe) to translate a video of german sign language into text. This model decodes a sequence of input features, where each input feature represents keypoints extracted from a video (body hands, upper body and face), into text.
|
36 |
|
@@ -46,13 +44,13 @@ TBD
|
|
46 |
```python
|
47 |
import torch
|
48 |
from transformers import WhisperForConditionalGeneration, AutoProcessor, AutoTokenizer, AutoConfig
|
|
|
49 |
|
50 |
device = "cuda:0" if torch.cuda.is_available() else "cpu"
|
51 |
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
|
52 |
|
53 |
-
# See custom config in model.py for configuration options.
|
54 |
-
|
55 |
# First load the config using AutoConfig
|
|
|
56 |
config = AutoConfig.from_pretrained(
|
57 |
"mrprimenotes/sign-whisper-german",
|
58 |
trust_remote_code=True,
|
@@ -101,6 +99,7 @@ model = AutoModel.from_pretrained(
|
|
101 |
# output.loss
|
102 |
# output.shape --> b x sq
|
103 |
|
|
|
104 |
train_dataset = YourSignDataset(...)
|
105 |
val_dataset = YourSignDataset(...)
|
106 |
|
|
|
29 |
### Summary
|
30 |
Whisper is a powerful speech recognition platform developed by OpenAI. This model has been specially optimized for converting sign language input features into german text.
|
31 |
|
|
|
|
|
32 |
### Applications
|
33 |
The model is based on 'primeline/whisper-large-v3-german' and used (in combination with google mediapipe) to translate a video of german sign language into text. This model decodes a sequence of input features, where each input feature represents keypoints extracted from a video (body hands, upper body and face), into text.
|
34 |
|
|
|
44 |
```python
|
45 |
import torch
|
46 |
from transformers import WhisperForConditionalGeneration, AutoProcessor, AutoTokenizer, AutoConfig
|
47 |
+
from datasets import load_dataset
|
48 |
|
49 |
device = "cuda:0" if torch.cuda.is_available() else "cpu"
|
50 |
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
|
51 |
|
|
|
|
|
52 |
# First load the config using AutoConfig
|
53 |
+
# See custom config in model.py for configuration options.
|
54 |
config = AutoConfig.from_pretrained(
|
55 |
"mrprimenotes/sign-whisper-german",
|
56 |
trust_remote_code=True,
|
|
|
99 |
# output.loss
|
100 |
# output.shape --> b x sq
|
101 |
|
102 |
+
# Load your dataset (e.g. mrprimenotes/sign-whisper-german-example)
|
103 |
train_dataset = YourSignDataset(...)
|
104 |
val_dataset = YourSignDataset(...)
|
105 |
|