Spaces:
Runtime error
Runtime error
Felix Marty
commited on
Commit
·
17ca086
1
Parent(s):
590064e
better text
Browse files- app.py +4 -2
- defaults.py +1 -1
app.py
CHANGED
|
@@ -15,11 +15,13 @@ with gr.Blocks() as demo:
|
|
| 15 |
"""
|
| 16 |
Let's try out TorchServe + BetterTransformer!
|
| 17 |
|
| 18 |
-
BetterTransformer is a feature made available with PyTorch 1.13. allowing to use a fastpath execution for encoder attention blocks.
|
| 19 |
|
| 20 |
-
As a one-liner, you can use BetterTransformer
|
| 21 |
|
| 22 |
```
|
|
|
|
|
|
|
| 23 |
better_model = BetterTransformer.transform(model)
|
| 24 |
```
|
| 25 |
|
|
|
|
| 15 |
"""
|
| 16 |
Let's try out TorchServe + BetterTransformer!
|
| 17 |
|
| 18 |
+
BetterTransformer is a stable feature made available with [PyTorch 1.13](https://pytorch.org/blog/PyTorch-1.13-release/) allowing to use a fastpath execution for encoder attention blocks.
|
| 19 |
|
| 20 |
+
As a one-liner, you can convert your 🤗 Transformers models to use BetterTransformer thanks to the [🤗 Optimum](https://huggingface.co/docs/optimum/main/en/index) library:
|
| 21 |
|
| 22 |
```
|
| 23 |
+
from optimum.bettertransformer import BetterTransformer
|
| 24 |
+
|
| 25 |
better_model = BetterTransformer.transform(model)
|
| 26 |
```
|
| 27 |
|
defaults.py
CHANGED
|
@@ -35,4 +35,4 @@ BATCH_SIZE = 8 # fixed!
|
|
| 35 |
|
| 36 |
HEADERS = {"Content-Type": "text/plain"}
|
| 37 |
ADDRESS_VANILLA = "http://3.83.142.46:8080/predictions/my_tc"
|
| 38 |
-
ADDRESS_BETTERTRANSFORMER = "http://3.95.
|
|
|
|
| 35 |
|
| 36 |
HEADERS = {"Content-Type": "text/plain"}
|
| 37 |
ADDRESS_VANILLA = "http://3.83.142.46:8080/predictions/my_tc"
|
| 38 |
+
ADDRESS_BETTERTRANSFORMER = "http://3.95.136.2:8080/predictions/my_tc"
|