T4 - bfloat 16 not support
Hi,
Thanks for your work.
I'd like to know if you plan to release an other version of the model in float16 instead of bfloat16.
Best regards,
+1, I think this can be supported easily by adding a device argument to the chat method call, which is then used here: https://huggingface.co/ucaslcl/GOT-OCR2_0/blob/main/modeling_GOT.py#L566
this notebook works on colab but with the L4 gpu not the T4 https://colab.research.google.com/drive/1J_2SyiGvbBt2ohg_aoMyXpc0xjg9LFJg?usp=sharing
T4 support colab , feel free to like if it's worked fine : https://colab.research.google.com/drive/1oMA9u4M_hT5gxd80TptA5ik2xnERCC8F?usp=sharing
T4 support colab , feel free to like if it's worked fine : https://colab.research.google.com/drive/1oMA9u4M_hT5gxd80TptA5ik2xnERCC8F?usp=sharing
@jeanflop Can you please share access to the colab file you shared?
T4 support colab , feel free to like if it's worked fine : https://colab.research.google.com/drive/1oMA9u4M_hT5gxd80TptA5ik2xnERCC8F?usp=sharing
@jeanflop Can you please share access to the colab file you shared?
sure , done
Why not load it in float32 then? It's just a sub-1B model.
Managed to run on CPU in Windows 10
Updated files in archive link
https://drive.google.com/file/d/1cUMwQyWDtk0XUsdYEKbrFlpUIU-Okl8x/view?usp=sharing
Note that the run.py file is located in the model folder.
/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1150: FutureWarning: resume_download is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use force_download=True.
warnings.warn(
/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1150: FutureWarning: resume_download is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use force_download=True.
warnings.warn(
/usr/local/lib/python3.10/dist-packages/gradio/components/dropdown.py:188: UserWarning: The value passed into gr.Dropdown() is not in the list of choices. Please update the list of choices to include: or set allow_custom_value=True.
warnings.warn(
Running on local URL: http://127.0.0.1:7860
To create a public link, set share=True in launch().
not run in broser