How to do inference on Embed function with multiple gpus?
How to do inference on Embed function with multiple gpus?
See the ModelGenerator documentation here
https://genbio-ai.github.io/ModelGenerator/usage/saving_outputs/#backbone-embedding-and-inference
This will automatically scale with available hardware.
model = Embed.from_config({"model.backbone": "aido_dna_7b",
}).eval()
How can I use in here? The code in the link is unclear.
We strongly recommend using the ModelGenerator CLI as the easiest way to predict/save embeddings. There are instructions for providing new files here.
https://genbio-ai.github.io/ModelGenerator/experiment_design/datafiles/
If you need to have the model online or in-memory for some reason, you can use standard Lightning or PyTorch tools for inference. The model you're loading is a LightningModule. You may use Lightning's Trainer to manage multi-device inference.
https://lightning.ai/docs/pytorch/stable/deploy/production_basic.html#enable-distributed-inference
Lightning can also be used with torchrun for multi-device workloads.
https://lightning.ai/docs/pytorch/stable/clouds/cluster_intermediate_2.html
"At their core, backbones are PyTorch nn.Module objects with a few extra interfaces. To implement a new backbone, subclass a backbone interface and implement the required methods."
When I check the model backbone, I saw that it uses PyTorch nn.Module backbones. Sorry, couldn't understand how to use the below link:
https://lightning.ai/docs/pytorch/stable/deploy/production_basic.html#enable-distributed-inference
Can you provide me an example code?