This is a continuation of my post here 5. I’m trying to deploy BERT for text classification with tensorflow. When I use the model.deploy() method, I can successfully get inferences from BERT. Here’s my problem: I have four different models for classification and I want to run them on the same instance, not multiple instances, to save on cost. So I tried using the MultiDataModel class, but I keep getting the following error:

image979×1189 72.2 KB

The CloudWatch logs don’t add any additional information, unfortunately. Here’s the structure of counterargument.tar.gz in the s3 bucket, which I cloned from my HuggingFace account and zipped.
counterargument.tar.gz
    config.json
    special_tokens_map.json
    tf_model.h5
    tokenizer.json
    tokenizer_config.json
    vocab.txt

The most puzzling thing about this error is that model.deploy() worked fine, but multi_model.deploy() doesn’t! Thanks in advance.