Any way to reduce the model size to <250mb?
Google Cloud's online prediction service has a 250mb limit for uploaded models. I don't think I have ever seen a BERT model that small. Casting all tensors to half precision reduces the model to ~350mb, is there any way to go even further than that?
Probably not - it would certainly be possible to make a smaller BERT model that would fit into this size, but all of the available pre-trained models have too many parameters, so you'd have to train it from scratch (which is very slow, and isn't something this repo supports yet).<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
