I’m building a distributed parameter/server type architecture and wanting to communicate model updates through table solutions on Azure.
I’m having a hard time finding any useful information about saving a models state_dict into a redis cache. I’ve given up on Azure Cosmos tables because of the size limit (64kb) per entity and looked toward redis since model state_dict params/weights are much larger, even for a small model.
Does anyone have any recommendations for me on how to pursue this?