I have a GRU model and the depth of my model is limited by my GPU’s memory. Would having two of the same GPU’s allow for twice the depth? Could I also use my SSD or RAM as memory instead (without losing GPU processing)?
In case it is case specific; I have a 2-layer GRU model with 1000 inputs and 500 hidden units (thats my current limit) and would like to increase it to 1000 hidden units. Also I’m aware that reducing batch size would help, but due to class imbalance I can’t reduce it.