[bnb optim] fixing test
**Note to reviewers: we are dealing with a slow test, so a green CI doesn't tell anything.**

This work is a continuation of https://github.com/huggingface/transformers/pull/21019 where this test failure was first reported by @ydshieh 

This PR:

- extends/improves the `run_trainer` wrapper which simplifies the bnb test
- drops the percentage-based asserts as those are quite meaningless - since they don't measure the memory used by the optimizer but the whole memory - it replaces them with the actual calculated saved memory expectation since we know exactly what the saved memory should be for a particular model. it's `6*params` bytes but not for `nn.Embedding` which gets fp32 - so let's measure that carefully. 

https://github.com/huggingface/transformers/blob/35a7052b61579cfe8df1a059d4cd3359310ec2d1/src/transformers/trainer.py#L1042-L1050

- drops the peak gpu memory comparison since on its own it's totally meaningless, in my testing I get both optims produce the same peak memory - what we care about is the total gpu memory.
- forces 1 gpu - so that the gpu memory usage is the same in all environments, to support 2+ gpus we need to have a different threshold for each - except this is totally unncessary
- switches to MBs everywhere, so it's much easier to debug 

Now, the test should be very deterministic on any gpu/platform. I still gave a small margin for differences.

You can read my notes in the test for the exact math.

@ydshieh. please verify it works on the CI and we will then merge it. Thank you!
_The documentation is not available anymore as the PR was closed or merged._<|||||>The new test now passes on CI runners! I will review the change and thank you @stas00 ❤️ !