@ntp Hey,
I am facing the same issue here. I have a new laptop with rtx2070 (not dowgraded maxQ )version I alsa see almost the same vram usage. I tried using the allow_growth option on tensorflow and more often than any of my other cards I see lots of allocation failure warnings on the console. My cards vram is not being used. It even has optimus so it should not even be touched by any other programs normally.
Where are you on the problem? Did you solve it?