Fix XGLM OOM on CI
# What does this PR do?

same as in #24122 and #24106 
_The documentation is not available anymore as the PR was closed or merged._<|||||>Going to merge as this is just the same as the other PRs. Don't need to bother too much the core maintainers.