Hi,
Unfortunately if you do many small operations, it’s going to be fairly slow. The overhead of starting the work on the GPU might be larger than the time you spend actually computing.
You can try to run on the CPU the small part of your model, increase the batch size to give more work to your gpu or find a way to agregate your estimators into a single one that performs one big operation.