I just wanted to share something I’ve been working on. It’s in the very early stages, but I was hoping to get some assistance in building it up - I think it could be really useful.
I realize that a full OpenCL port would be a huge amount of work (and I would like to be able to use models on OpenCL devices), so I decided to take matters into my own hands. This project allows you to train up a network in pytorch, then save each of the tensors (for now, individually but I’d love to change that) for weights/filters/bias and then load them into ArrayFire. Using ArrayFire as our OpenCL library, we then perform the forwards pass as usual. The easiest way I’ve found to go python -> C++ is through numpy and using their API. I have to do some index gymnastics because of the ArrayFire conventions, but so far I can initialize tensors for Conv2d and Linear layers in python, save them, load them into C++ and perform an inference. I realize it’s more cumbersome and not ideal for training and development, but it’s meant as a tool for deployment.
I would love to have help from people who know ArrayFire/pytorch better than I do - the next layers to do are pooling layers (maxpool, avgpool) and batchnorm (especially batchnorm). Help/suggestions for optimizing ArrayFire code (I’m a total newb with AF) would be awesome as well.
Project link: pytorch-inference 115
Please be gentle, I’ve only been working on it for about a day at this point so it’s still pretty rough.