I think that as a first step we could technically evaluate what we need to do at dependecies level in bazel to compile this target with a new option:
bazel build //tensorflow/python:all
without triggering TF c++ targets compilation.