text: Title: CI: more extensive tests that optional dependencies are optional
  Body: Currently we only test optional packages are not needed to `import cleanlab` on CI server before it has installed requirements-dev.txt:  https://github.com/cleanlab/cleanlab/blob/10abd7d1910b7db237203b0533e07d9f8dd91fe0/.github/workflows/ci.yml#L30
  
  Ideally, there would be more involved testing at this point (before the optional packages like tensorflow,pytorch,etc have been installed), eg. some subset of the fast-running unit tests that are known to not require these optional packages. 