Patches the full import failure and adds a test
The full import currently fails because some layers are imported when they do not exist.

This adds a test in `test_file_utils.py` by trying to import the entire transformers. This failed before the proposed fix.

Fixes https://github.com/huggingface/transformers/issues/10749