[Test] Add tests that no imports have been added that are not a hard dependency

[patrickvonplaten]

As seen in this PR: https://github.com/huggingface/diffusers/pull/2193 our tests can all pass even if dependencies are added to PRs that are no hard dependencies. The reason for this is that we use torchvision for example testing and thus have it in our docker container, but it's not a hard dependency and therefore at least one test should fail.

=> We should add a new test that verifies this.

Just putting up this issue as a reminder cc @williamberman @patil-suraj @pcuenca