fix "Expected all tensors to be on the same device, but found at least two devices" error

[yao-matrix]

1. when run pytest -rA tests/models/unets/test_models_unet_2d_condition.py::UNet2DConditionModelTests::test_load_sharded_checkpoint_device_map_from_hub_local on 8 devices(CUDA, XPU), there will be a RuntimeError "RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:4 and cuda:1!", fix it by moving to same device
2. enable one gpu-only case on accelerator(XPU test passed)

@sayakpaul , pls help review, thx.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[yao-matrix]

@SunMarc , could you pls help review? Thx very much.

[yao-matrix]

@SunMarc , done thx, pls help review again.

[yao-matrix]

@sayakpaul , pls help retrigger the ci, thx very much.