remove filecheck to enable symlinks

[fschlatt]

Enables streaming from local symlinks #7083

@lhoestq

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[lhoestq]

The CI is failing, looks like it breaks imagefolder loading.

I just checked fsspec internals and maybe instead we can detect symlink by checking islink and size to make sure it's a file

if info["type"] == "file" or (info.get("islink") and info["size"])

[lhoestq]

hmm actually size doesn't seem to filter symlinked directories, we need another way

[fschlatt]

Does fsspec perhaps allow resolving symlinks? Something like https://docs.python.org/3/library/pathlib.html#pathlib.Path.resolve

[lhoestq]

there is info["destination"] in case of a symlink, so maybe

if info["type"] == "file" or (info.get("islink") and info.get("destination") and os.path.isfile(info["destination"]))

[fschlatt]

I've added a fix which works with some temporary test files locally

(info["type"] == "file" or (info.get("islink") and os.path.isfile(os.path.realpath(filepath))))