dolphinscheduler icon indicating copy to clipboard operation
dolphinscheduler copied to clipboard

[Bug] [dolphinscheduler-storage-hdfs] When you create a new task, the resource file cannot be displayed hierarchically

Open JohnZp opened this issue 1 year ago • 1 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

image When creating a new task, the resource file cannot be displayed in a hierarchical manner, making it difficult to distinguish which file it is

What you expected to happen

image Ability to display files hierarchically

How to reproduce

IN HdfsStorageOperator,

    public List<StorageEntity> listFileStorageEntityRecursively(String resourceAbsolutePath) {
        exceptionIfPathEmpty(resourceAbsolutePath);

        List<StorageEntity> result = new ArrayList<>();

        LinkedList<String> foldersToFetch = new LinkedList<>();
        foldersToFetch.addLast(resourceAbsolutePath);

        while (!foldersToFetch.isEmpty()) {
            String absolutePath = foldersToFetch.pollFirst();
            RemoteIterator<LocatedFileStatus> remoteIterator = fs.listFiles(new Path(absolutePath), true);
            while (remoteIterator.hasNext()) {
                LocatedFileStatus locatedFileStatus = remoteIterator.next();
                result.add(transformFileStatusToResourceMetadata(locatedFileStatus));
            }
        }
        return result;
    }

Anything else

No response

Version

dev

Are you willing to submit PR?

  • [X] Yes I am willing to submit a PR!

Code of Conduct

JohnZp avatar Jul 17 '24 01:07 JohnZp

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] avatar Aug 20 '24 00:08 github-actions[bot]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

github-actions[bot] avatar Sep 07 '24 00:09 github-actions[bot]