dolphinscheduler
dolphinscheduler copied to clipboard
[Bug] [dolphinscheduler-storage-hdfs] When you create a new task, the resource file cannot be displayed hierarchically
Search before asking
- [X] I had searched in the issues and found no similar issues.
What happened
When creating a new task, the resource file cannot be displayed in a hierarchical manner, making it difficult to distinguish which file it is
What you expected to happen
Ability to display files hierarchically
How to reproduce
IN HdfsStorageOperator,
public List<StorageEntity> listFileStorageEntityRecursively(String resourceAbsolutePath) {
exceptionIfPathEmpty(resourceAbsolutePath);
List<StorageEntity> result = new ArrayList<>();
LinkedList<String> foldersToFetch = new LinkedList<>();
foldersToFetch.addLast(resourceAbsolutePath);
while (!foldersToFetch.isEmpty()) {
String absolutePath = foldersToFetch.pollFirst();
RemoteIterator<LocatedFileStatus> remoteIterator = fs.listFiles(new Path(absolutePath), true);
while (remoteIterator.hasNext()) {
LocatedFileStatus locatedFileStatus = remoteIterator.next();
result.add(transformFileStatusToResourceMetadata(locatedFileStatus));
}
}
return result;
}
Anything else
No response
Version
dev
Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.