hdfs3
hdfs3 copied to clipboard
dask
A wrapper for libhdfs3 to interact with HDFS from Python
@jrbourbeau and I are in the process of moving the default branch for this repo from master to main. - [ ] Changed in GitHub - [ ] Merged PR...
Hi, I'm getting a segfault when trying to create a connection using the HDFileSystem constructor. The code that I'm running is: ``` from hdfs3 import HDFileSystem hdfs = HDFileSystem([HOSTNAME], port=[PORT-NUM])...
I want to keep python3.7, How to do it? Step 5/11 : RUN conda install -c conda-forge hdfs3 ---> Running in 0a96e821f949 Solving environment: ...working... done ## Package Plan ##...
Hi, While auditing dependencies I found a particularly nasty [ReDoS](https://www.geeksforgeeks.org/understanding-redos-attack/) issue that is fairly simple to implement where client code is concerned. Versions Affected: hdfs3
Hi, I am trying to install hdfs3 using pip in centos. But when I try to use `HDFileSystem()`, it is throwing `Can not find the shared library: libhdfs3.so` ERROR. Steps...
HDFS does allow append, so could emulate rewrite-from-x (as needed for append by fastparquet) by truncation followed by open for append.
Hi all, what is needed to make this support Windows? I am in a need of a python HDFS client so I would be willing to develop that if it...
Team, After we enabled HDFS wire encryption on secure cluster, hdfs3 cat/put commands are not working. hdfs3 ls commands. i.e. Basically any command that reads/write data from/into HDFS are failing....
My code like below ``` from hdfs3 import HDFileSystem hdfs = HDFileSystem(host=HDFS_HOST, port=HDFS_PORT) df = hdfs.read_stockquantitylogs(input_path) df = process_df(df, process_stock_quantity_log, stack_hour=False) output_path = input_path.replace('/arch', '/clean', 1) hdfs.makedirs(dirname(output_path)) with hdfs.open(output_path, 'wb')...
