dynamometer
dynamometer copied to clipboard
linkedin
A tool for scale and performance testing of HDFS with a specific focus on the NameNode.
Fixes #63 Submitted the PR on behalf of @fangyurao. The PR was adopted from this commit: https://github.com/fangyurao/dynamometer/commit/0f077a6e8094b2154142f09bd034490fbfebfa85
This fixes the issue described in PR #77 regarding improper parsing of the FSImage XML when the CacheManagerSection is present.
After running start-dynamometer-cluster.sh and replay the prod audit log for some time, some simulated datanodes (containers) lost connection to the RM and when the Yarn application is killed, these containers...
Fixes #64 This PR changes blockMap of dyno DN from hash map to tree map. The fix was verified internally at Cloudera. Additional stress test will be needed to understand...
Dynamometer does NOT support negative block id's, which results in blocks with negative id's never being reported by any simulated DataNode. A change has been made to XMLParser.java in our...
In Hadoop 3.0/CDH5.7 and above, HDFS-9260 (Improve the performance and GC friendliness of NameNode startup and full block reports) changed the internal representation of block replicas, as well as the...
The block generation job has custom output logic to allow each reducer to output to multiple block files. When speculative execution is enabled, this can result in two copies of...
The Hadoop tarball to use during testing is specified and downloaded from an Apache mirror. Generally only the latest version of each branch is available, but right now, versions must...
Though the project has some tests, it is essentially just one monolithic integration test. We need more fine-grained unit tests to be able to more easily diagnose issues as they...