root icon indicating copy to clipboard operation
root copied to clipboard
verified publisher icon root-project

Update hadd docs

Open vepadulano opened this issue 3 years ago • 2 comments

Related to #11245 . The -O option of hadd can be used to force the recompression of the whole target tree (including branches and baskets) to the compression algorithm set with -f option. This is not clear in the current hadd help message. The first commit updates the help for the -O option, most probably this can be further improved so suggestions are welcome!

Other commits are style changes to the message for a clearer output and code.

After this PR:

$: hadd -h
usage: hadd [-a A] [-k K] [-T T] [-O O] [-v V] [-j J] [-dbg DBG] [-d D] [-n N]
            [-cachesize CACHESIZE]
            [-experimental-io-features EXPERIMENTAL_IO_FEATURES] [-f F]
            [-fk FK] [-ff FF] [-f0 F0] [-f6 F6]
            TARGET SOURCES

OPTIONS:
  -a                                   Append to the output
  -k                                   Skip corrupt or non-existent files, do not exit
  -T                                   Do not merge TTree objects
  -O                                   If hadd is used to merge TTree objects, compress the whole target tree
                                       (including its branches and baskets) with the desired compression
                                       algorithm. This can be set for example through the '-f' option
  -v                                   Explicitly set the verbosity level: 0 request no output, 99 is the
                                       default
  -j                                   Parallelize the execution in multiple processes
  -dbg                                 Parallelize the execution in multiple processes in debug mode (Does
                                       not delete partial files stored inside working directory)
  -d                                   Carry out the partial multiprocess execution in the specified
                                       directory
  -n                                   Open at most 'maxopenedfiles' at once (use 0 to request to use the
                                       system maximum)
  -cachesize                           Resize the prefetching cache use to speed up I/O operations (use 0 to
                                       disable)
  -experimental-io-features            Used with an argument provided, enables the corresponding experimental
                                       feature for output trees
  -f                                   Gives the ability to specify the compression level of the target file
                                       (by default 4)
  -fk                                  Sets the target file to contain the baskets with the same compression
                                       as the input files (unless -O is specified). Compresses the meta data
                                       using the compression level specified in the first input or the
                                       compression setting after fk (for example 206 when using -fk206)
  -ff                                  The compression level use is the one specified in the first input
  -f0                                  Do not compress the target file
  -f6                                  Use compression level 6 (See TFile::SetCompressionSettings for the
                                       supported range of values)
  TARGET                               Target file
  SOURCES                              Source files

vepadulano avatar Aug 26 '22 08:08 vepadulano

Starting build on ROOT-debian10-i386/soversion, ROOT-performance-centos8-multicore/cxx17, ROOT-ubuntu18.04/nortcxxmod, ROOT-ubuntu2004/python3, mac1015/cxx17, mac11/cxx14, windows10/cxx14 How to customize builds

phsft-bot avatar Aug 26 '22 08:08 phsft-bot

This pull request fixes 1 alert when merging 3b45969ea9daf0a65a6bc75a9c2b0ba566f7af48 into 0b49293d4d67dfecfe58e710616094b30a90130f - view on LGTM.com

fixed alerts:

  • 1 for Unused import

lgtm-com[bot] avatar Aug 26 '22 08:08 lgtm-com[bot]

Starting build on ROOT-debian10-i386/soversion, ROOT-performance-centos8-multicore/cxx17, ROOT-ubuntu18.04/nortcxxmod, ROOT-ubuntu2004/python3, mac1015/cxx17, mac11/cxx14, windows10/cxx14 How to customize builds

phsft-bot avatar Oct 04 '22 09:10 phsft-bot

Reverted the changes to the help text of O option. Kept other formatting changes, this is the current look of the help output

OPTIONS:
  -a                                   Append to the output
  -k                                   Skip corrupt or non-existent files, do not exit
  -T                                   Do not merge Trees
  -O                                   Re-optimize basket size when merging TTree
  -v                                   Explicitly set the verbosity level: 0 request no output, 99 is the
                                       default
  -j                                   Parallelize the execution in multiple processes
  -dbg                                 Parallelize the execution in multiple processes in debug mode (Does
                                       not delete partial files stored inside working directory)
  -d                                   Carry out the partial multiprocess execution in the specified
                                       directory
  -n                                   Open at most 'maxopenedfiles' at once (use 0 to request to use the
                                       system maximum)
  -cachesize                           Resize the prefetching cache use to speed up I/O operations(use 0 to
                                       disable)
  -experimental-io-features            Used with an argument provided, enables the corresponding experimental
                                       feature for output trees
  -f                                   Gives the ability to specify the compression level of the target file
                                       (by default 4)
  -fk                                  Sets the target file to contain the baskets with the same compression
                                       as the input files (unless -O is specified). Compresses the meta data
                                       using the compression level specified in the first input or the
                                       compression setting after fk (for example 206 when using -fk206)
  -ff                                  The compression level use is the one specified in the first input
  -f0                                  Do not compress the target file
  -f6                                  Use compression level 6 (see TFile::SetCompressionSettings for the
                                       supported range of values)
  TARGET                               Target file
  SOURCES                              Source files

vepadulano avatar Oct 04 '22 09:10 vepadulano

This pull request fixes 1 alert when merging e7b393a71713efe5c56e04cefa8dade989ba7ec1 into 338cea5ec8883c031b9e40bbea650688d2db92f2 - view on LGTM.com

fixed alerts:

  • 1 for Unused import

lgtm-com[bot] avatar Oct 04 '22 10:10 lgtm-com[bot]

Build failed on ROOT-ubuntu18.04/nortcxxmod. Running on sft-ubuntu-1804-2.cern.ch:/build/workspace/root-pullrequests-build See console output.

Failing tests:

And 170 more

phsft-bot avatar Oct 04 '22 10:10 phsft-bot

Build failed on ROOT-debian10-i386/soversion. Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build See console output.

Failing tests:

And 160 more

phsft-bot avatar Oct 04 '22 10:10 phsft-bot

Build failed on ROOT-ubuntu2004/python3. Running on root-ubuntu-2004-3.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.

Failing tests:

And 185 more

phsft-bot avatar Oct 04 '22 10:10 phsft-bot

Build failed on ROOT-performance-centos8-multicore/cxx17. Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build See console output.

Failing tests:

And 361 more

phsft-bot avatar Oct 04 '22 10:10 phsft-bot

Build failed on windows10/cxx14. Running on null:C:\build\workspace\root-pullrequests-build See console output.

Failing tests:

And 144 more

phsft-bot avatar Oct 04 '22 10:10 phsft-bot

Build failed on mac11/cxx14. Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build See console output.

Failing tests:

And 373 more

phsft-bot avatar Oct 04 '22 10:10 phsft-bot

Build failed on mac1015/cxx17. Running on macitois22.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build See console output.

Failing tests:

And 184 more

phsft-bot avatar Oct 04 '22 11:10 phsft-bot

@pcanal can we merge this? Now it only contains formatting changes and fixes a couple of typos. My previous comment shows the new output after this PR.

vepadulano avatar Jun 07 '23 07:06 vepadulano

Starting build on ROOT-debian10-i386/soversion, ROOT-performance-centos8-multicore/cxx17, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac11/noimt, mac12arm/cxx20, windows10/cxx14 How to customize builds

phsft-bot avatar Jun 22 '23 10:06 phsft-bot

Build failed on ROOT-ubuntu2204/nortcxxmod. Running on root-ubuntu-2204-1.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.

Warnings:

  • [2023-06-22T10:06:47.354Z] /home/sftnight/build/workspace/root-pullrequests-build/root/core/foundation/src/FoundationUtils.cxx:116:9: warning: variable ‘parent_path’ set but not used [-Wunused-but-set-variable]
  • [2023-06-22T10:07:36.383Z] /home/sftnight/build/workspace/root-pullrequests-build/root/core/foundation/src/FoundationUtils.cxx:116:9: warning: variable ‘parent_path’ set but not used [-Wunused-but-set-variable]

phsft-bot avatar Jun 22 '23 10:06 phsft-bot

Build failed on ROOT-debian10-i386/soversion. Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build See console output.

Warnings:

  • [2023-06-22T11:16:17.142Z] /home/sftnight/build/workspace/root-pullrequests-build/root/core/foundation/src/FoundationUtils.cxx:116:9: warning: variable ‘parent_path’ set but not used [-Wunused-but-set-variable]
  • [2023-06-22T11:17:14.458Z] /home/sftnight/build/workspace/root-pullrequests-build/root/core/foundation/src/FoundationUtils.cxx:116:9: warning: variable ‘parent_path’ set but not used [-Wunused-but-set-variable]

phsft-bot avatar Jun 22 '23 11:06 phsft-bot

Build failed on ROOT-ubuntu2004/python3. Running on root-ubuntu-2004-1.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.

Warnings:

  • [2023-06-22T11:30:20.277Z] /home/sftnight/build/workspace/root-pullrequests-build/root/core/foundation/src/FoundationUtils.cxx:116:9: warning: variable ‘parent_path’ set but not used [-Wunused-but-set-variable]
  • [2023-06-22T11:31:08.764Z] /home/sftnight/build/workspace/root-pullrequests-build/root/core/foundation/src/FoundationUtils.cxx:116:9: warning: variable ‘parent_path’ set but not used [-Wunused-but-set-variable]

phsft-bot avatar Jun 22 '23 11:06 phsft-bot

Build failed on ROOT-performance-centos8-multicore/cxx17. Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build See console output.

Warnings:

  • [2023-06-22T11:30:50.464Z] /data/sftnight/workspace/root-pullrequests-build/root/core/foundation/src/FoundationUtils.cxx:116:9: warning: variable ‘parent_path’ set but not used [-Wunused-but-set-variable]
  • [2023-06-22T11:37:02.387Z] /data/sftnight/workspace/root-pullrequests-build/root/core/foundation/src/FoundationUtils.cxx:116:9: warning: variable ‘parent_path’ set but not used [-Wunused-but-set-variable]

phsft-bot avatar Jun 22 '23 11:06 phsft-bot

Build failed on mac11/noimt. Running on macphsft23.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build See console output.

Warnings:

  • [2023-06-22T11:39:00.617Z] /Users/sftnight/build/workspace/root-pullrequests-build/root/core/foundation/src/FoundationUtils.cxx:116:9: warning: unused variable 'parent_path' [-Wunused-variable]
  • [2023-06-22T11:40:23.103Z] /Users/sftnight/build/workspace/root-pullrequests-build/root/core/foundation/src/FoundationUtils.cxx:116:9: warning: unused variable 'parent_path' [-Wunused-variable]

phsft-bot avatar Jun 22 '23 12:06 phsft-bot

Build failed on mac12arm/cxx20. Running on macphsft26.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build See console output.

Warnings:

  • [2023-06-22T12:46:36.429Z] /Users/sftnight/build/workspace/root-pullrequests-build/root/core/foundation/src/FoundationUtils.cxx:116:9: warning: unused variable 'parent_path' [-Wunused-variable]
  • [2023-06-22T12:47:17.361Z] /Users/sftnight/build/workspace/root-pullrequests-build/root/core/foundation/src/FoundationUtils.cxx:116:9: warning: unused variable 'parent_path' [-Wunused-variable]

phsft-bot avatar Jun 22 '23 13:06 phsft-bot