RedPajama-Data icon indicating copy to clipboard operation
RedPajama-Data copied to clipboard
verified publisher icon togethercomputer

Single machine download script and downloaded files check

Open MIracleyin opened this issue 2 years ago • 0 comments

  1. a script that could download files without slurm.
  2. checking downloaded files if the download failed, saving AWS bandwidth cost.

MIracleyin avatar Jun 16 '23 07:06 MIracleyin