jayvee
jayvee copied to clipboard
[BUG] `LocalFileExtractor` emits wrong error messag if the input file is too big
Steps to reproduce
- Execute this model (
big_file.csvis larger than 2GB):
pipeline Pipeline {
Extractor
-> ToTextFile
-> ToCSV
-> ToTable
-> Loader;
block Extractor oftype LocalFileExtractor {
filePath: "./big_file.csv";
}
block ToTextFile oftype TextFileInterpreter { }
block ToCSV oftype CSVInterpreter {
enclosing: '"';
}
block ToTable oftype TableInterpreter {
header: true;
columns: [
"C1" oftype integer,
"C2" oftype text,
"C3" oftype boolean,
];
}
block Loader oftype SQLiteLoader {
table: "Data";
file: "./Data.sqlite";
}
}
Description
- Expected: The interpreter should fail with the message that the file is too big
- Actual: The interpreter fails with a "file not found" message:
[Pipeline] Overview:
Blocks (5 blocks with 1 pipes):
-> Extractor (LocalFileExtractor)
-> ToTextFile (TextFileInterpreter)
-> ToCSV (CSVInterpreter)
-> ToTable (TableInterpreter)
-> Loader (SQLiteLoader)
error: File './big_file.csv' not found.
$In /home/jonas/Code/uni/hiwi/jayvee/pipeline.jv:14:2
14 | block Extractor oftype LocalFileExtractor {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15 | filePath: "./big_file.csv";
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
16 | }
| ^^^^^