apache
[Bug]: File metadata transform executes regardless if it has or not input rows
Apache Hop version?
2.13
Java version?
24
Operating system
Windows
What happened?
please look at the attached photo. according to the flow file metadata should not be executed when the file type is not csv or text, but still it throws error even if no hop is wired to it. see below the log lines:
2025/06/27 10:54:33 - LogInitParsing.0 - 2025/06/27 10:54:33 - LogInitParsing.0 - ------------> Linenr 1------------------------------ 2025/06/27 10:54:33 - LogInitParsing.0 - Parsing file details: 2025/06/27 10:54:33 - LogInitParsing.0 - 2025/06/27 10:54:33 - LogInitParsing.0 - filename = C:\dev\dt-generic-hop\workingFolder\decryptedFiles\TESLA_SiteImport_08MAR2025_1023289_144.csv 2025/06/27 10:54:33 - LogInitParsing.0 - short_filename = TESLA_SiteImport_08MAR2025_1023289_144.csv 2025/06/27 10:54:33 - LogInitParsing.0 - path = C:\dev\dt-generic-hop\workingFolder\decryptedFiles 2025/06/27 10:54:33 - LogInitParsing.0 - type = file 2025/06/27 10:54:33 - LogInitParsing.0 - exists = Y 2025/06/27 10:54:33 - LogInitParsing.0 - ishidden = N 2025/06/27 10:54:33 - LogInitParsing.0 - isreadable = Y 2025/06/27 10:54:33 - LogInitParsing.0 - iswriteable = Y 2025/06/27 10:54:33 - LogInitParsing.0 - lastmodifiedtime = 2025/06/24 12:00:34.959 2025/06/27 10:54:33 - LogInitParsing.0 - size = 589 2025/06/27 10:54:33 - LogInitParsing.0 - extension = csv 2025/06/27 10:54:33 - LogInitParsing.0 - uri = file:///C:/dev/dt-generic-hop/workingFolder/decryptedFiles/TESLA_SiteImport_08MAR2025_1023289_144.csv 2025/06/27 10:54:33 - LogInitParsing.0 - rooturi = file:///C:/ 2025/06/27 10:54:33 - LogInitParsing.0 - file_type = unknown 2025/06/27 10:54:33 - LogInitParsing.0 - 2025/06/27 10:54:33 - LogInitParsing.0 - ==================== 2025/06/27 10:54:33 - Get variables.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0) 2025/06/27 10:54:33 - Write to log.0 - 2025/06/27 10:54:33 - Write to log.0 - ------------> Linenr 1------------------------------ 2025/06/27 10:54:33 - Write to log.0 - FileType not supported 2025/06/27 10:54:33 - Write to log.0 - 2025/06/27 10:54:33 - Write to log.0 - filename = C:\dev\dt-generic-hop\workingFolder\decryptedFiles\TESLA_SiteImport_08MAR2025_1023289_144.csv 2025/06/27 10:54:33 - Write to log.0 - path = C:\dev\dt-generic-hop\workingFolder\decryptedFiles 2025/06/27 10:54:33 - Write to log.0 - short_filename = TESLA_SiteImport_08MAR2025_1023289_144.csv 2025/06/27 10:54:33 - Write to log.0 - file_type = unknown 2025/06/27 10:54:33 - Write to log.0 - 2025/06/27 10:54:33 - Write to log.0 - ==================== 2025/06/27 10:54:33 - Abort.0 - ERROR: Row nr 1 causing abort "Stop input processing": [C:\dev\dt-generic-hop\workingFolder\decryptedFiles\TESLA_SiteImport_08MAR2025_1023289_144.csv], [C:\dev\dt-generic-hop\workingFolder\decryptedFiles], [TESLA_SiteImport_08MAR2025_1023289_144.csv], [unknown] 2025/06/27 10:54:33 - Abort.0 - ERROR: Aborting after having seen 1 rows. 2025/06/27 10:54:33 - LogInitParsing.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0) 2025/06/27 10:54:33 - KeepNeccessaryFields.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0) 2025/06/27 10:54:33 - FileTypeFilter.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0) 2025/06/27 10:54:33 - Write to log.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0) 2025/06/27 10:54:33 - Abort.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0) java.lang.NullPointerException: Cannot invoke "org.apache.hop.core.row.IRowMeta.clone()" because the return value of "org.apache.hop.pipeline.transforms.filemetadata.FileMetadata.getInputRowMeta()" is null at org.apache.hop.pipeline.transforms.filemetadata.FileMetadata.processRow(FileMetadata.java:92) at org.apache.hop.pipeline.transform.RunThread.run(RunThread.java:54) at java.base/java.lang.Thread.run(Thread.java:1447) 2025/06/27 10:54:33 - File Metadata.0 - ERROR: Unexpected error 2025/06/27 10:54:33 - File Metadata.0 - ERROR: java.lang.NullPointerException: Cannot invoke "org.apache.hop.core.row.IRowMeta.clone()" because the return value of "org.apache.hop.pipeline.transforms.filemetadata.FileMetadata.getInputRowMeta()" is null 2025/06/27 10:54:33 - File Metadata.0 - at org.apache.hop.pipeline.transforms.filemetadata.FileMetadata.processRow(FileMetadata.java:92) 2025/06/27 10:54:33 - File Metadata.0 - at org.apache.hop.pipeline.transform.RunThread.run(RunThread.java:54) 2025/06/27 10:54:33 - File Metadata.0 - at java.base/java.lang.Thread.run(Thread.java:1447) 2025/06/27 10:54:33 - File Metadata.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=1) 2025/06/27 10:54:33 - IXD_PayloadDemultiplexer_ProcessFile - Pipeline detected one or more transforms with errors. 2025/06/27 10:54:33 - IXD_PayloadDemultiplexer_ProcessFile - Pipeline is killing the other transforms! 2025/06/27 10:54:33 - IXD_PayloadDemultiplexer_ProcessFile - Pipeline duration : 0.401 seconds [ 0.401" ] 2025/06/27 10:54:33 - IXD_PayloadDemultiplexer_ProcessFile - Execution finished on a local pipeline engine with run configuration 'local' 2025/06/27 10:54:33 - IXD_PayloadDemultiplexer_ProcessFile - Starting action [Abort workflow] 2025/06/27 10:54:33 - Abort workflow - ERROR: Seen row nr 3 : Fail Processing File Demultiplexer Workflow 2025/06/27 10:54:33 - IXD_PayloadDemultiplexer_ProcessFile - Stop workflow execution
as you can see from the logs, the file type is = unkown, triggering the write to log and abort action. but still the file metadata transform throws an error, even if no row was received as input.
Issue Priority
Priority: 1
Issue Component
Component: Transforms
based on what I can see from your (partial) logs, your File Metadata transform doesn't file on execution but on pipeline initialization. There's (again) room for improvement in the docs, but in short: when a pipeline starts, all active transforms (transforms that have active hops) are initialized. If your pipeline contains transforms with an incorrect configuration (missing filename, resuling in a NullPointerException), the initialization will fail. This seems to be what happened here.
2025/06/27 10:54:33 - File Metadata.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=1)
i can totally understand that you initialize a transform to be up and running and ready to process data, but it does not receive an input row! one is initialization and one is processing of data.....yet again i have more transforms wired after file metadata, why those do not fail also since they are all initialized at the same time right?
looks like I spoke too soon and you're right. The File Metadata transform indeed throws a NullPointerException if it doesn't receive any rows and gets the filename from a field.