LAPIS icon indicating copy to clipboard operation
LAPIS copied to clipboard

Check why grep shows many lines not containing the searched term

Open JonasKellerer opened this issue 1 year ago • 2 comments

When running docker logs lapis-lapis-open-1 | grep "9e45aa8e-3bdf-4ff0-a4b7-03eec0419b18" the output can look like this:

Picked up JAVA_TOOL_OPTIONS: -Djava.security.properties=/layers/paketo-buildpacks_bellsoft-liberica/java-security-properties/java-security.properties -XX:+ExitOnOutOfMemoryError -XX:ActiveProcessorCount=64 -XX:MaxDirectMemorySize=10M -Xmx318690624K -XX:MaxMetaspaceSize=98983K -XX:ReservedCodeCacheSize=240M -Xss1M -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summary -XX:+PrintNMTStatistics -Dorg.springframework.cloud.bindings.boot.enable=true line 1:25 mismatched input '.' expecting {<EOF>, '&', '|'} line 1:25 mismatched input '.' expecting {<EOF>, '&', '|'} line 1:25 mismatched input '.' expecting {<EOF>, '&', '|'} line 1:26 mismatched input '.' expecting {'&', '|', ')'} line 1:26 mismatched input '.' expecting {'&', '|', ')'} line 1:25 mismatched input '.' expecting {<EOF>, '&', '|'} line 1:25 mismatched input '.' expecting {<EOF>, '&', '|'} line 1:25 mismatched input '.' expecting {<EOF>, '&', '|'} line 1:151 extraneous input '&' expecting {'!', '(', 'MAYBE(', '[', ORF, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, NUMBER} line 1:151 extraneous input '&' expecting {'!', '(', 'MAYBE(', '[', ORF, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, NUMBER} line 1:152 extraneous input '&' expecting {'!', '(', 'MAYBE(', '[', ORF, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, NUMBER} line 1:3 mismatched input 'F' expecting NUMBER line 1:3 mismatched input 'F' expecting NUMBER line 1:3 mismatched input 'F' expecting NUMBER line 1:3 mismatched input 'F' expecting NUMBER line 1:4 mismatched input 'R' expecting {<EOF>, '&', '|'} line 1:4 mismatched input 'R' expecting {<EOF>, '&', '|'} line 1:4 mismatched input 'R' expecting {<EOF>, '&', '|'} line 1:4 mismatched input 'R' expecting {<EOF>, '&', '|'} line 1:3 mismatched input 'F' expecting NUMBER line 1:4 mismatched input 'R' expecting {<EOF>, '&', '|'} line 1:3 mismatched input 'F' expecting NUMBER line 1:4 mismatched input 'R' expecting {<EOF>, '&', '|'} line 1:3 mismatched input 'F' expecting NUMBER line 1:4 mismatched input 'R' expecting {<EOF>, '&', '|'} line 1:3 mismatched input 'F' expecting NUMBER line 1:4 mismatched input 'R' expecting {<EOF>, '&', '|'} line 1:3 mismatched input 'F' expecting NUMBER line 1:4 mismatched input 'R' expecting {<EOF>, '&', '|'} line 1:3 mismatched input 'F' expecting NUMBER line 1:4 mismatched input 'R' expecting {<EOF>, '&', '|'} 2024-03-05 08:43:33,095 INFO [http-nio-8080-exec-13313] [9e45aa8e-3bdf-4ff0-a4b7-03eec0419b18] org.genspectrum.lapis.silo.CachedSiloClient: Calling SILO: {"action":{"groupByFields":["date"],"type":"Aggregated"},"filterExpression":{"children":[{"children":[{"column":"country","value":"Germany","type":"StringEquals"}],"type":"Or"},{"column":"date","from":"2023-12-04","to":"2024-02-27","type":"DateBetween"},{"children":[{"column":"nextcladePangoLineage","value":"DV.7.1","includeSublineages":true,"type":"PangoLineage"}],"type":"Or"},{"children":[{"column":"host","value":"Human","type":"StringEquals"}],"type":"Or"}],"type":"And"}} 2024-03-05 08:43:33,104 INFO [http-nio-8080-exec-13313] [9e45aa8e-3bdf-4ff0-a4b7-03eec0419b18] org.genspectrum.lapis.silo.CachedSiloClient: Response from SILO: 200 2024-03-05 08:43:33,104 INFO [http-nio-8080-exec-13313] [9e45aa8e-3bdf-4ff0-a4b7-03eec0419b18] org.genspectrum.lapis.logging.RequestContextLogger: {"unixTimestamp":1709628213095,"responseTimeInMilliSeconds":9,"endpoint":"/gisaid/v2/sample/aggregated","filter":{"sequenceFilters":{"country":["Germany"],"dateFrom":["2023-12-04"],"dateTo":["2024-02-27"],"nextcladePangoLineage":["DV.7.1*"],"host":["Human"]},"nucleotideMutations":[],"aaMutations":[],"nucleotideInsertions":[],"aminoAcidInsertions":[],"fields":[{"fieldName":"date"}],"orderByFields":[],"isEmpty":false},"responseCode":200}

If no result is found by grep, then this shows lines without the searched term.

Check why this is the case and also clean up the logs. A grep of a request id should always contain the request id.

JonasKellerer avatar Mar 05 '24 09:03 JonasKellerer

Mayne it's because of the error logs: https://stackoverflow.com/a/38207098

Try 2>&1

chaoran-chen avatar Mar 05 '24 09:03 chaoran-chen

Most probably yes. But it's still weird that LAPIS writes something to stderr.

fengelniederhammer avatar Mar 05 '24 09:03 fengelniederhammer