lnav
lnav copied to clipboard
tstack
Unexpected log shows up in filter-in mode
lnav version v0.13.2
Describe the bug Error log popping up when I was using filter-in mode, filter by "intelligentReport]", I confirm the error log does not contain the filter string.
To Reproduce Steps to reproduce the behavior:
Monitor a log file, apply one filter-in rule, error log that should not appear pops in.
Are you talking about the log message with the stack trace at the bottom? I think that message is not being recognized by lnav as a separate message, rather it's a part of the previous message from "report-queue-consumer". Note that the text in the brackets is not highlighted like in the other messages. My guess is that some part of the message with the stack trace is not matched by one of the regexes in the java_log format. Would you be able to share the message with the stack trace (anonymized) so that we can adjust the regexes to accept it.
@tstack
- Yes, I was talking about the stack trace, and the trigger row too.
- The error message is part of the first row below, where the first row does not have the filter string I have set, but somehow it's still showing up in lnav.
- The whole error message itself does not contain the string I have set in filter-in mode with "intelligentReport]". The trace takes multiple rows, checked in vscode.
2025-10-24 14:08:14.441 [RMI TCP Connection(26)-127.0.0.1] WARN o.s.b.a.e.ElasticsearchRestClientHealthIndicator [] - Elasticsearch health check failed
org.elasticsearch.client.ResponseException: method [GET], host [http://****:9200], URI [/_cluster/health/], status line [HTTP/1.1 401 Unauthorized]
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/_cluster/health/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication token for REST request [/_cluster/health/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}
at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:347)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:313)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:288)
at org.springframework.boot.actuate.elasticsearch.ElasticsearchRestClientHealthIndicator.doHealthCheck(ElasticsearchRestClientHealthIndicator.java:60)
at org.springframework.boot.actuate.health.AbstractHealthIndicator.health(AbstractHealthIndicator.java:82)
at org.springframework.boot.actuate.health.HealthIndicator.getHealth(HealthIndicator.java:37)
at org.springframework.boot.actuate.health.HealthEndpoint.getHealth(HealthEndpoint.java:94)
at org.springframework.boot.actuate.health.HealthEndpoint.getHealth(HealthEndpoint.java:41)
at org.springframework.boot.actuate.health.HealthEndpointSupport.getLoggedHealth(HealthEndpointSupport.java:172)
at org.springframework.boot.actuate.health.HealthEndpointSupport.getContribution(HealthEndpointSupport.java:145)
at org.springframework.boot.actuate.health.HealthEndpointSupport.getAggregateContribution(HealthEndpointSupport.java:156)
at org.springframework.boot.actuate.health.HealthEndpointSupport.getContribution(HealthEndpointSupport.java:141)
at org.springframework.boot.actuate.health.HealthEndpointSupport.getHealth(HealthEndpointSupport.java:110)
at org.springframework.boot.actuate.health.HealthEndpointSupport.getHealth(HealthEndpointSupport.java:81)
at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:88)
at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:78)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282)
at org.springframework.boot.actuate.endpoint.invoke.reflect.ReflectiveOperationInvoker.invoke(ReflectiveOperationInvoker.java:74)
at org.springframework.boot.actuate.endpoint.annotation.AbstractDiscoveredOperation.invoke(AbstractDiscoveredOperation.java:60)
at org.springframework.boot.actuate.endpoint.jmx.EndpointMBean.invoke(EndpointMBean.java:124)
at org.springframework.boot.actuate.endpoint.jmx.EndpointMBean.invoke(EndpointMBean.java:97)
at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:803)
at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:802)
at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1472)
at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1310)
at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1405)
at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:360)
at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200)
at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:714)
at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:598)
at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:844)
at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:721)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:400)
at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:720)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
2025-10-24 14:08:14.524 [DubboRedisSubscribe] INFO org.apache.dubbo.registry.redis.RedisRegistry [] - [DUBBO] redis event: /dubbo/***.asynctask.rpc.AsyncTaskManager/providers = register, dubbo version: 3.3.0, current host: ******
2025-10-24 14:08:14.524 [master-change-event-executor] INFO org.apache.dubbo.registry.redis.RedisRegistry [] - [DUBBO] Subscribe: provider://****/*****.asynctask.rpc.AsyncTaskManager?anyhost=true&application=all&background=false&bind.ip=****&bind.port=20880&category=configurators&check=false&delay=-1&deprecated=false&dubbo=2.0.2&dynamic=true&executes=100&executor-management-mode=isolation&file-cache=true&generic=false&interface=****.asynctask.rpc.AsyncTaskManager&ipv6=*******&methods=addNewTask&pid=43805&prefer.serialization=lwJackson&qos.check=false&qos.enable=false&release=3.3.0&retries=0&serialization=lwJackson&service-name-mapping=true&service.filter=-authenticationPrepare&side=provider&threads=200&timeout=5000×tamp=1761286094305&weight=200, dubbo version: 3.3.0, current host: *******
2025-10-24 14:08:14.524 [DefaultQuartzScheduler_QuartzSchedulerThread] INFO com.novemberain.quartz.mongodb.dao.SchedulerDao [] - Scheduler instance '{}' not found.
2025-10-24 14:08:14.529 [RMI TCP Connection(26)-127.0.0.1] WARN o.s.boot.actuate.ldap.LdapHealthIndicator [] - LDAP health check failed
org.springframework.ldap.CommunicationException: localhost:389; nested exception is javax.naming.CommunicationException: localhost:389 [Root exception is java.net.ConnectException: Connection refused]
at org.springframework.ldap.support.LdapUtils.convertLdapException(LdapUtils.java:108)
at org.springframework.ldap.core.support.AbstractContextSource.createContext(AbstractContextSource.java:363)
at org.springframework.ldap.core.support.AbstractContextSource.getReadOnlyContext(AbstractContextSource.java:172)
at org.springframework.ldap.core.LdapTemplate.executeReadOnly(LdapTemplate.java:806)
at org.springframework.boot.actuate.ldap.LdapHealthIndicator.doHealthCheck(LdapHealthIndicator.java:50)
at org.springframework.boot.actuate.health.AbstractHealthIndicator.health(AbstractHealthIndicator.java:82)
at org.springframework.boot.actuate.health.HealthIndicator.getHealth(HealthIndicator.java:37)
at org.springframework.boot.actuate.health.HealthEndpoint.getHealth(HealthEndpoint.java:94)
at org.springframework.boot.actuate.health.HealthEndpoint.getHealth(HealthEndpoint.java:41)
at org.springframework.boot.actuate.health.HealthEndpointSupport.getLoggedHealth(HealthEndpointSupport.java:172)
at org.springframework.boot.actuate.health.HealthEndpointSupport.getContribution(HealthEndpointSupport.java:145)
at org.springframework.boot.actuate.health.HealthEndpointSupport.getAggregateContribution(HealthEndpointSupport.java:156)
at org.springframework.boot.actuate.health.HealthEndpointSupport.getContribution(HealthEndpointSupport.java:141)
at org.springframework.boot.actuate.health.HealthEndpointSupport.getHealth(HealthEndpointSupport.java:110)
at org.springframework.boot.actuate.health.HealthEndpointSupport.getHealth(HealthEndpointSupport.java:81)
at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:88)
at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:78)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282)
at org.springframework.boot.actuate.endpoint.invoke.reflect.ReflectiveOperationInvoker.invoke(ReflectiveOperationInvoker.java:74)
at org.springframework.boot.actuate.endpoint.annotation.AbstractDiscoveredOperation.invoke(AbstractDiscoveredOperation.java:60)
at org.springframework.boot.actuate.endpoint.jmx.EndpointMBean.invoke(EndpointMBean.java:124)
at org.springframework.boot.actuate.endpoint.jmx.EndpointMBean.invoke(EndpointMBean.java:97)
at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:803)
at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:802)
at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1472)
at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1310)
at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1405)
at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:360)
at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200)
at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:714)
at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:598)
at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:844)
at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:721)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:400)
at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:720)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: javax.naming.CommunicationException: localhost:389
at java.naming/com.sun.jndi.ldap.Connection.<init>(Connection.java:251)
at java.naming/com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:141)
at java.naming/com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1620)
at java.naming/com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2848)
at java.naming/com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:349)
at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxFromUrl(LdapCtxFactory.java:229)
at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:189)
at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:247)
at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
at java.naming/javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:520)
at java.naming/javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:305)
at java.naming/javax.naming.InitialContext.init(InitialContext.java:236)
at java.naming/javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:154)
at org.springframework.ldap.core.support.LdapContextSource.getDirContextInstance(LdapContextSource.java:42)
at org.springframework.ldap.core.support.AbstractContextSource.createContext(AbstractContextSource.java:351)
... 43 common frames omitted
Caused by: java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.Net.connect0(Native Method)
at java.base/sun.nio.ch.Net.connect(Net.java:589)
at java.base/sun.nio.ch.Net.connect(Net.java:578)
at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:583)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
at java.base/java.net.Socket.connect(Socket.java:751)
at java.base/java.net.Socket.connect(Socket.java:686)
at java.base/java.net.Socket.<init>(Socket.java:555)
at java.base/java.net.Socket.<init>(Socket.java:324)
at java.base/javax.net.DefaultSocketFactory.createSocket(SocketFactory.java:267)
at java.naming/com.sun.jndi.ldap.Connection.createConnectionSocket(Connection.java:350)
at java.naming/com.sun.jndi.ldap.Connection.createSocket(Connection.java:283)
at java.naming/com.sun.jndi.ldap.Connection.<init>(Connection.java:230)
... 58 common frames omitted
It looks like the regex pattern being used is tasko:
https://github.com/tstack/lnav/blob/977b3eba3a81e1ec40448e5edc6ac28749e19a12/src/formats/java_log.json#L14-L16
The thread capture in the pattern isn't matching the message because there is a space and parens in the thread name. You can patch the format by copying the following chunk of json and putting it in the formats/installed directory in your lnav config directory (either ~/.lnav/ or ~/.config/lnav). Call the file java_log_patch.json or something similar. After applying it, the snippet you attached seems to be recognized for me.
{
"$schema": "https://lnav.org/schemas/format-v1.schema.json",
"java_log": {
"title": "Java log format",
"description": "Log format used by log4j and output by most java programs",
"url": "",
"regex": {
"tasko": {
"pattern": "^(?<timestamp>\\d{4}-\\d{2}-\\d{2}[ T]\\d{2}:\\d{2}:\\d{2}(,|\\.)\\d{3}(?:Z|[-+]\\d{2}:?\\d{2})?)\\s+\\[\\s*(?<thread>[\\w ()\\-\\.]+)\\s*\\]\\s+(?<level>ERROR|WARN|INFO|DEBUG|TRACE)\\s+(?<class>[\\w.]+)\\s+(-\\s+)*(?<body>.*)$"
}
}
}
}
@tstack Found a new one, smilar, with filter-in value "\[email\]":
6h59m12s││2025-10-27 18:01:35.353 [email-1] INFO ***.mail.EmailQueueListener [] - [email] Start to listen general email redis queue │
6h59m12s││2025-10-27 18:01:35.353 [email-2] INFO ***.mail.EmailQueueListener [] - [email] Start to listen alert email redis queue │
6h59m42s││2025-10-27 18:02:05.797 [email-1] INFO ***.mail.EmailQueueListener [] - [email] send email: from ***@***, to [***], subject: *** │
6h59m44s││2025-10-27 18:02:07.247 [email-1] ERROR ***.mail.EmailQueueListener [] - [email] send email to [***] failed │
6h59m44s││org.springframework.mail.MailAuthenticationException: Authentication failed; nested exception is javax.mail.AuthenticationFailedException: 535 Error: authentication failed │
6h59m44s││ │
6h59m44s││ at org.springframework.mail.javamail.JavaMailSenderImpl.doSend(JavaMailSenderImpl.java:440) │
6h59m44s││ at org.springframework.mail.javamail.JavaMailSenderImpl.send(JavaMailSenderImpl.java:361) │
6h59m44s││ at org.springframework.mail.javamail.JavaMailSenderImpl.send(JavaMailSenderImpl.java:356) │
6h59m44s││ at ***.mail.EmailQueueListener.dealWrapper(EmailQueueListener.java:150) │
6h59m44s││ at ***.mail.EmailQueueListener.listenQueue(EmailQueueListener.java:76) │
6h59m44s││ at ***.mail.EmailQueueListener.lambda$init$0(EmailQueueListener.java:65) │
6h59m44s││ at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) │
6h59m44s││ at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) │
6h59m44s││ at java.base/java.lang.Thread.run(Thread.java:1583) │
6h59m44s││Caused by: javax.mail.AuthenticationFailedException: 535 Error: authentication failed │
6h59m44s││ │
6h59m44s││ at com.sun.mail.smtp.SMTPTransport$Authenticator.authenticate(SMTPTransport.java:947) │
6h59m44s││ at com.sun.mail.smtp.SMTPTransport.authenticate(SMTPTransport.java:858) │
6h59m44s││ at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:762) │
6h59m44s││ at javax.mail.Service.connect(Service.java:342) │
6h59m44s││ at org.springframework.mail.javamail.JavaMailSenderImpl.connectTransport(JavaMailSenderImpl.java:518) │
6h59m44s││ at org.springframework.mail.javamail.JavaMailSenderImpl.doSend(JavaMailSenderImpl.java:437) │
6h59m44s││ ... 8 common frames omitted │
7h11m35s││2025-10-27 18:13:59.067 [email-1] INFO ***.mail.EmailQueueListener [] - [email] Start to listen general email redis queue │
7h11m35s││2025-10-27 18:13:59.067 [email-2] INFO ***.mail.EmailQueueListener [] - [email] Start to listen alert email redis queue │
7h12m02s││2025-10-27 18:14:25.613 [email-1] INFO ***.mail.EmailQueueListener [] - [email] send email: from ***@***, to [***], subject: 【*** │
▌ Line 558 parser details. Press p to hide this panel. Press CTRL-] to focus on this panel │
Received Time: 2025-10-27T18:14:25.613+0800 — 2 minutes and 25 seconds ago Format: %Y-%m-%d %H:%M:%S.%L Default Zone: none │
Permalink: #*** │
Pattern: /java_log/regex/tasko = ^(?<timestamp>\d{4}-\d{2}-\d{2}[ T]\d{2}:\d{2}:\d{2}(,|\.)\d{3}(?:Z|[-+]\d{2}:?\d{2})?)\s+\[\s*(?<thread>[\w ()\-\.]+)\s*\]\s+(?<level>ERROR|WARN|INFO|DEBUG|TRACE)\s+(?<class>[\w.]+)\s+(-\s│
Known message fields for table java_log: │
├ ◆ 📊 thread = email-1 │
├ ◆ 📊 level = INFO │
├ ◆ 📊 class = ***.mail.EmailQueueListener │
Discovered fields for logline table from message format: [#] # [#] # email: #, to [#], subject: #, isAlertEmail: # │
├ col_0 = │
├ col_1 = - │
├ col_2 = email │
├ col_3 = send │
├ email = from ***@*** │
├ to_0 = *** │
├ subject = *** │
└ isAlertEmail = false
@tstack Update: It's kinda good to have the error messages showing up in the filter-in mode even though it's not matched by filter-in regex. Cause then you know the error linked with a line that matches the regex. I would consider this as a feature, not a bug lmao