lnav icon indicating copy to clipboard operation
lnav copied to clipboard

Rewriter for dynamically discovered fields?

Open lheinold opened this issue 7 months ago • 0 comments

I've got a log where each line is a FIX message. FIX messages have the format "=|=|...|=" with an arbitrary number of tags. Each tag is a number with a name that can be looked up (ex. tag 8 is version, tag 9 is message length, tag 35 is message type). I'd like to rewrite each tag to its associated English name.

For example:

Thu Jul 10 11:52:05.809	RX: 8=FIX.4.2|9=73|35=A|...

would be rewritten to

Thu Jul 10 11:52:05.809	RX: Version=FIX.4.2|Length=73|MsgType=A|...

lnav is already dynamically discovering the fields I want to rewrite:

 Discovered fields for logline table from message format:       #: #=FIX.4.2|#=#|#=A| 
├ col_0 = RX                                                                                                                                                                                               
├ col_1 = 8                                                                                                                                                                                                
├ col_2 = 9                                                                                                                                                                                                
├ col_3 = 73                                                                                                                                                                                               
├ col_4 = 35
...

I know I can have a rewriter that does this for specific fields, but apparently you can't create a regex with a dynamic number of repeated capturing groups. (FIX has hundreds of tags so manually defining a field for each tag in the regex isn't really an option)

I'd like to do something like:

for each column in logline:
    ;SELECT FieldName FROM Tags WHERE FieldTag=:column IF isnumeric(:column)

I'm aware that might pick up on some non tags (ex. it would get the 73 from 9=73), but that's fine.

Is there any way to do this, either with a format file or with a command inside lnav itself?

lheinold avatar Jul 10 '25 18:07 lheinold