lnav icon indicating copy to clipboard operation
lnav copied to clipboard

Rewriting nested fields not working as expected

Open lheinold opened this issue 7 months ago • 0 comments

lnav version head of tree (master) built from source - 0.13.0-rc6-2-gfd9467f-dirty

Describe the bug Rewriting a field within another rewritten field is not working as expected.

For example, I have this log line: (filename: wirelog.log

Wed Jul 09 04:53:15.446	RX: 8=FIX.4.4|9=90|35=A|34=1|49=SENDER|56=TARGET|52=20250709-20:53:15.445|98=0|108=90|141=Y|553=RED|554=PASS|10=137|

This message has the following fields:

 ├ ◆ 📊 direction = RX
 ├ ◆ 📊 body_msg  = 8=FIX.4.4|9=90|35=A|34=1|49=SENDER|56=TARGET|52=20250709-20:53:15.445|98=0|108=90|141=Y|553=RED|554=PASS|10=137|
 ├ ◆ 📊 msg_type  = A

I am rewriting both body_msg and msg_type. The rewriter forbody_msg adds a new line before every |, and the msg_type rewriter replaces the msg_type character with its string representation (A=Logon). Each rewriter works fine by itself.

Expected behavior:

Wed Jul 09 04:53:15.446	RX: 8=FIX.4.4
|9=90
|35=Logon
|34=1
|49=SENDER
|56=TARGET
|52=20250709-20:53:15.445
|98=0
|108=90
|141=Y
|553=RED
|554=PASS
|10=137
|

Actual behavior:

Wed Jul 09 04:53:15.446	RX: 8=FIX.4.4
|9=90
|3LogonSENDER
|56=TARGET
|52=20250709-20:53:15.445
|98=0
|108=90
|141=Y
|553=RED
|554=PASS
|10=137
|

To Reproduce Log format json:

{
    "$schema": "https://lnav.org/schemas/format-v1.schema.json",
    "wirelog" : {
        "title" : "wirelog",
        "file-pattern" : ".*wirelog",
        "regex" : {
            "with_time" : {
                "pattern" : "^(?<timestamp>\\S{3} \\S{3} .{2} \\d{2}:\\d{2}:\\d{2}\\.\\d{3}).(?<direction>\\S{2}): (?<body_msg>.*35=(?<msg_type>\\S).*|.*)$"
             }
        },
        "value" : {
            "msg_type" : {
                "kind": "string",
                "identifier" : false,
                "rewriter": ";SELECT Type from MsgType WHERE TagValue=:msg_type"
            },
            "body_msg" : {
                "kind" : "string",
                "identifier" : false,
                "rewriter" : ";SELECT replace(:body_msg, '|', '\n|')"
            },
            "direction": {
                "kind": "string",
                "identifier": true
            }
        },
        "timestamp-format" : [ "%a %b %e %H:%M:%S.%L", "%a %b  %e %H:%M:%S.%L" ],
        "sample" : [
            {
                "line" : "Wed Sep 22 09:49:47.569 TX: 8=FIX.4.2|"
            }
        ]
    }
}

msgtype.sql: (inside format directory)

CREATE TABLE MsgType (
    TagValue varchar(2),
    Type varchar(255)
 );

 INSERT INTO MsgType(TagValue, Type)
 VALUES
('0', 'Heartbeat'),
('1', 'TestRequest'),
('2', 'ResendRequest'),
('3', 'Reject'),
('4', 'SequenceReset'),
('5', 'Logout'),
('A', 'Logon');

lheinold avatar Jul 11 '25 19:07 lheinold