Question: custom format introducing blank lines between log entries

[oswcab]

Hi. Using lnav to process and read through custom logs produced by our product. Logs are JSON lines format and looks like:

{"#node":"node-1","sev":"INFO","host":"node-1","pid":"2668","cont":"registry","msg":"2602:807:900e:d5::2 - - [10/Apr/2022:20:17:03 +0000] \"PATCH /v2/openshift/blobs/uploads/a0eb5c54-602c-4b55-bfe1-715f62fc7046?_state=aEUsD5xSCKh-o7KFx_htZ8Aasc2izj5tC3XQ17IzNuR7Ik5hbWUiOiJvcGVuc2hpZnQiLCJVVUlEIjoiYTBlYjVjNTQtNjAyYy00YjU1LWJmZTEtNzE1ZjYyZmM3MDQ2IiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDIyLTA0LTEwVDIwOjE3OjAzLjA0NTY3NjMyN1oifQ%3D%3D HTTP/2.0\" 202 0 \"\" \"Go-http-client/2.0\"\n","proc":"conmon","#ts":"2022-04-10T20:17:25.912357+00:00","#src":"logger.systemd","#fabricID":"'0x0309'","#mgmtIP":"''"}
...

The custom format we created is this:

{
    "$schema": "https://lnav.org/schemas/format-v1.schema.json",
    "ksdf_fabric_log": {
        "title": "KSDF fabric JSON log format",
        "description": "Logger format as created by the logging pipeline inside a KSDF fabric",
        "url": "TODO: Probably a page in Confluence",
        "json": true,
        "hide-extra": true,
        "line-format": [
            {
                "field": "#ts"
            },
            " ",
            {
                "field": "#node"
            },
            " ",
            {
                "field": "host"
            },
            " ",
            {
                "field": "#fabricID"
            },
            " ",
            {
                "field": "proc"
            },
            " ",
            {
                "field": "pid"
            },
            " ",
            {
                "field": "sev",
                "min-width": 7
            },
            " ",
            {
                "field": "cont"
            },
            " ",
            {
                "field": "msg"
            }
        ],
        "timestamp-field": "#ts",
        "ordered-by-time": true,
        "level-field": "sev",
        "level": {
            "error": "ERR",
            "warning": "WARNING",
            "info": "INFO",
            "debug": "DEBUG"
        },
        "body-field": "msg",
        "opid-field": "pid",
        "value": {
            "#ts": {
                "kind": "string"
            },
            "#node": {
                "kind": "string",
                "identifier": true
            },
            "host": {
                "kind": "string",
                "identifier": true
            },
            "#fabricID": {
                "kind": "string",
                "identifier": true
            },
            "proc": {
                "kind": "string",
                "identifier": true
            },
            "pid": {
                "kind": "string",
                "identifier": true
            },
            "cont": {
                "kind": "string",
                "identifier": true
            },
            "sev": {
                "kind": "string"
            },
            "msg": {
                "kind": "string"
            }
        }
    }
}

Which works well. The only issue we noticed is that blank lines between log entries are shown in lnav when the format is applied. So far, looking at the documentation and googling for this has not given any answer as to why. Could anyone with more experience please take a quick look to the format code to see if the reason is evident? Thanks!

[tstack]

It looks like the msg field ends with a line-feed, so that's where the extra line is coming from. I'll need to make some changes to allow the line to be trimmed.

[oswcab]

Makes perfect sense. Thanks for the answer!

[tstack]

I've pushed a change to automatically strip line feeds, hopefully that works for you.