logtron icon indicating copy to clipboard operation
logtron copied to clipboard

logger gets stuck in write

Open strongliang opened this issue 9 years ago • 2 comments

node 0.10.32, cluster mode, disk backend only

logger stops writing after running for sometime. upon inspection of the logger object, the buffer is full.

worker 40> logger.streams.disk
{ _writableState:
   { highWaterMark: 1000,
     objectMode: true,
     needDrain: true,  // normally is false
     ending: false,
     ended: false,
     finished: false,
     decodeStrings: true,
     defaultEncoding: 'utf8',
     length: 1000,
     writing: true,
     corked: 0,
     sync: true,
     bufferProcessing: false,
     onwrite: [Function],
     writecb: [Function],
     writelen: 1,
     buffer:
      [ [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object],
        [Object] ...],  // omit repeated lines
    pendingcb: 1000,
    prefinished: false,
    errorEmitted: false },

a comparison of stuck and healthy logger in the same app (left: stuck; right: healthy) image image

strongliang avatar Nov 10 '16 17:11 strongliang

That means something is wrong with kafka, disk or sentry. Something is not flushing.

Raynos avatar Nov 11 '16 22:11 Raynos

I think there could be some issue with this version of winston. If I use a newer winston (2.3.0) directly, I don't have this issue.

strongliang avatar Nov 12 '16 16:11 strongliang