logtron
logtron copied to clipboard
logger gets stuck in write
node 0.10.32, cluster mode, disk backend only
logger stops writing after running for sometime. upon inspection of the logger object, the buffer is full.
worker 40> logger.streams.disk
{ _writableState:
{ highWaterMark: 1000,
objectMode: true,
needDrain: true, // normally is false
ending: false,
ended: false,
finished: false,
decodeStrings: true,
defaultEncoding: 'utf8',
length: 1000,
writing: true,
corked: 0,
sync: true,
bufferProcessing: false,
onwrite: [Function],
writecb: [Function],
writelen: 1,
buffer:
[ [Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object],
[Object] ...], // omit repeated lines
pendingcb: 1000,
prefinished: false,
errorEmitted: false },
a comparison of stuck and healthy logger in the same app (left: stuck; right: healthy)

That means something is wrong with kafka, disk or sentry. Something is not flushing.
I think there could be some issue with this version of winston. If I use a newer winston (2.3.0) directly, I don't have this issue.