Possible Memory Leak?

[robaca]

We are using the d3 module 1.23.1 to create thumbnails for images with code like the following:

this.gm(originalStream, 'someImage.jpg')
    .options({ timeout: 15000 })
    .autoOrient()
    .unsharp(0, 0.5)
    .resize(250, 500, '>')
    .stream('jpg', (err: Error, stdout: any) => {
        if (err) {
            return reject(err)
        }
        const someOutputStream = // create output stream
        stdout
            .pipe(someOutputStream)
            .on('finish', resolve)
            .on('error', reject)
    })

When calling this code multiple times, it seems that d3 stores some buffer references and we have get a memory leak. This is how a diff between two heapsnapshots looks like:

Do we have to do some cleanup to finally free resources?

[hymair]

@robaca did you resolve this? We ran into the same thing.

[robaca]

Hi @hymair we stopped using the module at that time in favor of directly spawning the executable.

[hymair]

Thanks for the quick reply @robaca!

Do you keep the process running once spawned or you exit after each operation? Can you maybe share a code snippet?

[hymair]

I was using this package to convert a pdf to an image. Here's what worked for me by spawning the executable directly:

import { spawn } from 'child_process'

/**
 * Converts a PDF buffer to a JPEG buffer with all pages concatenated vertically
 * @param pdfBuffer - The input PDF buffer
 * @returns Promise resolving to the JPEG buffer
 */
export async function pdfToImage(pdfBuffer: Buffer): Promise<Buffer> {
    return new Promise<Buffer>((resolve, reject) => {
        // Spawn convert process reading from stdin and writing to stdout
        const convert = spawn('convert', [
            '-density',
            '300',
            '-quality',
            '100',
            '-append',
            'pdf:-', // Read from stdin
            'jpeg:-', // Write to stdout
        ])

        const chunks: Buffer[] = []
        let errorOutput = ''

        // Collect output data
        convert.stdout.on('data', (chunk) => {
            chunks.push(Buffer.from(chunk))
        })

        // Collect error messages
        convert.stderr.on('data', (chunk) => {
            errorOutput += chunk.toString()
        })

        // Handle errors spawning the process
        convert.on('error', (error) => {
            reject(
                new Error(`Failed to spawn convert process: ${error.message}`),
            )
        })

        // Handle process completion
        convert.on('exit', (code) => {
            if (code !== 0) {
                reject(
                    new Error(
                        `Convert process failed with code ${code}: ${errorOutput}`,
                    ),
                )
                return
            }
            resolve(Buffer.concat(chunks))
        })

        // Write PDF data to stdin and close it
        convert.stdin.write(pdfBuffer)
        convert.stdin.end()
    })
}

[robaca]

Can't tell much more as it's some time ago and I do not have access to that code any more. But if I remember correctly, we worked on files as we had some issues stdin/stdout in certain situations (like signed PDFs etc).