graphql-php icon indicating copy to clipboard operation
graphql-php copied to clipboard

Using Deferred produces memory leak?

Open Danbka opened this issue 4 years ago • 7 comments

I'm using Deferred to avoid N+1 problem, but with a big amount of entities, it produces out of memory error.

Even without buffer it allocates a lot of memory:

'resolve' => function($blogStory) {
    return new GraphQL\Deferred(function () {
        return true;
    });
}

Is there a way to use it more properly?

Danbka avatar Oct 10 '21 20:10 Danbka

Can you do a minimal reproduction of this and do some actual measurements?

spawnia avatar Oct 11 '21 07:10 spawnia

Yes, there is a synthetic example without DB and buffers:

$authors = [];
for ($i = 0; $i <= 100; $i++) {
    $authors[$i] = ['name' => 'Name ' . $i];
}

$books = [];
for ($i = 0; $i <= 4000; $i++) {
    $books[$i] = ['title' => 'Title ' . $i, 'authorId' => rand(0, 100)];
}

$authorType = new Type\Definition\ObjectType([
    'name' => 'Author',
    'fields' => [
        'name' => [
            'type' => Type\Definition\Type::string(),
            'resolve' => function ($rootValue, $args) {
                return $rootValue['name'];
            }
        ],
    ],
]);

$bookType = new Type\Definition\ObjectType([
    'name' => 'Book',
    'fields' => [
        'title' => [
            'type' => Type\Definition\Type::string(),
            'resolve' => function ($rootValue, $args) {
                return $rootValue['title'];
            }
        ],
        'author' => [
            'type' => $authorType,
            'resolve' => function ($rootValue, $args) use ($authors) {
//                        return new Deferred(function() use ($authors, $rootValue) {
//                            return $authors[$rootValue['authorId']];
//                        });
                return $authors[$rootValue['authorId']];
            }
        ],
    ],
]);

$queryType = new Type\Definition\ObjectType([
    'name' => 'Query',
    'fields' => [
        'getBooks' => [
            'type' => Type\Definition\Type::listOf($bookType),
            'resolve' => function ($rootValue, $args) use ($books) {
                return $books;
            }
        ],
    ],
]);

$schema = new Type\Schema([
    'query' => $queryType
]);

$data = $request->getParsedBody() ?? [];
$data += ['query' => null, 'variables' => null];

$result = \GraphQL\GraphQL::executeQuery(
    $schema,
    $data['query'],
    null,
    $this->container,
    $data['variables']
);

\TGLog::getInstance(\TGLog::NS_API)->debug((string)(int)(memory_get_peak_usage(true) / 1024 / 1024));

It allocates around 4Mb memory.

But when I start using Deferred (line 35) it allocates around 60Mb memory.

Danbka avatar Oct 11 '21 13:10 Danbka

@spawnia Hi there, I added the minimum reproduction: https://github.com/Danbka/php-graphql-deffred Can you have a look please?

Danbka avatar Jan 28 '22 14:01 Danbka

I'm experiencing similar issues. However when I perform the exact same query there's a random chance of about 50% to get an out of memory error. The other 50% my query executes just fine. Also everything runs fine without Deferred but then the response time is unacceptibly high which is to be expected.

laurooyen avatar Jan 31 '22 09:01 laurooyen

@laurooyen let me know please if you find a solution.

Danbka avatar Jan 31 '22 09:01 Danbka

@spawnia Hi there, I added the minimum reproduction: https://github.com/Danbka/php-graphql-deffred Can you have a look please?

Any news on the topic?

Danbka avatar Jan 04 '23 09:01 Danbka