Performance degradation when upgrading from Mongoose v4 to v8 (findOne slower than find)

[0Ams]

Prerequisites

• [X] I have written a descriptive issue title

Mongoose version

8.6.3

Node.js version

v18.20.4

MongoDB version

4.4.25

Operating system

Linux

Operating system version (i.e. 20.04, 11.3, 10)

No response

Issue

Description:

I am currently in the process of upgrading Mongoose from version 4 to version 8 in our project, and I've encountered a significant performance issue.

Summary:

• v4 performance: Acceptable speeds.
• v8 performance: Approximately 40% slower for findOne queries, despite find queries operating as expected (and even faster than before).

Details:

• Indexes and schema remain identical between v4 and v8.
• No special configurations or optimizations were applied during the upgrade.
• find queries exhibit improved performance under v8, but findOne queries are noticeably slower.
• same setting & same environment & same index

Steps to Reproduce:

• Run the same query using findOne on both Mongoose v4 and v8.
• Compare the query performance between the two versions.

Expected Behavior:

• Performance should be on par or improved in v8, compared to v4.

Actual Behavior:

• findOne queries are approximately 30% slower in v8 than in v4.

[IslandRhythms]

Mongoose version 4.13.21 1.938ms

const mongoose = require('mongoose');


const testSchema = new mongoose.Schema({
    name: String,
    val: Number
});


const Test = mongoose.model('Test', testSchema);

async function main() {
    for (let i = 0; i < 100; i++) {
        const doc = new Test();
        doc.name = 'Test',
        doc.val = i;
        doc.save(function(err) {

        });
    }
}

async function run() {
    Test.findOne({}, function (err, doc) {
        console.log('doc', doc);
    });
}

(async () => {
    await mongoose.connect('mongodb://localhost:27017/performance-test');
    // await mongoose.connection.dropDatabase();
    await main();
    console.time('performanceTest')
    await run();
    console.timeEnd('performanceTest');
    console.log(mongoose.version)
    process.exit(0);
})();

Mongoose version 8.7.0 10.779ms

const mongoose = require('mongoose');


const testSchema = new mongoose.Schema({
    name: String,
    val: Number
});


const Test = mongoose.model('Test', testSchema);

async function main() {
    for (let i = 0; i < 100; i++) {
        await Test.create({
            name: 'Test',
            val: i
        })
    }
}

async function run() {
    const res = await Test.findOne();
    console.log(res);
}

(async () => {
    await mongoose.connect('mongodb://localhost:27017/performance-test');
    await mongoose.connection.dropDatabase();
    await main();
    console.time('performanceTest')
    await run();
    console.timeEnd('performanceTest');
    console.log(mongoose.version)
    process.exit(0);
})();

[billouboq]

if we lean the two do we get the same results ? To know if it's mongoose related or driver related

[IslandRhythms]

lean causes a +/- change of 1 ms.

[billouboq]

So might be driver related I guess

[0Ams]

As for the comparison I made, it was not about saving tests but rather a comparison of the commands like findOne, find, and lean. I am sharing part of a test results. the slowness was observed not only in the actual application connected to MongoDB, but also in a test code using a mongodb-memory-server.

Result

• documents count is 10k

[email protected]

🚀  mongoose version: 4.7.7  🚀
# Schema Validation time: 144.62 ms
# Insert time: 462.90 ms
# FindOne time: 2342.01 ms
# FindOneLean time: 1122.42 ms
# Find time: 711.96 ms

[email protected]

🚀      mongoose version: 8.7.1     🚀
# Schema Validation time: 66.71 ms
# Insert time: 357.36 ms
# FindOne time: 2944.64 ms
# FindOneLean time: 1519.38 ms
# Find time: 722.51 ms

Only the contents of the replica connection were implemented according to the version, and there was a difference when index, shard, and schema were all used in the same state

[0Ams]

I'm writing this comment to check if any other tasks are in progress for performance improvement. 😄

[vkarpov15]

Considering the following basic script:

const mongoose = require('mongoose');
  
const testSchema = new mongoose.Schema({
    name: String,
    val: Number
});


const Test = mongoose.model('Test', testSchema);

async function main() {
    for (let i = 0; i < 1; i++) {
        const doc = new Test();
        doc.name = 'Test',
        doc.val = i;
        await doc.save();
    }
}

(async () => {
    await mongoose.connect('mongodb://localhost:27017/performance_test_mongoose');
    await main();
    console.time('findOneTest');
    for (let i = 0; i < 1000; ++i) {
      const res = await Test.findOne();
    } 
    console.timeEnd('findOneTest');
    process.exit(0);
})();

Here's the runtimes I'm seeing locally:

4.x: 341ms
5.x: 342ms
6.x: 387ms
7.x: 398ms
8.x: 400ms

The most pronounced jump is between Mongoose 5 and Mongoose 6. Specifically Mongoose 6.0.0, because 6.0.0 has similar performance number to 6.x.

For the sake of comparison, consider the following MongoDB Node driver script

const mongodb = require('mongodb');
  
(async () => {
    const client = await mongodb.MongoClient.connect('mongodb://127.0.0.1:27017/performance_test');
    const db = client.db('performance_test');

    await db.collection('tests').deleteMany({});
    await db.collection('tests').insertOne({ name: 'test' });

    console.time('findOneTest');
    for (let i = 0; i < 1000; ++i) {
      const res = await db.collection('tests').findOne();
    }
    console.timeEnd('findOneTest');
    console.time('findTest');
    for (let i = 0; i < 1000; ++i) {
      const res = await db.collection('tests').find().toArray();
    }
    console.timeEnd('findTest');
    process.exit(0);
})();

Runtimes:

2.x: 261ms
3.x: 210ms
4.x: 278ms
5.x: 285ms
6.x: 310ms

So the performance degradation from Mongoose 5 to 6 can be fully explained by upgrading from MongoDB Node driver 3 to 4. I'll raise an issue on MongoDB JIRA.

I did notice that our fix for #7398 https://github.com/Automattic/mongoose/commit/3f52dfbe8ff0f9942148d92584106e5604660d54 includes some logic that tracks a stack trace every time a query is executed, which is the only potential low hanging fruit we've found so far to noticeably improve Mongoose's handling of findOne(). Removing that would be a breaking change, but something to consider.

[vkarpov15]

I opened MongoDB JIRA ticket https://jira.mongodb.org/browse/NODE-6499 to see if there's some feedback from the MongoDB Node driver team, and I'll keep looking for ways to improve findOne() perf on Mongoose's end.

[vkarpov15]

We made some small micro-optimizations, but the reality is that _executionStack = new Error().stack is responsible for nearly 20% of the execution time. We can't remove that without a breaking change, but we will in our next major release.

[vkarpov15]

Fixed by #15298