LiteDB performance issues

[GuyPago]

I've noticed 2 main problems in LiteDB usages in this project. The first one is the (very) common usage of the Table property, which unlike MongoRepository's implementation, this:

   // LiteDBRepository.cs
   public virtual IQueryable<T> Table => Collection.Query().ToEnumerable().AsQueryable();

will always get all documents from the collection (as ToEnumerable executes), making AsQueryable() instructions occur on the machine rather than the DB. Imagine a large store having 10M products while running a liteDB instance.

The 2nd problem is the fact that LiteDB doesn't provide async methods. Which means the ToEnumerable call above blocks the calling thread. Not me mention the usage of await Task.FromResult below that does nothing but add an async handling overhead while still blocking, but is indeed needed to implement IRepository.

    // LiteDBRepository.cs
    public virtual async Task<T> GetByIdAsync(string id)
    {
        return await Task.FromResult(GetById(id));
    }

Adding the two problems together might introduce a significant performance impact so i'd consider either:

1. Use https://github.com/mlockett42/litedb-async to "handle" the 2nd problem. (I can create a migration PR)
2. Highly endorse MongoDB usage over LiteDB.

[KrzysztofPajak]

Thank you for your detailed analysis and valuable feedback. You are absolutely right — LiteDB has significant limitations when it comes to working with IQueryable and asynchronous operations. Since this project is primarily focused on MongoDB as the main database, LiteDB is intended only as a lightweight, embedded option for very small stores.

I will consider the possibility of incorporating LiteDB.Async in the future, but at this point, I prefer keeping LiteDB integration as simple as possible.

[GuyPago]

Thank you for your feedback, I have to say that I've learned a lot by exploring the project. But that was also my point - I've learned that liteDB has significant performance impact after exploring the code, so as a possible user that serve my store via the easier embedded approach using LiteDB, I'd have no idea.

So my 2nd possible solution I've mentioned is to pass this information. For example we can mention here: That its only recommended for small stores or something like that.

If it might make a user to abandon GrandNode due to a laggy store - It'd be nice to prevent it😃.