feat: save data into pageTree table after processing a batch of pages
While re-building a page tree, all the existing page details are fetched from the database and a tree array is created at the application side. Once the tree is created, the pageTree table is truncated & the objects in the tree array are saved to the pageTree table in chunks. When there are a lot of records present in the page table, it takes a lot of time to build the tree array at the application side causing the database connections to timeout and it is causing the rebuild-tree job to fail.
In order to avoid idle connections, data should be saved right after enough number of records are added to the tree array.
The downside of this approach is that the pageTree table will be empty or incomplete during the rebuild. Users will experience empty or broken navigation if the rebuild takes a long time to complete.
The worker maintains a pool of DB connections and this isn't a transaction. So whether the connection times out shouldn't matter in this case. A new one will be established automatically if needed.