Item order is broken in BatchGet and in queries using hydration

[iOSonntag]

Describe the bug When performing a regular BatchGet operation the order is broken (even with preserveBatchOrder: true).

The same goes for querying a KEYS_ONLY table using the hydrate: true option. The order is returned correctly when using includeKeys: true, ignoreOwnership: true but the hydration breaks the correct order. Which makes sense to me as it uses a BatchGet internally.

ElectroDB Version Specify the version of ElectroDB you are using 2.14.0

Some example code (containing my workaround)

const response = await UserAccount.query.by_questionsSubmittedUserId_KEYS_ONLY({}).go({
  order: 'desc',
  ignoreOwnership: true,
  includeKeys: true,
  cursor: cursor,
});

if (response.data.length === 0)
{
  return {
    data: [],
    cursor: null,
  };
}

const correctOrder: string[] = [];

const requestedUsers = response.data.map((dynamoDbItem: any) =>
{
  const userId = dynamoDbItem.sk.split('#userid_')[1];
  correctOrder.push(userId);
  return {
    userId: userId,
  };
});

const userBatchResponse = await UserAccount.get(requestedUsers).go({
  table: useTable(),
  preserveBatchOrder: true,
});

const badOrderedUsers: UserAccountEntity[] = userBatchResponse.data.filter((user) => user !== null && user !== undefined) as UserAccountEntity[];
const users: UserAccountEntity[] = correctOrder.map((userId) => badOrderedUsers.find((user) => user.userId === userId)).filter((user) => user !== undefined) as UserAccountEntity[];

I confirmed this behaviour on multiple Entities and different scenarios, currently I use 6 workarounds like the one above.

[tywalch]

Hi @iOSonntag 👋

It sounds like you're you're reporting a two different issues:

1. The execution option preserveBatchOrder doesn't seem to be working as expected.
2. Order is not maintained with the execution option hydrate

If the above is correct, do you have anyway to recreate the behavior you're seeing so that I can look closer into what might be happening?

[iOSonntag]

Alright let me first finish this mega project I am working on right now and then I will get some playground to work that will recreate the issue.

Right now everything is fixed via workarounds so I am good.

Also note that I would be happy to become one of the maintainers / core maintainers of electrodb. I really love this framework, and I know what a pain you have to go through to get something that good on top of DynamoDB. I wrote a DynamoDB framework myself years ago and was quit happy with that until I heard of electrodb - then I switched haha.

If you see there any possibility just let me know @tywalch