Unexpected Data Sync Across Stores and Empty Persister Path in TinyBase Server

[patrykwegrzyn]

Describe the bug

Unexpected Data Sync Across Stores and Empty Persister Path in TinyBase

Hi there!

First off, let me say this is a fantastic library—thank you for the effort that’s gone into it!

I’ve been playing around with tinybase and hit a couple of snags. It’s entirely possible I’m missing something obvious, but I wanted to reach out to clarify. Here’s my setup:

Server Code

import { WebSocketServer } from "ws";
import { createWsServer } from "tinybase/synchronizers/synchronizer-ws-server";

const wsserver = new WebSocketServer({ port: 8040 });

const server = createWsServer(wsserver, (pathid) => {
  console.log("pathid ", pathid);
});

setInterval(() => {
  console.log(server.getStats());
  console.log(server.getPathIds());
}, 1000);

Client 1

This client creates a mergeable store with ID store1 and incrementally sets the count value.

import { createMergeableStore } from "tinybase";
import { createWsSynchronizer } from "tinybase/synchronizers/synchronizer-ws-client";
import ReconnectingWebSocket from "reconnecting-websocket";

const store = createMergeableStore("store1");
const synchronizer = await createWsSynchronizer(
  store,
  new ReconnectingWebSocket("ws://localhost:8040")
);

async function main() {
  let i = 0;

  await synchronizer.startSync();
  setInterval(() => {
    console.log("setting count", i);

    store.setValues({ count: i });
    console.log(store.getValues());
    i++;
  }, 1000);
}

main();

Client 2

This one creates a mergeable store with ID store1 as well and just prints values.

import { createMergeableStore } from "tinybase";
import { createWsSynchronizer } from "tinybase/synchronizers/synchronizer-ws-client";
import ReconnectingWebSocket from "reconnecting-websocket";

const store = createMergeableStore("store1");
const synchronizer = await createWsSynchronizer(
  store,
  new ReconnectingWebSocket("ws://localhost:8040")
);

async function main() {
  await synchronizer.startSync();
  setInterval(() => {
    console.log("get client2 store1", store.getValues());
  }, 1000);
}

main();

Client 3

This client creates a mergeable store with ID store2 and just prints values.

import { createMergeableStore } from "tinybase";
import { createWsSynchronizer } from "tinybase/synchronizers/synchronizer-ws-client";
import ReconnectingWebSocket from "reconnecting-websocket";

const store = createMergeableStore("store2");
const synchronizer = await createWsSynchronizer(
  store,
  new ReconnectingWebSocket("ws://localhost:8040")
);

async function main() {
  await synchronizer.startSync();
  setInterval(() => {
    console.log("get client3 store2", store.getValues());
  }, 1000);
}

main();

---

The Issues

1. All clients seem to be receiving data for store1, even though client3 is using store2.
2. The server's createPersisterForPath function always returns an empty string.

Any insight into what might be causing these issues would be greatly appreciated. Let me know if I’ve missed something, or if you’d like me to provide more details or tweak my setup.

Thanks again for the great library and for taking the time to look into this!

Cheers

Your Example Website or App

No response

Steps to Reproduce the Bug or Issue

No response

Expected behavior

No response

Screenshots or Videos

No response

Platform

• OS: [Linux]

Additional context

No response

[ajimae]

For the server, to persist client data to a file path, you will need to have the path to the file specified. E.g

import { WebSocketServer } from "ws";
import { createMergeableStore } from "tinybase";
import { createFilePersister } from "tinybase/persisters/persister-file";
import { createWsServer } from "tinybase/synchronizers/synchronizer-ws-server";

const store = createMergeableStore();
const wsserver = new WebSocketServer({ port: 8040 });

const server = createWsServer(wsserver, (pathid) => {
  console.log("pathid ", pathid);
  return createFilePersister(
    store,
    ".db/data.json" // mkdir `.db` the dir .db must exists for TinyBase to create data.json file
  );
});

setInterval(() => {
  console.log(server.getStats());
  console.log(server.getPathIds());
}, 1000);

For the clients, I can't really tell, but one thing you could do is to listen for message events in the client synchronizer synchronizer, or even error event maybe we could see if there is an error.

import { createMergeableStore } from "tinybase";
import { createWsSynchronizer } from "tinybase/synchronizers/synchronizer-ws-client";
import ReconnectingWebSocket from "reconnecting-websocket";

const store = createMergeableStore("store2");
const synchronizer = await createWsSynchronizer(
  store,
  new ReconnectingWebSocket("ws://localhost:8040")
);

async function main() {
  await synchronizer.startSync();
  await synchronizer.load();
  await synchronizer.save();

  // listeners
  sync.getWebSocket().addEventListener("message", () => {})
  sync.getWebSocket().addEventListener("error", () => {})

  setInterval(() => {
    console.log("get client3 store2", store.getValues());
  }, 1000);
}

main();

[Edit]: I think there is a known issue here on syncing multiple stores over a single WebSocket connection. Apparently, it seems currently we cannot sync multiple client stores over a single WebSocket connection

I hope it helps

[patrykwegrzyn]

Hi @ajimae, Thanks for your response! After taking a closer look, I think my issue might be a bit different. From what I can see in the code, the pathid should be provided in the callback. It originates from the WebSocket connection and gets passed to configureServerClient, which ultimately invokes the optional createPersisterForPath.

However, in my case, the pathid is always empty.

Regarding the clients, I'm not using a single WebSocket connection. The example above actually reflects three separate client node processes.

[khairul169]

Hi @ajimae, Thanks for your response! After taking a closer look, I think my issue might be a bit different. From what I can see in the code, the pathid should be provided in the callback. It originates from the WebSocket connection and gets passed to configureServerClient, which ultimately invokes the optional createPersisterForPath.

However, in my case, the pathid is always empty.

Regarding the clients, I'm not using a single WebSocket connection. The example above actually reflects three separate client node processes.

Hi there, i think your pathid are always empty because you need to provide it on the websocket url instead of store name. Could you maybe try to change the websocket url to something like this:

• ws://localhost:8040/store1
• ws://localhost:8040/store2
• ws://localhost:8040/store3

[ajimae]

Hi @ajimae, Thanks for your response! After taking a closer look, I think my issue might be a bit different. From what I can see in the code, the pathid should be provided in the callback. It originates from the WebSocket connection and gets passed to configureServerClient, which ultimately invokes the optional createPersisterForPath. However, in my case, the pathid is always empty. Regarding the clients, I'm not using a single WebSocket connection. The example above actually reflects three separate client node processes.

Hi there, i think your pathid are always empty because you need to provide it on the websocket url instead of store name. Could you maybe try to change the websocket url to something like this:

• ws://localhost:8040/store1
• ws://localhost:8040/store2
• ws://localhost:8040/store3

Exactly, the pathId is the pathname of the WebSocket url, this will ultimately be passed your server.