tiled icon indicating copy to clipboard operation
tiled copied to clipboard

Scientific Metadata Search Engine client service using TypeSense

Open Kezzsim opened this issue 2 years ago • 0 comments

We can successfully instantiate and connect to a new Typesense instance with this .yml file and the command tiled serve config config.yml

authentication:
  # The default is false. Set to true to enable any HTTP client that can
  # connect to _read_. An API key is still required to write.
  allow_anonymous_access: false
  single_user_api_key: "secret"  # for dev
trees:
  - path: /
    tree: catalog
    args:
      uri: "sqlite+aiosqlite:///:memory:"
      # or, uri: "sqlite+aiosqlite:////catalog.db"
      # or, "postgresql+asyncpg://..."
      writable_storage: "tmp/"
      init_if_not_exists: true
      typesense_client:
        api_key: "secret"
        schemas:
        nodes:
          - host: "http://localhost"
            port: 8108
            protocol: "http"

However there is still a lot of specific work to be done, namely items mentioned in #605 :

  • [X] Set up Typesense client dependencies
  • [X] Instantiate a client and have it connect with a typesense server
  • [ ] Create a bluesky event model reader that generates a standard base schema
  • [ ] Save schemas to files that are extensible and include-able by name so the entire schema does not have to be explicitly set each time.
    • (e.g. you could have a schema file for each beamline at NSLS-II with some of their metadata shape imparted)
  • [ ] Hook into Context.__init__
    • [ ] Initialize Typesense instance by iterating over all data in the tiled instance
    • [ ] Upsert search engine data on database insertion by registering [after_insert] sqlalchemy hook

Kezzsim avatar Mar 22 '24 21:03 Kezzsim