pinot icon indicating copy to clipboard operation
pinot copied to clipboard

Expand Full Text Search Capabilities

Open atris opened this issue 4 years ago • 2 comments

Our FTS capabilities are pretty limited right now -- REGEXP_MATCH does a primitive regex match and TEXT_MATCHES is simply a wrapper around basic Lucene level searching (does not expose fuzzy queries or the option to rank).

This issue states a proposal for expanding our FTS capabilities with the following initial ideas:

  1. Support LIKE operator (https://www.w3schools.com/sql/sql_like.asp).
  2. Improve TEXT_MATCH (see #7215 , #7216 and #7217 ).
  3. Allow ranked searches (something on the lines of https://www.postgresql.org/docs/9.5/textsearch-controls.html#TEXTSEARCH-RANKING)

atris avatar Jul 28 '21 08:07 atris

this is an relatively old issue, and some things are already implemented.

Coming from the ecommerce field where good text search is an absolute must-have, the question is: why shouldn't search be from the same quality in real-time analytics??

here are some screenshots of the backend settings of a great lucene integration to a shop software (magento addon by extendware) yes they are a little aged and don't include the latest things like using embeddings for similarity search, but they are still a perfect baseline.. => Maybe some of these settings and their functionality are a good inspiration for advancing full text search in pinot further:

lucene-search-for-magento-1-350

lucene-search-for-magento-1-f43

lucene-search-for-magento-1-eba

hpvd avatar Feb 14 '24 12:02 hpvd

maybe this is fyi @kkrugler

hpvd avatar Feb 14 '24 12:02 hpvd