GSA
CKAN elasticsearch support
Feature/what we're after
Have CKAN support ElasticSearch; a much more highly utilized service that is available out of the box on many platforms
Anticipated/hypothesized benefits
- Less development/management time
- No need for SOLR in SSB
Measurements/metrics
- SOLR is no longer needed for data.gov catalog; and management of system is much easier
References/background
It seems from research and code review, and CKAN is currently tightly coupled with SOLR but does not need to be. There have been multiple discussions/attempts and making this more configurable (both through core and through an extension), but neither came to fruition. Many core CKAN devs support the idea of the work, but clearly don't have the capacity to make it happen. This would be a huge boon/boost for CKAN moving forward and may help tip the scales for future teams considering open source data management services long term (as spinning up an elasticsearch instance in AWS, GCP, cloud.gov etc is incredibly easy, while SOLR is not).
- https://gitter.im/ckan/chat?at=60ed9f5d48f9b1084a29fe86
- https://gitter.im/ckan/chat?at=5f3e6f81750a2741303063c5
- https://github.com/ckan/ckan/pull/3118
- https://logz.io/blog/solr-vs-elasticsearch/ (good overview of the pros & cons of solr and elasticsearch; our current pain points are all of the solr cons and from a high level it seems like we wouldn't hit the elasticsearch cons)
Have not entered this into GitHub projects yet -- currently a limit on number of epics on board to 25
This doesn't solve the problem, but a number of people have worked on and/or maintained code that can alternatively search solr and elasticsearch. https://github.com/o19s/hello-ltr being one.