apache
[Feature] [BanyanDB] Support field indexing but not stored
Search before asking
- [X] I had searched in the issues and found no similar feature requirement.
Description
This is a very special feature of ElasticSearch @wankai123 found from Zipkin using ElasticSearch to save spans.
In saving traces, tags are indexed as query conditions to make users easier to locate important traces, but meanwhile, the whole trace segment is already stored as binary(base64 encoded) in the no-indexing field. So most indexed tags are actually duplicated from the data table perspective.
I hope we could have a way to define a field/column of a table indexing(by different index types) but also not stored. All fields in this type are not allowed in projection clearly because they are stored, OAP could read those values from the binary field directly.
@wankai123 is going to propose this in ElasticSearch storage implementation.
Use case
No response
Related issues
No response
Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
As I understood, this is what exactly BanyanBD is doing.
We only duplicate indexed tags which can be used in the projection. In the meanwhile, we store the whole binary data in non-indexed tags.
The key of this issue is just tracking the tags not stored. I add more to other readers about context only.