[Feature] Instrument banyand server

[hanahmily]

Search before asking

• [X] I had searched in the issues and found no similar feature requirement.

Description

Add a series of metrics to the banyand server to measure the internal status of memory, file system, and individual modules. Furthermore, the SkyWalking OAP server could collect them to analyze, and UI could render them.

Use case

No response

Related issues

No response

Are you willing to submit a PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[wu-sheng]

https://github.com/apache/skywalking/issues/9047#issuecomment-1125604323

@kezhenxu94 @Superskyyy @remGigo Let's continue the discussion here. My suggestion is either we make this task on go2sky purely or go2sky + banyandb instrumentation (advanced one).

[kezhenxu94]

My suggestion is either we make this task on go2sky purely or go2sky + banyandb instrumentation (advanced one).

We could do this, adding meter feature to go2sky is definitely a big 👍 .

But why we have to couple BanyanDB and go2sky. I mean:

• We can use Prometheus protocol to provide BanyanDB metrics just like how satellite did, OAP can already consume these metrics.
• We can add go2sky into BanyanDB if we want to add more instrumentation like traces (I believe we don't need that tho)
• We can add a independent project in go2sky to add meter feature.

SO there is no dependency between go2sky and BanyanDB

[wu-sheng]

We don't choose Prometheus for now to BanyanDB. We had a discussion before. OTEL fetcher already causes many concerns. I agree that the tasks for students or new contributors should not have a dependency, so I suggest updating the task description.

[wu-sheng]

Quoted https://github.com/apache/skywalking/issues/9047#issuecomment-1125620275

So, there is no concern here.

[kezhenxu94]

Quoted #9047 (comment)

So, there is no concern here.

Ok, so to sum up, an OSPP project depends on 2 non-OSPP tasks.

#9047 depends on this issue to finish the task 👇

• Monitor the health status of servers using observability APIs

But this issue depends on go2sky to provide meter feature...

[hanahmily]

#9047 depends on this issue to finish the task 👇

• Monitor the health status of servers using observability APIs

But this issue depends on go2sky to provide meter feature...

The basic observability APIs are ready for CLI to access. They are based on Prometheus exporter go libs. The relevant feats in this thread could be based on them.

[kaori-seasons]

I'd like to know where I need to bury the dots and what configuration format I need to render in yaml in order to keep track of the relevant details

[lujiajing1126]

I'd like to know where I need to bury the dots and

I guess "bury the dots" means what kind of operations and information we should collect or record?

what configuration format I need to render in yaml in order to keep track of the relevant details

I suppose according to Hongtao's comment https://github.com/apache/skywalking/issues/9048#issuecomment-1125633917 above, Prom's text-based format is enough now.

[kaori-seasons]

@lujiajing1126 I will first track the relevant process of BanYanDB. Be familiar with the general process. In order to collect information such as memory, file system and the internal state of each module

[kaori-seasons]

Hello everyone, by tracking the code of skywalking-banyandb stress test. I tracked that in the KV class, statistics can be passed through dbOpts Some storage metrics of banyandb, code location At present, I want to add the buried point function of go2sky to the code segment here to obtain the storage information of banyandb. I don't know if my understanding is correct?

@hanahmily Can you give me some advice?

[kaori-seasons]

In addition, I think the indicators should be aggregated here. It is only necessary to report the indicators to the OAP backend of skywalking based on go2sky.

See code location

[wu-sheng]

I don't think go2sky has meter APIs for now. Besides all instrument works, you should begin to discuss those APIs at the go2sky repo with project leader @arugal and other members.

[kaori-seasons]

I don't think go2sky has meter APIs for now. Besides all instrument works, you should begin to discuss those APIs at the go2sky repo with project leader @arugal and other members.

Ok, I am investigating the feasibility of adapting the go2sky api to the meter api. After sorting out the plan, I will communicate with the relevant maintainers in the go2sky project in the form of google doc

[Superskyyy]

@complone Hi, since the go2sky meter api is undertaken by a student through OSPP please try communicate with the mentor first so to prevent collision.

[kaori-seasons]

@complone您好，由于go2sky Meter api是学生通过OSPP承担的，请先尝试与导师沟通，以免发生碰撞。

ok thanks for letting me know

[Superskyyy]

@complone您好，由于go2sky Meter api是学生通过OSPP承担的，请先尝试与导师沟通，以免发生碰撞。

I'm connecting with the student to confirm some details, will update to you.

Update: the student will discuss the matter with mentor and update here.

[hanahmily]

The BanyanDB server has implemented the meter module.