rocketmq icon indicating copy to clipboard operation
rocketmq copied to clipboard

[Enhancement] When there are too many consumers and there are too many RocketMQ client threads

Open imgoby opened this issue 1 year ago • 6 comments

Before Creating the Enhancement Request

  • [X] I have confirmed that this should be classified as an enhancement rather than a bug/feature.

Summary

When there are too many consumers and there are too many RocketMQ client threads, details: NettyClientWorkerThread_1: 4746 threads Client House Keeping Service: 2373 threads CleanExpireMsgSchechedThread_: 4378 threads RebalanceServic: 2373 threads NettyClientSele: 2373 threads

How to resolve?

Motivation

reduce threads, reduce memory.

Describe the Solution You'd Like

Merge timers, schedule executor

Describe Alternatives You've Considered

Merge timers, schedule executor

Additional Context

No response

imgoby avatar Dec 07 '24 12:12 imgoby

Hi, I want to know how the client is used. Will multiple consumers be created in the same process?

yx9o avatar Dec 10 '24 09:12 yx9o

yes. in the same process. it is a rocketmq client. the process is a java application using springboot+rocketmq For example, there is a topic: hello_topic, Then there were 3000 consumers who consumed the topic according to the conditions and found that the process memory occupied 10G

imgoby avatar Dec 10 '24 09:12 imgoby

Is instanceName set?

yx9o avatar Dec 10 '24 10:12 yx9o

No, can this parameter solve it?

imgoby avatar Dec 10 '24 11:12 imgoby

Rocketmq grpc client will create a thread pool for every consumer or producer. So I modify the client source code to fix this problem。

Yelijah avatar Dec 10 '24 14:12 Yelijah

Image

chenchengwangS avatar Apr 22 '25 03:04 chenchengwangS