[FR] 外部ck ttl 删除很慢

[yhcloud123]

Search before asking

• [X] I had searched in the issues and found no similar feature requirement.

Description

有遇到过外部ck，ttl 3d到期实际删除很慢吗，对比两套环境一样的配置，采集数据量小的会严格遵循ttl，数据量大一点的的ck ttl到期会删除一小部分数据（如删除近10天随机过期数据），比如最开始几天的部分一直没有删除

或者可以测试建表设置参数ttl_only_drop_parts 会不会好点 这块有什么建议吗

Use case

1

Related issues

No response

Are you willing to submit a PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[1473371932]

This may be a bug inherent in ClickHouse, occasionally failing to clean up as expected. You might try restarting ClickHouse to potentially resolve this issue. Note: If you are using an external ClickHouse and data storage exceeds 80%, DeepFlow-Server will not trigger automatic cleanup

[lzf575]

@yhcloud123 v6.5 和 main版本 新建表会增加ttl_only_drop_parts 这个配置了。 你也可以 手动修改表，加上这个配置看看，是否有效果。例如 alter table flow_log.l7_flow_log_local MODIFY SETTING ttl_only_drop_parts=1;