go-graphite
How do you think about adding user and metric isolation support?
This is just a discussion.
I have the following situation: I would like to provide the graphite stack as a service to a number of users, let them use it freely. Metric names thus can easily collide. So I think there'd better have some kind of isolation mechanisms by the stack.
For example, let's assume there exists a user system, each user has a unique id. When a user sends in a metric, the metric name is prefixed with the user's id before written to disk. When a user queries metrics, all queried metric names are prefixed with the id before hitting carbonzipper, and stripped again before sending results to user.
On the receiving side, it's relatively easy to implement. For example I can write a simple HTTP gateway which do the authenticating and prefixing and then relay metrics to backend graphite. But on the rendering side, it's somehow complicated. The query target could be a complicated expression, and even contains wildcards, also, there exists functions which manipulates metric names (e.g. aliasSub), it's difficult to implement transparent gateway without manipulating query expressions. So I think the best place to implement this feature is inside carbonapi.
carbonapi does not necessary to implement authentication feature, it can just check if there exists an HTTP header (like X-PREFIX), if such an header exists, then do the prefixing work, otherwise operate normally. Let some HTTP gateway (e.g. nginx) do the authenticating and set the header correspondingly.
How do you think about this feature? If you agree to this feature, do you have any better ideas on how to implementing it? If you're willing to include this feature into carbonapi, I would like to contribute. If you don't want to make carbonapi such complicated, I can, maybe, fork and maintain a independent project.
Possibly relevant from graphite(tm) "REMOTE_STORE_METRIC_PREFIX": https://github.com/graphite-project/graphite-web/pull/1269#issuecomment-287659510
aliasSub and other name-manging functions are not really a concern, I think, as carbonapi must parse the arguments to locate the starting metrics to fetch.
I think such a thing would want to maintain the prefixed name throughout as much of the pipeline as possible to avoid contaminating caches. For that reason carbonapi seems a better place than carbonzipper (or carbonstore).
Sorry I come back so late for this discussion, it's glad that this issue is still open.
REMOTE_STORE_METRIC_PREFIX mentioned in graphite-project/graphite-web#1269 is in indeed what I need. It seems graphite-web has not made any progress on that issue yet. I think I can open a branch to try to implement this feature at any time. But since you're refactoring carbonapi and carbonzipper, I think I'd better wait for the new carbonapi and zipper to be stable. So do you have a time plan on the refactor work?