matrixone icon indicating copy to clipboard operation
matrixone copied to clipboard

[Subtask]: refactor metric/log filepath as {account}/{megerd|log}/yyyy/mm/dd/{datatype}/*.csv

Open xzxiong opened this issue 3 years ago • 0 comments

Parent Issue

Parenet Issue #5080

Detail of Subtask

refactor metric/log data in path {account}/{megerd|log}/yyyy/mm/dd/{datatype}/*.csv

  • {account}/log/yyyy/mm/dd/{datatype}/*.csv, store raw data directly from MO Node
  • {account}/merged/yyyy/mm/dd/{datatype}/*.csv, store data from {account}/log/yyyy/mm/dd/{datatype}/*.csv which merged into one file (or a few files)

Describe implementation you've considered

we have 5 datatype: metric, statement, span, log, error

  • data metric store in path {account}/*/yyyy/mm/dd/metric/*.csv
  • data statement store in path {account}/*/yyyy/mm/dd/statement/*.csv
  • data span, log, error store in path sys/*/yyyy/mm/dd/log/*.csv
    • these 3 type data will be store in one table (just named table log)
    • just access by sys account.

Additional information

No response

xzxiong avatar Oct 13 '22 04:10 xzxiong