doris icon indicating copy to clipboard operation
doris copied to clipboard

[feature-wip](statistics) FE queries BE by SQL

Open weizhengte opened this issue 3 years ago • 0 comments

Proposed changes

Query BE from FE using MySQL protocol. This function is currently mainly used for statistics module. FE obtains statistics by SQL query BE, such as column maximum value, minimum value, etc.

The main implementation of a similar SQL client, through the MySQL protocol for data interaction. The basic process is as follows:

  1. Build a similar SQL client in FE to establish a connection;
  2. Authentication and data interaction based on MySQL protocol;
  3. Execute SQL and parse the resulting packets of SQL execution.

The usage process is as follows(the following code does no exception handling):

Connection con = new Connection("db");
con.init();
QueryResultSet query = con.query("SELECT * FROM table1");
List<String> columns = query.getColumns();
while (query.next()) {
    for (String column : columns) {
        System.out.println(query.getColumnValue(column));
    }
}
con.close();

Note: this is a tool module as statistics, it will not affect the original code, also will not affect the use of users. It can even be removed if necessary in the future.

Relevant reference:

  • https://mariadb.com/kb/en/connection/
  • https://dev.mysql.com/doc/internals/en/client-server-protocol.html
  • https://dev.mysql.com/doc/internals/en/com-query-response.html#text-resultset-row

Problem Summary:

Describe the overview of changes.

Checklist(Required)

  1. Does it affect the original behavior: (Yes/No/I Don't know)
  2. Has unit tests been added: (Yes/No/No Need)
  3. Has document been added or modified: (Yes/No/No Need)
  4. Does it need to update dependencies: (Yes/No)
  5. Are there any changes that cannot be rolled back: (Yes/No)

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

weizhengte avatar Jun 07 '22 03:06 weizhengte