Tracking issues of `Better Project Structures`

[PsiACE]

Background

We have a large number of crates, but they are not properly managed and categorised. Typical problems include

• Many crates re-expose crates that have already been published. (tokio, tracing)
• All split crates are put into the common directory. (user, formats)

Basic Goals

• [ ] A clean root dir structure, all crates in src dir and correctly categorised
• [ ] Splitting binary targets and shared libraries.
• [ ] The reclassification of unit and integration tests. and #5001
• [ ] #7004
• [ ] Optimising Cargo.toml for releasing to crates.io

[Xuanwo]

Will we have an RFC for the new layout?

[PsiACE]

Will we have an RFC for the new layout?

I will enrich it in this issue as it progresses. Perhaps the testing section will have a relatively formal RFC.

[Xuanwo]

A clean root dir structure, all crates in src dir and correctly categorised

So we will remove common and move them into src with well-designed crate names?

[PsiACE]

So we will remove common and move them into src with well-designed crate names?

Most libraries will contain a databend prefix, Finally it looks like this

src
├── cmds // all cmds
│   ├── query.rs
│   ├── metabench.rs // or move to tools dir
│   ├── metactl.rs // or move to tools dir
│   └── meta.rs
├── meta // all meta shared libs
│   ├── databend-meta-api
│   ├── databend-meta-types
│   └── ...
├── query // all query shared libs
│   ├── databend-ast
│   ├── databend-planner
│   └── ...
├── utils // common
│   ├── databend-runtime // some common shared-libs
│   ├── rangemap // for those more generic crates, they do not require the databend prefix
│   └── ...
└── workspace-hack // hakari