DolphinDBPlugin
DolphinDBPlugin copied to clipboard
[New Feature] Development List TO DO
To do list for new Contributors
Contributions are welcome, here is a list for new contributors:
| No | Plugin | Content | Priority | Difficulty | Contributer | Status | Issue/PR |
|---|---|---|---|---|---|---|---|
| 1 | clickhouse | low | medium | ||||
| 2 | hdf5 | add docs about the perfect way of loading data and use cases. | medium | low | |||
| 3 | httpClient | support for windows | high | medium | |||
| 4 | kafka | support for kerberos | low | medium | |||
| 5 | kafka | support for windows | high | medium | |||
| 6 | kdb | support for using loadTextEx | medium | medium | |||
| 7 | kdb | support for reading compressed data of type q IPC (1), snappy (3), L4Z (4) | high | high | |||
| 8 | mqtt | support for arm | low | medium | |||
| 9 | mqtt | publish stream engine calculation results directly to the mqtt server | medium | medium | |||
| 10 | mysql | add docs about return type | medium | low | |||
| 11 | mysql | support for type blob | high | high | |||
| 12 | mysql | support for type decimal | high | high | |||
| 13 | oracle | high | high | ||||
| 14 | parquet | support for reading column of type repeated | high | high | |||
| 15 | parquet | support for windows | high | medium | |||
| 16 | signal | support for rfft, irfft | high | high | |||
| 17 | xgboost | upgrade the library of xgboost to version 1.7 | medium | medium | |||
| 18 | zip | support for arm | low | medium | |||
| 19 | zip | support for compressing folders | high | medium |
Item No.6 kdb+'s tasks: Is it referring to direct file access (similar to loadFile), bypassing q process (similar to loadTable)?
Yes, the way of loadTable/File is totally loading the data into DolphinDB's in-memory table. Thus comes a problem if we do not have enough memory,especially for community users. So we want to have a method loadTextEx that directly loading data into DolphinDB's DFS table, meanwhile reduce the memory usage, for example, each batch costs 512MB.