cube
cube copied to clipboard
Published
1 month ago
•
cube-js
cube-js
Cubestore - Minio s3 remotefs, undefined subpath consistency error.
Check List
- [ ] Tests has been run in packages where changes made if available
- [ ] Linter has been run for changed code
- [ ] Tests for the changes have been added if not covered yet
- [ ] Docs have been added / updated if required
Issue Reference this PR resolves
[For example #12]
During each file upload, a consistency error occurred. This could have been due to the fact that if there is no defined 'subpath,' a '/' is still appended to the beginning of the path. In the case of a defined subpath, this does not happen, as the subpath becomes the first value received, rather than the '/' symbol.
Error:
2023-11-23 09:33:46,454 ERROR [cubestore::util] <pid:1> Error during Metastore upload: CubeError { message: "File metastore-1700732026259/CURRENT can't be listed after upload. Either there's Cube Store cluster misconfiguration, or storage can't provide the required consistency.", backtrace: "", cause: Internal }
Error: Error during upload of asdasdasd_rollup_gg1crwol_pitg0ves_1ils6st-0.csv.gz create table: CREATE TABLE asdasdasd_rollup_gg1crwol_pitg0ves_1ils6st (`asd_asd__guid` varchar(64), `asd__record_main_type` varchar(255), `asd__record_sub_type` varchar(255), `asd__record_title` varchar(255), `asd__created_at_day` timestamp, `asd__count` bigint): Internal: File temp-uploads/asdasdasd_rollup_gg1crwol_pitg0ves_1ils6st-0.csv.gz can't be listed after upload. Either there's Cube Store cluster misconfiguration, or storage can't provide the required consistency.
at /cube/node_modules/@cubejs-backend/cubestore-driver/src/CubeStoreDriver.ts:326:25
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
The latest updates on your projects. Learn more about Vercel for Git ↗︎
8 Ignored Deployments
| Name | Status | Preview | Comments | Updated (UTC) |
|---|---|---|---|---|
| examples-angular-dashboard | ⬜️ Ignored (Inspect) | Visit Preview | Nov 23, 2023 1:42pm | |
| examples-react-d3 | ⬜️ Ignored (Inspect) | Visit Preview | Nov 23, 2023 1:42pm | |
| examples-react-dashboard | ⬜️ Ignored (Inspect) | Visit Preview | Nov 23, 2023 1:42pm | |
| examples-react-data-table | ⬜️ Ignored (Inspect) | Visit Preview | Nov 23, 2023 1:42pm | |
| examples-react-highcharts | ⬜️ Ignored (Inspect) | Visit Preview | Nov 23, 2023 1:42pm | |
| examples-react-material-ui | ⬜️ Ignored (Inspect) | Visit Preview | Nov 23, 2023 1:42pm | |
| examples-react-pivot-table | ⬜️ Ignored (Inspect) | Visit Preview | Nov 23, 2023 1:42pm | |
| examples-vue-query-builder | ⬜️ Ignored (Inspect) | Visit Preview | Nov 23, 2023 1:42pm |
![vercel[bot] avatar](https://avatars.githubusercontent.com/in/8329?v=4 "Published by a pub.dev verified publisher"){kind=small}
@htsnorman I believe we'd need a test for this one. I'm wondering how we can do one in docker.
hello @paveltiunov
i create a fast docker test where the error can be reproduced.
here : https://github.com/htsnorman/cube-cubestore-minios3-tester
the "readme" has all the info
@paveltiunov
i build my branch cubestore:
(base) sofalvin@norberts-mbp cubestore % docker build -t cube:nrmn ./
[+] Building 1.6s (33/33) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 2.00kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 191B 0.0s
=> [internal] load metadata for docker.io/library/debian:bullseye-slim 1.5s
=> [internal] load metadata for docker.io/library/rust:1-bullseye 1.4s
=> [auth] library/rust:pull token for registry-1.docker.io 0.0s
=> [auth] library/debian:pull token for registry-1.docker.io 0.0s
=> [builder 1/21] FROM docker.io/library/rust:1-bullseye@sha256:16ab9fc5fee47ad0329b831a4838cc2de91437dbe77d49c6d1c1f0370d3f02db 0.0s
=> [stage-1 1/4] FROM docker.io/library/debian:bullseye-slim@sha256:5aab272aa24713622bfac9dba239bc7488d9979b0d82d19a9dffccd99292154d 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 21.15kB 0.0s
=> CACHED [stage-1 2/4] WORKDIR /cube 0.0s
=> CACHED [stage-1 3/4] RUN set -ex; apt-get update; apt-get install -y libssl1.1 curl 0.0s
=> CACHED [builder 2/21] RUN rustup update && rustup default nightly-2022-06-22 && rustup component add --toolchain nightly-2022-06-22 rustfmt; 0.0s
=> CACHED [builder 3/21] RUN apt update && apt upgrade -y && DEBIAN_FRONTEND=noninteractive apt-get install -y software-properties-common pkg-config wget gnupg git apt-transport-https ca-certificates && wget -O - 0.0s
=> CACHED [builder 4/21] RUN update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-15 100 0.0s
=> CACHED [builder 5/21] RUN update-alternatives --install /usr/bin/clang clang /usr/bin/clang-15 100 0.0s
=> CACHED [builder 6/21] RUN update-alternatives --install /usr/bin/cc cc /usr/bin/clang-15 100 0.0s
=> CACHED [builder 7/21] RUN update-alternatives --install /usr/bin/c++ c++ /usr/bin/clang++-15 100 0.0s
=> CACHED [builder 8/21] WORKDIR /usr/src 0.0s
=> CACHED [builder 9/21] WORKDIR /build/cubestore 0.0s
=> CACHED [builder 10/21] COPY Cargo.toml . 0.0s
=> CACHED [builder 11/21] COPY Cargo.lock . 0.0s
=> CACHED [builder 12/21] COPY cubehll cubehll 0.0s
=> CACHED [builder 13/21] COPY cubezetasketch cubezetasketch 0.0s
=> CACHED [builder 14/21] COPY cuberpc cuberpc 0.0s
=> CACHED [builder 15/21] COPY cubestore-sql-tests cubestore-sql-tests 0.0s
=> CACHED [builder 16/21] COPY cubestore/Cargo.toml cubestore/Cargo.toml 0.0s
=> CACHED [builder 17/21] RUN mkdir -p cubestore/src/bin && echo "fn main() {print!("Dummy main");} // dummy file" > cubestore/src/bin/cubestored.rs 0.0s
=> CACHED [builder 18/21] RUN [ "1" -eq "1" ] && export RUSTFLAGS="-C target-feature=+avx2"; cargo build --release -p cubestore 0.0s
=> CACHED [builder 19/21] COPY package.json package.json 0.0s
=> CACHED [builder 20/21] COPY cubestore cubestore 0.0s
=> CACHED [builder 21/21] RUN [ "1" -eq "1" ] && export RUSTFLAGS="-C target-feature=+avx2"; cargo build --release -p cubestore 0.0s
=> CACHED [stage-1 4/4] COPY --from=builder /build/cubestore/target/release/cubestored . 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:9e1450cd793157625f5bcb72feace1ff8ed666f8059e38312936c5801b6a6493 0.0s
=> => naming to docker.io/library/cube:nrmn
and try my docker compose tester without define subpath:
(base) sofalvin@norberts-mbp cubestore % docker compose ps
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
cubestore-cubestore_router-1 cube:nrmn "./cubestored" cubestore_router About a minute ago Up 42 seconds 0.0.0.0:3030->3030/tcp, 0.0.0.0:3306->3306/tcp, 0.0.0.0:5432->5432/tcp
cubestore-cubestore_worker_1-1 cube:nrmn "./cubestored" cubestore_worker_1 About a minute ago Up About a minute 3306/tcp
cubestore-minio-1 minio/minio "/usr/bin/docker-entrypoint.sh server --console-address :19001 --quiet /data" minio About a minute ago Up About a minute 0.0.0.0:19001->19001/tcp, 0.0.0.0:9002->9000/tcp
(base) sofalvin@norberts-mbp cubestore %
(base) sofalvin@norberts-mbp cubestore %
(base) sofalvin@norberts-mbp cubestore % mysql -h localhost
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)
(base) sofalvin@norberts-mbp cubestore % mysql -h 127.0.0.1
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 5.1.10-alpha-msql-proxy
Copyright (c) 2000, 2023, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> create schema demoapp;
create table demoapp.demoapp(id varchar(100),age int);
insert into demoapp.+------+---------+
| id | name |
+------+---------+
| 1 | demoapp |
+------+---------+
1 row in set (0.01 sec)
demoapp(id,age) values('1',3);
select * from demomysql>
mysql> create table demoapp.demoapp(id varchar(100),age int);
insert into demoapp.demoapp(id,age) values('1',3);
select * from demoapp.demoapp;+------+------------+-----------+-------------------------------------------------------------------------------------------------------------+-----------+---------------+----------+----------+-----------------------------------+-----------------+---------+--------+------------------+----------------+---------------+---------------------------+--------------------------+------------------+-------------------------+---------------------------+
| id | table_name | schema_id | columns | locations | import_format | has_data | is_ready | created_at | build_range_end | seal_at | sealed | select_statement | source_columns | stream_offset | unique_key_column_indices | aggregate_column_indices | seq_column_index | location_download_sizes | partition_split_threshold |
+------+------------+-----------+-------------------------------------------------------------------------------------------------------------+-----------+---------------+----------+----------+-----------------------------------+-----------------+---------+--------+------------------+----------------+---------------+---------------------------+--------------------------+------------------+-------------------------+---------------------------+
| 1 | demoapp | 1 | [{"name":"id","column_type":"String","column_index":0},{"name":"age","column_type":"Int","column_index":1}] | NULL | NULL | false | true | 2023-12-01 12:02:20.757914718 UTC | NULL | NULL | false | NULL | NULL | NULL | NULL | | NULL | NULL | NULL |
+------+------------+-----------+-------------------------------------------------------------------------------------------------------------+-----------+---------------+----------+----------+-----------------------------------+-----------------+---------+--------+------------------+----------------+---------------+---------------------------+--------------------------+------------------+-------------------------+---------------------------+
1 row in set (0.01 sec)
mysql>
mysql> insert into demoapp.demoapp(id,age) values('1',3);
Query OK, 0 rows affected (0.08 sec)
mysql>
mysql> select * from demoapp.demoapp;
+------+------+
| id | age |
+------+------+
| 1 | 3 |
+------+------+
1 row in set (0.11 sec)
everything fine.
@nrmn2492 I mean, we need a smoke test for minio. Unfortunately, we weren't able to build one upon this contribution, but we need one in order to move forward as there's no way to validate right now if the change works or not.