extensions icon indicating copy to clipboard operation
extensions copied to clipboard

[firestore-bigquery-export] Value for argument "limit" is not a valid integer

Open tgpsantos opened this issue 3 years ago • 1 comments

[REQUIRED] Step 2: Describe your configuration

[REQUIRED] Step 3: Describe the problem

Hi guys,

Today I followed the import existing documents guideline but this problem occurred

imagem

Any idea on what's causing it?

PS: There is another issue, for some reason every time I run the script I have to introduce the values 2x (as it's also seen in the picture above)

tgpsantos avatar Mar 11 '22 20:03 tgpsantos

Further investigation shows an issue when using the multi-thread option that will need to be fixed.

As a temporary solution, users should select no for this prompt.

dackers86 avatar Mar 16 '22 19:03 dackers86

Hi, I'm getting the same result even when selecting No when prompted with the question of multiple threads. I tried every configuration that I could think of, but it always resulted in this same error message. I'm following the steps shown here https://github.com/firebase/extensions/blob/master/firestore-bigquery-export/guides/IMPORT_EXISTING_DOCUMENTS.md.
Screenshot (22)

nextlevelwebapps avatar Mar 30 '23 16:03 nextlevelwebapps

still getting same error even set 'No' for the multiple threads :( any other solution?

Roo2190 avatar May 11 '23 01:05 Roo2190

Hi, I'm getting the same result even when selecting No when prompted with the question of multiple threads. I tried every configuration that I could think of, but it always resulted in this same error message. I'm following the steps shown here https://github.com/firebase/extensions/blob/master/firestore-bigquery-export/guides/IMPORT_EXISTING_DOCUMENTS.md. Screenshot (22)

still getting same error even set 'No' for the multiple threads :( any other solution?

You can use this

npx @firebaseextensions/fs-bq-import-collection \
  --non-interactive \
  --project=[your_project] \
  --source-collection-path=[your_collection] \
  --query-collection-group=false \
  --dataset=[your_dataset] \
  --table-name-prefix=[your_table] \
  --batch-size=300 \
  --dataset-location=us \
  --multi-threaded=false \
  --use-new-snapshot-query-syntax=false \
  --use-emulator=false

diepminh707 avatar May 19 '23 07:05 diepminh707

Yes, for anyone confused about this (I was stuck for like an hour). This will work if you run the script non-interactively. Just use all the required flags all in one command. Here is an example

➜  ~ npx @firebaseextensions/fs-bq-import-collection \
--non-interactive \
--project {GCP PROJECT_ID} \
--source-collection-path {FIRESTORE COLLECTION PATH NAME} \
--query-collection-group {FALSE} \
--dataset {USER FRIENDLY NAME FOR THE BQ DATASET} \
--table-name-prefix {TABLE NAME PREFIX} \
--dataset-location us

If the table is called data_table_raw_changelog, just put data_table for table-name-prefix

kevin-hammond avatar Jul 25 '23 00:07 kevin-hammond