qgisprocess icon indicating copy to clipboard operation
qgisprocess copied to clipboard
verified publisher icon r-spatial

Error parsing JSON format data in qgisprocess::qgis_run_algorithm()

Open CC3729 opened this issue 9 months ago • 2 comments

Things to check beforehand

Are QGIS plugins the cause?

The problem remains with all plugins disabled.

Description of the observed problem

  • Problem Description While using the qgis_run_algorithm function in the qgisprocess package in R, I frequently encounter the following error:
Error in `processx::run("cmd.exe", c("/c", "call", path, args), ...)`:
! System command 'cmd.exe' failed
---
Exit status: 1
Stderr:
Could not parse JSON parameters: [json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal
This error occurs when running the sagang:sagawetnessindex algorithm via qgis_run_algorithm. A similar error also appears when testing with the native:buffer algorithm.
  • Two pieces of code using the qgisprocess::qgis_run_algorithm function.

1. sagang:sagawetnessindex algorithm

r
data("study_area", "random_points", "comm", package = "spDataLarge")
dem = raster(system.file("raster/dem.tif", package = "spDataLarge"))
ndvi = raster(system.file("raster/ndvi.tif", package = "spDataLarge"))

qgisprocess::qgis_enable_plugins("processing_saga_nextgen")
qgisprocess::qgis_show_help("sagang:sagawetnessindex")

ep = qgisprocess::qgis_run_algorithm(
  alg = "sagang:sagawetnessindex",
  DEM = dem,
  SLOPE_TYPE = 1,
  SLOPE = tempfile(fileext = ".sdat"),
  AREA = tempfile(fileext = ".sdat"),
  .quiet = TRUE
)

2. native:buffer algorithm:

r
result <- qgis_run_algorithm("native:buffer", INPUT = random_points, DISTANCE = 50)

3. Manual Execution Using OSGeo4W Shell If the processing content of the sagang:sagawetnessindex algorithm is manually made into a JSON file, I named the file params.json here, and running the command in the OSGeo4W shell, the algorithm executes successfully:

C:\OSGeo4W\bin\qgis_process-qgis.bat --json run "sagang:sagawetnessindex" - < C:/Temp/params.json

The contents of params.json:


{
  "inputs": {
    "DEM": "C:/R_Library/spDataLarge/raster/dem.tif",
    "AREA": "C:/Temp/area.sdat",
    "SLOPE": "C:/Temp/slope.sdat",
    "AREA_MOD": "C:/Temp/area_mod.sdat",
    "TWI": "C:/Temp/twi.sdat",
    "SUCTION": 10,
    "AREA_TYPE": 0,
    "SLOPE_TYPE": 1,
    "SLOPE_MIN": 0,
    "SLOPE_OFF": 0.1,
    "SLOPE_WEIGHT": 1
  }
}

Software Configuration

  • QGIS Version: 3.42.2-Münster
  • qgisprocess Package Version: 0.4.1.9000
  • Plugin Enabled:
qgis_pluguins()
QuickOSM                TRUE
grassprovider           TRUE
processing              TRUE
processing_saga_nextgen TRUE
  • The configuration appears successful:
r
>qgisprocess::qgis_configure()
getOption('qgisprocess.path') was not found.
Trying Sys.getenv('R_QGISPROCESS_PATH'): 'C:/OSGeo4W/bin/qgis_process-qgis.bat'
Success!
Now using 'qgis_process' at 'C:/OSGeo4W/bin/qgis_process-qgis.bat'.
>>> If you need another installed QGIS instance, run `qgis_configure()`;
    see `?qgis_configure` if you need to preset the path of 'qgis_process'.

QGIS version is now set to: 3.42.2-Münster
Using JSON for output serialization.
Using JSON for input serialization.
4 out of 4 available processing provider plugins are enabled.
You now have access to 1306 algorithms from 8 QGIS processing providers.

Saving configuration to 'C:\R_Temp\R-qgisprocess\R-qgisprocess\Cache/cache-0.4.1.9000.rds'
Use qgis_algorithms(), qgis_providers(), qgis_plugins(), qgis_path() and
qgis_version() to inspect the cache environment.

The execution result always reports an error. I don't know where the configuration issue is, and it doesn't seem to be related to the algorithm used. Image

Minimal reproducible example

``

Backend versions

qgis_version(query = TRUE, debug = TRUE) [1] "3.42.2-Münster"

Using qgisprocess 0.4.1.9000

Versions reported by 'qgis_process':

QGIS 3.42.2-Münster 'Münster' (402273f8c2) QGIS code revision 402273f8c2 Qt version 5.15.13 Python version 3.12.10 GDAL/OGR version 3.10.3 PROJ version 9.6.0 EPSG Registry database version v12.004 (2025-03-02) GEOS version 3.13.1-CAPI-1.19.2 SQLite version 3.46.1 OS Windows 11 Version 2009

QGIS plugin state

qgis_plugins(query = TRUE)

A tibble: 4 × 2

name enabled
1 QuickOSM TRUE
2 grassprovider TRUE
3 processing TRUE
4 processing_saga_nextgen TRUE

R session info

sessioninfo::session_info() ─ Session info ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────── setting value version R version 4.2.2 (2022-10-31 ucrt) os Windows 10 x64 (build 22631) system x86_64, mingw32 ui RStudio language (EN) collate Chinese (Simplified)_China.utf8 ctype Chinese (Simplified)_China.utf8 tz Asia/Taipei date 2025-05-06 rstudio 2022.12.0+353 Elsbeth Geranium (desktop) pandoc NA

─ Packages ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── package * version date (UTC) lib source assertthat 0.2.1 2019-03-21 [3] CRAN (R 4.2.3) cachem 1.0.6 2021-08-19 [3] CRAN (R 4.2.2) callr 3.7.3 2022-11-02 [3] CRAN (R 4.2.2) class 7.3-20 2022-01-16 [1] CRAN (R 4.2.2) classInt 0.4-11 2025-01-08 [1] CRAN (R 4.2.2) cli 3.6.5 2025-04-23 [1] CRAN (R 4.2.2) codetools 0.2-18 2020-11-04 [1] CRAN (R 4.2.2) crayon 1.5.2 2022-09-29 [3] CRAN (R 4.2.2) DBI 1.2.3 2024-06-02 [1] CRAN (R 4.2.2) devtools 2.4.5 2022-10-11 [3] CRAN (R 4.2.3) digest 0.6.31 2022-12-11 [3] CRAN (R 4.2.2) dplyr 1.1.2 2023-04-20 [3] CRAN (R 4.2.3) e1071 1.7-16 2024-09-16 [1] CRAN (R 4.2.2) ellipsis 0.3.2 2021-04-29 [3] CRAN (R 4.2.2) fastmap 1.1.1 2023-02-24 [3] CRAN (R 4.2.3) fs 1.5.2 2021-12-08 [3] CRAN (R 4.2.2) generics 0.1.3 2022-07-05 [3] CRAN (R 4.2.2) glue 1.8.0 2024-09-30 [1] CRAN (R 4.2.2) htmltools 0.5.4 2022-12-07 [3] CRAN (R 4.2.2) htmlwidgets 1.6.2 2023-03-17 [3] CRAN (R 4.2.3) httpuv 1.6.11 2023-05-11 [3] CRAN (R 4.2.3) insight 0.19.3 2023-06-29 [3] CRAN (R 4.2.3) jsonlite 2.0.0 2025-03-27 [1] CRAN (R 4.2.2) KernSmooth 2.23-20 2021-05-03 [1] CRAN (R 4.2.2) later 1.3.1 2023-05-02 [3] CRAN (R 4.2.3) lattice 0.20-45 2021-09-22 [1] CRAN (R 4.2.2) lifecycle 1.0.4 2023-11-07 [1] CRAN (R 4.2.3) magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.2.2) memoise 2.0.1 2021-11-26 [3] CRAN (R 4.2.2) mime 0.12 2021-09-28 [3] CRAN (R 4.2.0) miniUI 0.1.1.1 2018-05-18 [3] CRAN (R 4.2.3) performance 0.10.4 2023-06-02 [3] CRAN (R 4.2.3) pillar 1.10.2 2025-04-05 [1] CRAN (R 4.2.2) pkgbuild 1.4.2 2023-06-26 [3] CRAN (R 4.2.3) pkgconfig 2.0.3 2019-09-22 [3] CRAN (R 4.2.2) pkgload 1.3.2.1 2023-07-08 [3] CRAN (R 4.2.3) plyr 1.8.8 2022-11-11 [3] CRAN (R 4.2.2) prettyunits 1.1.1 2020-01-24 [3] CRAN (R 4.2.2) pROC 1.18.5 2023-11-01 [1] CRAN (R 4.2.3) processx * 3.8.6 2025-02-21 [1] CRAN (R 4.2.2) profvis 0.3.8 2023-05-02 [3] CRAN (R 4.2.3) promises 1.2.0.1 2021-02-11 [3] CRAN (R 4.2.3) proxy 0.4-27 2022-06-09 [1] CRAN (R 4.2.2) ps 1.9.1 2025-04-12 [1] CRAN (R 4.2.2) purrr 1.0.1 2023-01-10 [3] CRAN (R 4.2.2) qgisprocess * 0.4.1.9000 2025-05-06 [1] Github (paleolimbot/qgisprocess@b8863a7) R6 2.6.1 2025-02-15 [1] CRAN (R 4.2.2) rappdirs 0.3.3 2021-01-31 [3] CRAN (R 4.2.2) raster * 3.6-26 2023-10-14 [3] CRAN (R 4.2.3) RColorBrewer 1.1-3 2022-04-03 [3] CRAN (R 4.2.0) Rcpp 1.0.14 2025-01-12 [1] CRAN (R 4.2.2) remotes 2.5.0 2024-03-17 [3] CRAN (R 4.2.3) rlang 1.1.6 2025-04-11 [1] CRAN (R 4.2.2) rstudioapi 0.15.0 2023-07-07 [3] CRAN (R 4.2.3) sessioninfo 1.2.2 2021-12-06 [3] CRAN (R 4.2.3) sf * 1.0-19 2024-11-05 [1] CRAN (R 4.2.2) shiny 1.7.5 2023-08-12 [3] CRAN (R 4.2.3) sp * 2.0-0 2023-06-22 [3] CRAN (R 4.2.3) stringi 1.8.7 2025-03-27 [1] CRAN (R 4.2.2) stringr 1.5.1 2023-11-14 [1] CRAN (R 4.2.3) terra 1.7-71 2024-01-31 [3] CRAN (R 4.2.3) tibble 3.2.1 2023-03-20 [3] CRAN (R 4.2.3) tidyselect 1.2.0 2022-10-10 [3] CRAN (R 4.2.2) units 0.8-5 2023-11-28 [1] CRAN (R 4.2.3) urlchecker 1.0.1 2021-11-30 [3] CRAN (R 4.2.3) usethis 2.2.2 2023-07-06 [3] CRAN (R 4.2.3) utf8 1.2.5 2025-05-01 [1] CRAN (R 4.2.2) vctrs 0.6.5 2023-12-01 [1] CRAN (R 4.2.3) withr 3.0.2 2024-10-28 [1] CRAN (R 4.2.2) xtable 1.8-4 2019-04-21 [3] CRAN (R 4.2.2) zip 2.3.0 2023-04-17 [3] CRAN (R 4.2.3)

[1] C:/R_Library [2] C:/Program Files/R/R-4.2.2/library

Additional context

No response

CC3729 avatar May 06 '25 06:05 CC3729

Thank you for your bug report, and for demonstrating that it works on the command line.

This needs attention, so hoping to look at it soon but I'm low on time resources. If I can't reproduce the problem locally (I only have access to a Win 10 system), I'll likely come back later with some things you can try out so that we can find where the problem sits. The essential part of qgis_run_algorithm() is that it calls qgis_run(), which creates the call that processx::run() must run on the command line. Somehow this gets stuck on your system with the JSON method.

For a start, you could run in verbose mode with qgis_run_algorithm(..., .quiet = FALSE) which might (or not) give an extra hint.

A probable workaround for you now, since the problem seems to be with the JSON format, is to not go via JSON at all, by setting options(qgisprocess.use_json_output = FALSE), run qgis_configure() again, and then try again. As this should prevent the usage of JSON input (using the legacy input method instead), the possibilities are more limited but you can still use most algorithms in this way.

Of course if someone else would already like to start debugging, then you're very welcome.

florisvdh avatar May 11 '25 09:05 florisvdh

I believe the issue might stem from how command-line calls handle file paths. Is it possible that the default file path contains characters that cause errors when executing commands or referencing parameter files? For instance, if the path includes spaces or Chinese characters (my Windows 11 system username is in Chinese, and the path is C:\Users\陈辰\AppData\Roaming), this could interfere with proper execution.

Additionally, I have checked the JSON file format using an online tool (https://jsonlint.com/), and no issues were detected. I also tried processing the JSON file locally using OSGeo4W shell, and the results were successfully obtained. I have also repeatedly tried running qgis_configure(), but it has not resolved the issue. This suggests that the problem might be related to how the JSON method is invoked in the current environment rather than an issue with the JSON format itself. 

For reference, I have attached the JSON file I used for testing to this email. I am happy to assist in further debugging!

登高望远 @.***

     Original

From:Floris Vanderhaeghe @.> Sent Time:2025-05-11- 17:55 To:r-spatial/qgisprocess @.> Cc:CC3729 @.>, Author @.> Subject:Re: [r-spatial/qgisprocess] Error parsing JSON format data inqgisprocess::qgis_run_algorithm() (Issue #225)

florisvdh left a comment (r-spatial/qgisprocess#225)

Thank you for your bug report, and for demonstrating that it works on the command line.

This needs attention, so hoping to look at it soon but I'm low on time resources. If I can't reproduce the problem locally (I only have access to a Win 10 system), I'll likely come back later with some things you can try out so that we can find where the problem sits. The essential part of qgis_run_algorithm() is that it calls qgis_run(), which creates the call that processx::run() must run on the command line. Somehow this gets stuck on your system with the JSON method.

For a start, you could run in verbose mode with qgis_run_algorithm(..., .quiet = FALSE) which might (or not) give an extra hint.

A probable workaround for you now, since the problem seems to be with the JSON format, is to not go via JSON at all, by setting options(qgisprocess.use_json_output = FALSE), run qgis_configure() again, and then try again. As this should prevent the usage of JSON input (using the legacy input method instead), the possibilities are more limited but you can still use most algorithms in this way.

Of course if someone else would already like to start debugging, then you're very welcome.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

CC3729 avatar May 12 '25 03:05 CC3729