Unit and integration tests

[renkun-ken]

Currently, we only have very limited testing of the features of vscode-R.

We might need to put a demo R package and R script folder in the repo and run tests with opening the folder as a workspace folder and test at least the following features:

• Language server startup
• Session watcher
• R markdown commands

[gowerc]

Just want to say that I think given the size of this project now that it would be valuable to add more tests mostly for the obvious reasons

• Tests act as proof that someones contribution is working as expected
• Tests prevent regressions and alert us to breakages in the extension

That being said testing in vscode does not appear to be straight forward. One issue in particular seems to be that you can't switch between workspaces (which I figured would be critical for us testing different setups/configurations).

Having said that it seems to be easy enough to launch the tests on a single specific directory though by editing the launch.json file to specify the target directory:

    {
      "name": "Extension Tests",
      "type": "extensionHost",
      "request": "launch",
      "runtimeExecutable": "${execPath}",
      "args": [
        "${workspaceFolder}/test/sample-project",   // <--------------------------
        "--extensionDevelopmentPath=${workspaceFolder}",
        "--extensionTestsPath=${workspaceFolder}/out/test/suite/index"
      ],
      "outFiles": [
        "${workspaceFolder}/out/test/**/*.js"
      ],
      "preLaunchTask": "npm: pretest"
    }

Potentially one idea could be to have multiple test scripts each loading a separate directory and then placing conditions on test suites to only fire if a certain directory is active ?

Likewise I'm happy to help writing some of these tests as it will be a good opportunity to learn, that being said I will need some help / pointers as I've really been struggling to get my head around what is even possible with the vscode api sorry :(

[github-actions[bot]]

This issue is stale because it has been open for 365 days with no activity.

[ElianHugh]

unstale

[renkun-ken]

Currently, the testing no longer works in all recent PRs (#1290, #1296) where test fails with TypeError: util.config is not a function. No clue how to resolve this. Tried for a while but nothing worked.

[ElianHugh]

just spitballing - is it possible that there's a namespace collision between util.ts and the node util module?

[renkun-ken]

Tried renaming util to utils but no luck.

[ElianHugh]

It's interesting that it's occurring just with knit.ts... I'll try to play around with this and see if something works

Edit:

Some interesting links:

• https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
• https://github.com/GabrielBB/xvfb-action/issues/30

Could have something to do with the headless tests? I can't replicate the issue locally

[renkun-ken]

I can't replicate it, either. Would be nice if we could make it work somehow again.

[ElianHugh]

I believe I have a fix! Will PR shortly

Edit: See #1302

[ElianHugh]

An issue I've found when approaching unit testing rmd knitting is that I have to mock the vscode API (e.g. activeTextEditor). There doesn't seem to be a straightforward way of mocking?

[ElianHugh]

we can probably leverage testthat or similar packages within child processes for consistency?

[github-actions[bot]]

This issue is stale because it has been open for 365 days with no activity.

[ElianHugh]

unstale