cargo test fails on archive from crates.io

[igor-raits]

error: couldn't read src/css-parsing-tests/component_value_list.json: No such file or directory (os error 2)
   --> src/tests.rs:112:20
    |
112 |     run_json_tests(include_str!("css-parsing-tests/component_value_list.json"), |input| {
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: couldn't read src/css-parsing-tests/one_component_value.json: No such file or directory (os error 2)
   --> src/tests.rs:120:20
    |
120 |     run_json_tests(include_str!("css-parsing-tests/one_component_value.json"), |input| {
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: couldn't read src/css-parsing-tests/declaration_list.json: No such file or directory (os error 2)
   --> src/tests.rs:131:20
    |
131 |     run_json_tests(include_str!("css-parsing-tests/declaration_list.json"), |input| {
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: couldn't read src/css-parsing-tests/one_declaration.json: No such file or directory (os error 2)
   --> src/tests.rs:141:20
    |
141 |     run_json_tests(include_str!("css-parsing-tests/one_declaration.json"), |input| {
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: couldn't read src/css-parsing-tests/rule_list.json: No such file or directory (os error 2)
   --> src/tests.rs:149:20
    |
149 |     run_json_tests(include_str!("css-parsing-tests/rule_list.json"), |input| {
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: couldn't read src/css-parsing-tests/stylesheet.json: No such file or directory (os error 2)
   --> src/tests.rs:159:20
    |
159 |     run_json_tests(include_str!("css-parsing-tests/stylesheet.json"), |input| {
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: couldn't read src/css-parsing-tests/one_rule.json: No such file or directory (os error 2)
   --> src/tests.rs:169:20
    |
169 |     run_json_tests(include_str!("css-parsing-tests/one_rule.json"), |input| {
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: couldn't read src/css-parsing-tests/stylesheet_bytes.json: No such file or directory (os error 2)
   --> src/tests.rs:197:24
    |
197 |     run_raw_json_tests(include_str!("css-parsing-tests/stylesheet_bytes.json"),
    |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: couldn't read src/css-parsing-tests/color3.json: No such file or directory (os error 2)
   --> src/tests.rs:348:21
    |
348 |     run_color_tests(include_str!("css-parsing-tests/color3.json"), |c| c.ok().to_json())
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: couldn't read src/css-parsing-tests/color3_hsl.json: No such file or directory (os error 2)
   --> src/tests.rs:354:21
    |
354 |     run_color_tests(include_str!("css-parsing-tests/color3_hsl.json"), |c| c.ok().to_json())
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: couldn't read src/css-parsing-tests/color3_keywords.json: No such file or directory (os error 2)
   --> src/tests.rs:361:21
    |
361 |     run_color_tests(include_str!("css-parsing-tests/color3_keywords.json"), |c| c.ok().to_json())
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: couldn't read src/css-parsing-tests/An+B.json: No such file or directory (os error 2)
   --> src/tests.rs:367:20
    |
367 |     run_json_tests(include_str!("css-parsing-tests/An+B.json"), |input| {
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: couldn't read src/css-parsing-tests/urange.json: No such file or directory (os error 2)
   --> src/tests.rs:377:20
    |
377 |     run_json_tests(include_str!("css-parsing-tests/urange.json"), |input| {
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: couldn't read src/css-parsing-tests/component_value_list.json: No such file or directory (os error 2)
   --> src/tests.rs:403:20
    |
403 |     run_json_tests(include_str!("css-parsing-tests/component_value_list.json"), |input| {
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to 14 previous errors

[emilio]

I think this was done on purpose on a9497fb3 to not bloat the archive

[emilio]

(Not that this is necessarily wontfix I guess, just pointing that out)

This was done in #116 to help with https://bugzilla.mozilla.org/show_bug.cgi?id=1336655.

[SimonSapin]

@ignatenkobrain Why is it useful to run tests on the crates.io archive? PRs are merged into master after tests pass on Travis-CI.

[igor-raits]

@SimonSapin, it is useful to verify that crate package within distribution works correctly.

@emilio, I don't really against excluding test files. But if you do that, please make sure that such tests are not run instead of having compilation errors.

[SimonSapin]

@ignatenkobrain There are a few unit tests that don’t rely on these JSON files, so they could run from the crates.io archive. I’m not convinced that doing that (instead of just verifying that the crate builds) is worthwhile, but I’d take a PR that adds a json-tests Cargo feature enabled by default and adds corresponding #[cfg(feature = "json-tests")] attributes on the relevant tests.

That way you could run cargo test --no-default-features from crates.io archives. To try it without publishing a new version, run cargo package then cd into target/package/cssparser-*/.

[emilio]

Yeah I don't see a much better way to do it than what @SimonSapin suggested above... We could check the existence of the json file, but that'd risk tests actually passing on automation if those are deleted, which is something I'd like to avoid.