opensearch-api-specification icon indicating copy to clipboard operation
opensearch-api-specification copied to clipboard

[BUG] query_insights tests flaky

Open dblock opened this issue 1 year ago • 4 comments

What is the bug?

From https://github.com/opensearch-project/opensearch-api-specification/actions/runs/12242182451/job/34148934118?pr=711

FAILED  insights/top_queries.yaml (tests/plugins/query_insights/insights/top_queries.yaml)
    FAILED  CHAPTERS
        PASSED  Retrieve default top queries.
            PASSED  REQUEST BODY
            PASSED  RESPONSE STATUS
            PASSED  RESPONSE PAYLOAD BODY
            PASSED  RESPONSE PAYLOAD SCHEMA
            SKIPPED RESPONSE OUTPUT VALUES
        FAILED  Retrieve top queries by latency.
            PASSED  PARAMETERS
                PASSED  type
            PASSED  REQUEST BODY
            PASSED  RESPONSE STATUS
            PASSED  RESPONSE PAYLOAD BODY
            FAILED  RESPONSE PAYLOAD SCHEMA (data/top_queries/0/source/query/bool/must/0/match/title/zero_terms_query MUST be equal to one of the allowed values: all, none, data/top_queries/0/source/query/bool/must must be object, data/top_queries/0/source/query/bool/must must be object, data/top_que, ...)

How can one reproduce the bug?

Unclear.

What is the expected behavior?

Tests to pass consistently.

What is your host/environment?

CI/CD GHA

Do you have any screenshots?

Screenshot 2024-12-09 at 1 48 38 PM

Do you have any additional context?

Likely a response that varies if you query too slow/fast/out of order.

dblock avatar Dec 09 '24 18:12 dblock

@ansjcy if you have any idea what this API intermittently returns that doesn't match the spec (a null?) and can correct it, that would be much appreciated.

dblock avatar Dec 16 '24 17:12 dblock

Thanks for filing the bug! let us try to reproduce it.

ansjcy avatar Jan 08 '25 05:01 ansjcy

I think the reason could be the source query container object in core api spec https://github.com/opensearch-project/opensearch-api-specification/pull/625/files#diff-f5951672705e3c806f3629fd05a4bb778b068f061a146005f7dc551942033774R140 is missing certain fields for zero_terms_query ? This is similar to https://github.com/opensearch-project/opensearch-api-specification/pull/641/files.

ansjcy avatar Jan 13 '25 22:01 ansjcy

But why is it intermittent?

dblock avatar Jan 13 '25 23:01 dblock