amphtml "A more real AMPHTML image ad" test flaky in INABOX

Integration tests in test-amphtml-ads.js are flaky on Headless Chrome in experiment B (INABOX_LITE):

DESCRIBE => A more real AMPHTML image ad
  DESCRIBE => ATF within friendly frame or safe frame
    DESCRIBE =>  
      IT => should properly render ad in a friendly iframe with viewability pings
        ✗ Error: Timeout of 15000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
●
      IT => should properly render ad in a safe frame with viewability pings
        ✗ Error: Timeout of 15000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
●
  DESCRIBE => No Host Script within friendly frame or safe frame
    DESCRIBE =>  
      IT => should properly render ad in a friendly iframe with viewability pings
        ✗ Error: Timeout of 15000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
●
      IT => should properly render ad in a safe frame with viewability pings
        ✗ Error: Timeout of 15000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

[23:25:01] HeadlessChrome 76.0.3809 (Linux 0.0.0): Executed 266 of 440 (Skipped 174) 4 FAILED

Two flakes on master today.

https://travis-ci.org/ampproject/amphtml/builds/582846281 https://travis-ci.org/ampproject/amphtml/builds/582910186

/to @lannka

Sep 09 '19 23:09 dreamofabear

Tried the tests locally, they fail reliably with the following recurring error in the console:

Sep 10 '19 19:09 zombifier

thanks @zombifier for the screenshot! I know what happened. Will fix.

Sep 10 '19 19:09 lannka

Looks like this is failing again on master under "Experiment B Tests". :(

DESCRIBE => A more real AMPHTML image ad
  DESCRIBE => ATF within friendly frame or safe frame
    DESCRIBE =>  
      IT => should properly render ad in a friendly iframe with viewability pings
        ✗ Error: Timeout of 15000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
●
      IT => should properly render ad in a safe frame with viewability pings
        ✗ Error: Timeout of 15000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
●
  DESCRIBE => No Host Script within friendly frame or safe frame
    DESCRIBE =>  
      IT => should properly render ad in a friendly iframe with viewability pings
        ✗ Error: Timeout of 15000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
●
      IT => should properly render ad in a safe frame with viewability pings
        ✗ Error: Timeout of 15000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

[21:47:58] HeadlessChrome 77.0.3865 (Linux 0.0.0): Executed 267 of 442 (Skipped 175) 4 FAILED

https://travis-ci.org/ampproject/amphtml/jobs/585777965

Sep 16 '19 22:09 dreamofabear

woops, let me mute it.

Sep 16 '19 22:09 lannka

Reopening because this test is repeatedly failing again. E.g. https://app.circleci.com/pipelines/github/ampproject/amphtml/1549/workflows/cdbc541d-1fdb-467a-9a28-6612a08f542f/jobs/14788/parallel-runs/0/steps/0-108

/cc @ampproject/wg-monetization

Feb 09 '21 20:02 rsimha

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

Aug 12 '22 00:08 stale[bot]

"A more real AMPHTML image ad" test flaky in INABOX_LITE experiment