venom icon indicating copy to clipboard operation
venom copied to clipboard
verified publisher icon ovh

Executor Web - Refactoring

Open kevinramage opened this issue 3 years ago • 4 comments

Change the web driver client to a new one (now, agouti is now maintain).

  • Facilitate integration with proxy, headless
  • Improve log management (Possibility to see REST call between web driver and browser in debug mode)
  • Improve error management
  • Add new synchronisation features
  • Add Javascript code feature
  • Compatible with W3C

kevinramage avatar May 12 '22 13:05 kevinramage

Hi, Your branch has some conflict, could you rebase it ? It seems that the integration test tests/web.yml is failing with a panic:

• TestSuite Web (./web.yml)
 	• TestCase-Get-URL-and-check-titlepanic: runtime error: invalid memory address or nil pointer dereference
goroutine 19 [running]:
runtime/debug.Stack()
	/usr/local/go/src/runtime/debug/stack.go:24 +0x65
github.com/confluentinc/bincover.RunTest.func1()
	/go/pkg/mod/github.com/confluentinc/[email protected]/instrument_bin.go:84 +0x39
panic({0x12fe3e0, 0x1db9010})
	/usr/local/go/src/runtime/panic.go:1038 +0x215
github.com/ovh/venom/executors/web.Executor.Run({{0x0, {0x0, 0x0, 0x0}, {0x0, 0x0}, 0x0, 0x0, 0x0, 0x0, ...}, ...}, ...)
	/workspace/executors/web/web.go:152 +0xc6
github.com/ovh/venom.executor.Run({{0x15ddcc0, 0xc000342e10}, {0xc0001b1c4a, 0x3}, 0x0, {0x1ff49e8, 0x0, 0x0}, 0x0, 0x0, ...}, ...)
	/workspace/types_executor.go:130 +0x5c
github.com/ovh/venom.(*Venom).runTestStepExecutor(0xc000342b40, {0x15f9340, 0xc0024f0bd0}, {0x1614820, 0xc001194d80}, 0xc0002c6c00, 0xc0024f04b0)
	/workspace/process_teststep.go:143 +0x24b
github.com/ovh/venom.(*Venom).RunTestStep(0xc000342b40, {0x15f9340, 0xc0024f0ba0}, {0x1614820, 0xc001194d80}, 0xc0002c6c00, 0x2, 0xc0024f04b0)
	/workspace/process_teststep.go:36 +0x290
github.com/ovh/venom.(*Venom).runTestSteps(0xc000ead288, {0x15f9340, 0xc001bff140}, 0xc0002c6c00)
	/workspace/process_testcase.go:252 +0xf28
github.com/ovh/venom.(*Venom).runTestCase(0x145107b, {0x15f9340, 0xc001bff050}, 0xc000412aa8, 0xc0002c6c00)
	/workspace/process_testcase.go:138 +0x4d1
github.com/ovh/venom.(*Venom).runTestCases(0xc000342b40, {0x15f9340, 0xc001bff050}, 0xc000412aa8)
	/workspace/process_testsuite.go:87 +0x405
github.com/ovh/venom.(*Venom).runTestSuite(0xc000342b40, {0x15f92d0, 0xc0000de000}, 0xc000412aa8)
	/workspace/process_testsuite.go:67 +0xbaa
github.com/ovh/venom.(*Venom).Process(0xc000342b40, {0x15f92d0, 0xc0000de000}, {0xc0003c2c00, 0xc0000de000, 0xc000396600})
	/workspace/process.go:153 +0x145
github.com/ovh/venom/cmd/venom/run.glob..func2(0x1dc6260, {0xc0003c2c00, 0x5c, 0x60})
	/workspace/cmd/venom/run/cmd.go:402 +0x785
github.com/spf13/cobra.(*Command).execute(0x1dc6260, {0xc0003c2600, 0x5c, 0x60})
	/go/pkg/mod/github.com/spf13/[email protected]/command.go:850 +0x60e
github.com/spf13/cobra.(*Command).ExecuteC(0x1dc5fc0)
	/go/pkg/mod/github.com/spf13/[email protected]/command.go:958 +0x3ad
github.com/spf13/cobra.(*Command).Execute(...)
	/go/pkg/mod/github.com/spf13/[email protected]/command.go:895
github.com/ovh/venom/cmd/venom.main()
	/workspace/cmd/venom/main.go:20 +0x31
github.com/confluentinc/bincover.RunTest(0x14d0890)
	/go/pkg/mod/github.com/confluentinc/[email protected]/instrument_bin.go:93 +0x21b
github.com/ovh/venom/cmd/venom.TestBincoverRunMain(0x0)
	/workspace/cmd/venom/main_test.go:12 +0x25
testing.tRunner(0xc00025bd40, 0x14d0888)
	/usr/local/go/src/testing/testing.go:1259 +0x102
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1306 +0x35a

fsamin avatar May 19 '22 08:05 fsamin

Hi @kevinramage , thank you for this refactoring.

I didn't manage to run the example:

./venom run w.yml
	  [trac] writing venom.25.log
 • TestSuite Web (w.yml)
 	• TestCase-Google-searchStarting ChromeDriver 104.0.5112.79 (3cf3e8c8a07d104b9e1260c910efb8f383285dc5-refs/branch-heads/5112@{#1307}) on port 9515
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
 FAILURE
Unable create new session: an error occured during create session action: an error occured during post request: session - Invalid status code 500
Unable to start web driver: an error occured during session starting: port 9515 not available
Unable to start web driver: an error occured during session starting: port 9515 not available
Unable to start web driver: an error occured during session starting: port 9515 not available

with the testsuite (from readme)

name: TestSuite Web
vars:
  web:
    driver: chrome
    width: 1920
    height: 1080
    args:
    - 'browser-test'
    prefs:
      profile.default_content_settings.popups: 0
      profile.default_content_setting_values.notifications: 1
    timeout: 60
    debug: true
testcases:
- name: TestCase Google search
  steps:
  - type: web
    action:
      navigate:
        url: https://www.google.fr
    assertions:
    - result.title ShouldEqual Google
    - result.url ShouldEqual https://www.google.fr
  - type: web
    action:
      find: input[name="q"]
    assertions:
     - result.find ShouldEqual 1
  - type: web
    action:
      fill:
      - find: input[name="q"]
        text: "venom ovh"
  - type: web
    action:
      click:
        find: input[value="Recherche Google"]
        wait: 1
    screenshot: googlesearch.png

did I miss something somewhere?

yesnault avatar Aug 03 '22 16:08 yesnault

Hello @yesnault ,

You probably have an application running on port 9515, can you check this point with a command like "ps -ef" or "netstat -ano" ? I update example code provided in README

Regards

kevinramage avatar Aug 22 '22 14:08 kevinramage

Looks really nice! Would love to see it merged!

buger avatar Sep 01 '22 14:09 buger

Hi @yesnault, do you always have an issue with port unavailable ?

kevinramage avatar Oct 12 '22 20:10 kevinramage

chromedriver updated, I have another err now:

❯ ./venom run w.yml
	  [trac] writing venom.6.log
 • TestSuite Web (w.yml)
 	• TestCase-Google-searchStarting ChromeDriver 106.0.5249.61 (511755355844955cd3e264779baf0dd38212a4d0-refs/branch-heads/5249@{#569}) on port 58636
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
Starting ChromeDriver 106.0.5249.61 (511755355844955cd3e264779baf0dd38212a4d0-refs/branch-heads/5249@{#569}) on port 58640
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
Starting ChromeDriver 106.0.5249.61 (511755355844955cd3e264779baf0dd38212a4d0-refs/branch-heads/5249@{#569}) on port 58644
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
Starting ChromeDriver 106.0.5249.61 (511755355844955cd3e264779baf0dd38212a4d0-refs/branch-heads/5249@{#569}) on port 58648
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
 FAIL
Unable to resize the page: failed to retrieve window: request unsuccessful: invalid session id
Unable to resize the page: failed to retrieve window: request unsuccessful: invalid session id
Unable to resize the page: failed to retrieve window: request unsuccessful: invalid session id
Unable to resize the page: failed to retrieve window: request unsuccessful: invalid session id
final status: FAIL

yesnault avatar Oct 18 '22 15:10 yesnault

Hello @yesnault, Thanks for this feedback I try to reproduce the issue and I don't reproduce it in my side Can you set log level to Debug (Set web.debug to true OR set web.logLevel to DEBUG) and check file named venom.x.log ? In this refacto, you can see all rest calls. With the rest calls, we can in details the root cause of this issue.

kevinramage avatar Oct 25 '22 20:10 kevinramage

@yesnault : do you use the same version between chromedriver and chromium?

some useful links:

  • https://chromedriver.chromium.org/downloads/version-selection
  • https://omahaproxy.appspot.com/ (to find a revision by version)

jpatovh avatar Mar 26 '23 12:03 jpatovh

@yesnault I retry code, I can run web tests in my side. Do yo you always have the issue ? Do you have the good version of the driver ?

kevinramage avatar Apr 21 '23 20:04 kevinramage

Hi @kevinramage, I'll recheck with @blaryjp hints this week with same chromium and chrome driver version.

yesnault avatar Apr 25 '23 09:04 yesnault

@kevinramage ok, so the error is now clear:

Apr 25 11:32:31.907 [DEBU] Response body: {"value":{"error":"unknown error","message":"unknown error: cannot find Chrome binary","stacktrace":"0   chromedriver                        0x0000000109a13ae8 chromedriver + 4958952\n1   chromedriver                        0x0000000109a0b7b3 chromedriver + 4925363\n2   chromedriver                        0x00000001095cd1d6 chromedriver + 475606\n3   chromedriver                        0x00000001095f755c chromedriver + 648540\n4   chromedriver                        0x00000001095f5069 chromedriver + 639081\n5   chromedriver                        0x00000001096406e9 chromedriver + 947945\n6   chromedriver                        0x000000010963fa7f chromedriver + 944767\n7   chromedriver                        0x00000001096369e3 chromedriver + 907747\n8   chromedriver                        0x0000000109600aef chromedriver + 686831\n9   chromedriver                        0x000000010960200e chromedriver + 692238\n10  chromedriver                        0x00000001099cf9ad chromedriver + 468...

I don't have Chrome (but chromium ok). In you lib, can you add the possibility to specify the chromium / chromium installation on the chromedriver options (options.setBinary, cf. https://sites.google.com/a/chromium.org/chromedriver/capabilities#TOC-Using-a-Chrome-executable-in-a-non-standard-location) ? Do you test it on osx too ?

Thank you,

yesnault avatar Apr 25 '23 09:04 yesnault

Thank you for the investigation.

I agree with Yvonnick about the possibility to specify the CHROME_PATH/CHROMIUM_PATH. Is it also possible to pass the CHROMEDRIVER_PATH also? I've it in a custom folder on my side.

jpatovh avatar Apr 26 '23 09:04 jpatovh

Thanks for this feedback, I will check soon this point

kevinramage avatar Apr 26 '23 11:04 kevinramage

Hi @yesnault @blaryjp I see your merge request to "kevinramage-web-refacto" branch, I add corrections in this PR #663 I add options to set browser binary, driver binary and driver port. VenomWeb dependency manage a new restriction about relative path. Regards,

kevinramage avatar May 01 '23 19:05 kevinramage

I backported from #663 executor web updates (closing #663, not rebase master)

yesnault avatar May 02 '23 14:05 yesnault

@kevinramage @blaryjp It's ok now, thank you.

❯ venom run -v web.yml
	  [trac] writing venom.2.log
 • TestSuite Web (web.yml)
 	• TestCase-Get-URL-and-check-title
 		• web PASS
 		• web PASS
 		• web PASS
 		• web PASS
	  	  [info] title is '' (web.yml:24)
final status: PASS

I check integration test

yesnault avatar May 02 '23 15:05 yesnault

@kevinramage @blaryjp thank you, it's ok now

❯ venom run -v web.yml
	  [trac] writing venom.2.log
 • TestSuite Web (web.yml)
 	• TestCase-Get-URL-and-check-title
 		• web PASS
 		• web PASS
 		• web PASS
 		• web PASS
	  	  [info] title is '' (web.yml:24)
final status: PASS

yesnault avatar May 02 '23 15:05 yesnault