How to wait until page is completely loaded

[yycc97]

Hello! First of all, I'm really appreciate your work about SeleniumVBA and I have used it for a while in my daily work.

Now I am trying to automate some web work using it and it works well. But sometimes, because the office system is very old and process speed can be very slow, so very often when webdriver has already proceed to the next step before the page is totally refreshed(the contents remain unchanged), the StaleElementReferenceException/No Such Element/ error occurs and even some procedures can be skipped without being processed.

To solve the problem, I have used different waits such as "implicit wait","WaitUntilDisplayd","MaxWait" but have no luck(since the contents in the page are same before and after reload so the wait is true?) Is there any other way to solve this problem?

Best regards

[GCuser99]

Hi @yycc97,

Thanks for the kind words of appreciation and sorry you are having problems.

It would help us help you if you can provide more detail around your system specs (OS version, Office version, Office bit-ness), which SeleniumVBA solution you are using (DLL, Excel .xlsm, or Access .accdb), and provide minimal reproducible code sample showing what you have tried. With that, I'm sure we can help you.

Cheers,

Mike

[yycc97]

Thank you for your prompt answer! I am using Excel.xlsm,

Below is my system details: OS Version(Windows11, 22621.3737) Microsoft Edge(127.0.2651.74 64bit) ps. as for saying office system, I mean my company's procurement system which can only be used on Microsoft Edge.

And here is my code: `Sub WebDriver_test() ...... driver.FindElement(by.XPath, "/html/body/form/div[4]/span/div/table/tbody/tr[1]/td/div[1]/div/table/tbody/tr/td[1]/table/tbody/tr[1]/td[2]/div/select").Click Set week = driver.FindElement(by.ID, "ctl00_MainContent_ReportViewer1_ctl04_ctl03_ddValue") week.SelectByValue "1" driver.WaitUntilDisplayed week(or WaitUntilNotDisplay that I have tried) -----before processing to the second step, there's possibility that page fresh can be very slow----- Set DataType = driver.FindElement(by.XPath, "/html/body/form/div[4]/span/div/table/tbody/tr[1]/td/div[1]/div/table/tbody/tr/td[1]/table/tbody/tr[2]/td[2]/div/div/input[1]") DataType.Click driver.Wait 1000 driver.ScrollTo 0, driver.GetScrollHeight / 2 driver.Wait 1000 driver.FindElement(by.ID, "ctl00_MainContent_ReportViewer1_ctl04_ctl07_divDropDown_ctl17").ScrollIntoView driver.Wait 1000 driver.ExecuteScript "var trans=document.getElementById('ctl00_MainContent_ReportViewer1_ctl04_ctl07_divDropDown_ctl15');" & "trans.click();" driver.Wait

End Sub`

And every time with the input and click, the page refreshes without contents changed. Please tell me if you need more details, thanks🙇‍♀️

Best regards

[GCuser99]

Thanks. By the way, to add a code block in GitHub issues or discussions, use three backticks at the front and end of the block. On my keyboard it's the key on upper left that shares with the "~" character. If you want color syntax, then add "VBA" after the first set of backticks. Like this:

```VBA
If mycode Then
```

Can you try this and let us know exactly where it fails, and also what was printed in your Debug window?

Sub WebDriver_test()
'......
driver.ImplicitMaxWait = 10000 '<--- maybe you already tried this?

driver.FindElement(by.XPath, "/html/body/form/div[4]/span/div/table/tbody/tr[1]/td/div[1]/div/table/tbody/tr/td[1]/table/tbody/tr[1]/td[2]/div/select").Click

driver.Wait 1000 'give the refresh a little time to get started

'Use implicit wait set above for page to refresh after Click event
Set week = driver.FindElement(by.ID, "ctl00_MainContent_ReportViewer1_ctl04_ctl03_ddValue")
Debug.Print "week element interactable?", week.IsDisplayed
driver.WaitUntilDisplayed week
Debug.Print "week element interactable?", week.IsDisplayed
week.SelectByValue "1"

driver.Wait 1000 'give the refresh a little time to get started

'-----before processing to the second step, there's possibility that page fresh can be very slow-----
'Use implicit wait set above for page to refresh after Input event
Set DataType = driver.FindElement(by.XPath, "/html/body/form/div[4]/span/div/table/tbody/tr[1]/td/div[1]/div/table/tbody/tr/td[1]/table/tbody/tr[2]/td[2]/div/div/input[1]")
Debug.Print "DataType element interactable?", DataType.IsDisplayed
driver.WaitUntilDisplayed DataType 
Debug.Print "DataType element interactable?", DataType.IsDisplayed
DataType.Click

driver.Wait 1000
driver.ScrollTo 0, driver.GetScrollHeight / 2
driver.Wait 1000
driver.FindElement(by.ID, "ctl00_MainContent_ReportViewer1_ctl04_ctl07_divDropDown_ctl17").ScrollIntoView
driver.Wait 1000
driver.ExecuteScript "var trans=document.getElementById('ctl00_MainContent_ReportViewer1_ctl04_ctl07_divDropDown_ctl15');" & "trans.click();"
driver.Wait

'As an aside, any reason why you are not do this instead of ExecuteScript?
'driver.FindElement(by.ID, "ctl00_MainContent_ReportViewer1_ctl04_ctl07_divDropDown_ctl15").Click
'......
End Sub

[GCuser99]

Edit to above - added a short explicit Wait after each Click/Input event to let the page refresh get started.

[yycc97]

Sorry to take so long to reply. As you can see, I don't have much coding experience so I don't know how to show you debug window cause I don't understand what those codes mean. Sorry for the inconvenience this may cause.

Anyway, I have tried your code. Most of the time, it goes well, but still, when the page loading time becomes longer, NoSuchElement Found error occurs. Also, I have tried VBA's function [Sleep] and it works somehow. So I guess the reason is SeleniumWebdriver is so fast that VBA always falls behind it? I don't know......

Sorry can't give you more details about this problem, but again your work helps a lot !!!

Cheers

[GCuser99]

Ok, glad that you have found a solution. Can you tell me what line(s) in the code the NoSuchElement error occurs? And also can you say where in the code you inserted the Window's API Sleep command to fix the problem?

Thanks!

[yycc97]

Sure. The error usually occurs in the codes as below:

    'input Item_No
    'Part 1
    driver.ImplicitMaxWait = 10000
    Set itemNo = driver.FindElement(by.XPath, "/html/body/form/div[4]/span/div/table/tbody/tr[1]/td/div[1]/div/table/tbody/tr/td[1]/table/tbody/tr[3]/td[2]/div/div/input[2]")
    Debug.Print "itemNo element interactable?", itemNo.IsDisplayed
    driver.WaitUntilDisplayed itemNo
    Debug.Print "itemType element interactable?", itemNo.IsDisplayed
    itemNo.Click
    driver.Wait 1000
    Set Item = driver.FindElementByName("ctl00_MainContent_ReportViewer1_ctl04_ctl11_ctl01")
    driver.SwitchToFrame Item
    driver.Wait
   '......
    Debug.Print "itemNo element interactable?", itemNo.IsDisplayed
    itemNo.Click
    driver.Wait 1000

    'Select item_summary
    'Part 2
    driver.ImplicitMaxWait = 10000
    Set itemsummary = driver.FindElement(by.ID, "ctl00_MainContent_ReportViewer1_ctl04_ctl09_ddValue")
    Debug.Print "itemsummary element interactable?", itemsummary.IsDisplayed
    driver.WaitUntilDisplayed itemsummary
    Debug.Print "itemsummary element interactable?", itemsummary.IsDisplayed
    itemsummary.Click
    driver.Wait 1000
    itemsummary.SelectByValue "1"
    driver.Wait 1000
    
    'Select company
    'Part 3
    driver.ImplicitMaxWait = 10000
    Set company = driver.FindElement(by.XPath, "/html/body/form/div[4]/span/div/table/tbody/tr[1]/td/div[1]/div/table/tbody/tr/td[1]/table/tbody/tr[1]/td[5]/div/div/input[1]")
    Debug.Print "company element interactable?", company.IsDisplayed
    driver.WaitUntilDisplayed company
    Debug.Print "company element interactable?", company.IsDisplayed
    company.Click
    driver.Wait 1000
    
    '-----Then the error occurs-----(No ToExcel Element: Unable to locate element)
    driver.FindElement(by.ID, "ctl00_MainContent_ReportViewer1_ctl04_ctl05_divDropDown_ctl02").Click
    driver.Wait
    driver.FindElement(by.ID, "ctl00_MainContent_ReportViewer1_ctl04_ctl05_divDropDown_ctl03").Click
    driver.Wait

Usually, after the Part 1 procession is over, the page refresh would take some time, so within the refresh time, the code seems to go to Part 3 straightforwardly and the error occurs cause Part 2's input action is neglected, that's why the code can't find the element(I guess). So I inserted the Sleep command in the line between Part 1 and 2 to stop the code from proceeding too fast.

By the way, I searched how to open the Debug window, and my Devtools's source doesn't show the Debug window, so maybe it's my company's computer setting so I can't use it? Thanks!

[GCuser99]

Ok, I now understand that you are using Office on the Web - not a desktop app. I will have to learn how to automate the Web version of Excel before I can replicate.

[6DiegoDiego9]

@GCuser99, unfortunately Excel on the web doesn't support VBA; it only supports Office Scripts, which are TypeScript/JavaScript that use the Office Scripts JavaScript APIs to interact with an Excel workbook.

@yycc97, just look for and open this view:

This is the Immediate window (aka Debug window), where your Debug.Print instructions print the output. We use this as a handy alternative to MsgBox when debugging code in VBA.

[GCuser99]

I was going off of this - not sure what it means now...

ps. as for saying office system, I mean my company's procurement system which can only be used on Microsoft Edge.

[yycc97]

Ok, I now understand that you are using Office on the Web - not a desktop app. I will have to learn how to automate the Web version of Excel before I can replicate.

Yes! I am using Microsoft 365. So glad that you have found the reason, it's a pity that Excel on the web doesn't support this.

I was going off of this - not sure what it means now...

ps. as for saying office system, I mean my company's procurement system which can only be used on Microsoft Edge.

Just ignore this and sorry for the confusion.

@GCuser99 @6DiegoDiego9 Thank you for teaching me and I have learned a lot👍😁

[GCuser99]

@yycc97, I use Microsoft Office 365 Desktop and have not had any problem. "Microsoft 365 Office on the Web" is a different version that runs through a browser like Edge or Chrome. When you are using Excel, are you using it through a browser?

Thanks for your patience,

Mike

[yycc97]

Sure, I always appreciate your help. Yes, I am using it through a browser. And I tried your code in Microsoft Office 365 Desktop today and it worked smoothly.

[GCuser99]

Oh, that is interesting! Thanks for letting us know @yycc97.