Internet Explorer "scrape"

[darbid]

option could you please check again

myHTMLDoc.all.Item("REPORT").Value = 2

I took your sample HTML and put it into a html file and then ran it and it works for me.

Below I have added the html file I used.

You need to change the extension back to html - then as a test make your webbrowser control url the path to this file.

Then run my suggestion. as above.

 

[option]

You're method does work for that html doc you provided, along with a few other sites I've tried(this forums search page was also successfully automated on my end). I suspect that since the form name on the webpage is the same name as the combo box, except they're cased different. The form is "report" and the combo box is "REPORT". Im guessing the code is trying to set the form value to 2, not the combo box. This would explain the "object doesn't support this property or method" error.

 

[MarkK]

You can also handle the DocumentComplete event of the WebBrowser control and implement conditional page navigation. For example, say your code clicks a button on a page. The DocuementComplete event is fired by the browser control for each document it loads. You can handle this event and with Select Case statements determine what page loaded, read or write data from or to that page, and click another button, and so on, and creating pretty sophisticated page navigating screen scrapers. Here's some code you might find interesting that I use to scrape a real-estate site for listings...

[FONT="Verdana"][SIZE="1"]Private WithEvents m_wb As SHDocVw.WebBrowser

Private m_param As cSearchParam

'********************************************************************
'*                                                                                                   Jun 26 2009
'*       Properties>
'*
'********************************************************************
Property Get SearchParam() As cSearchParam
'  Custom class that provides search parameters for property searches
   If m_param Is Nothing Then Set m_param = New cSearchParam
   Set SearchParam = m_param
End Property

'********************************************************************
'*                                                                                                   Jun 26 2009
'*       Form Events>
'*
'********************************************************************
Private Sub Form_Load()
   Set m_wb = Me.WebBrowser0.Object
End Sub

Private Sub Form_Resize()
   With Me.WebBrowser0
      .Width = Me.InsideWidth
      .Height = Me.InsideHeight
   End With
End Sub

'********************************************************************
'*                                                                                                   Jun 26 2009
'*       Methods>
'*
'********************************************************************
Sub StartScrape()
   Dim URL As String
   URL = "http://www.someurl.com/propertySearch.aspx"
   m_wb.Navigate URL
End Sub

'********************************************************************
'*                                                                                                   Jun 24 2009
'*       Web Browser Events>
'*
'********************************************************************
Private Sub m_wb_DocumentComplete(ByVal pDisp As Object, URL As Variant)
On Error GoTo handler
   Dim baseURL As String
   
   'separate query string from the url
   baseURL = Split(URL, "?")
   
   'do actions based on the url
   Select Case baseURL(0)
      Case "http://www.someurl.com/propertySearch.aspx"
         'do actions based on the querystring
         Select Case baseURL(1)
            Case "action=search"   'searching
               With pDisp.Document
                  .getElementById("_ctl0__ctl0_elLocation_elProvinces").Value = 3
                  .getElementById("_ctl0__ctl0_elLocation_txtCity").Value = Me.SearchParam.CityList
                  .getElementById("_ctl0__ctl0_elFinancialDetails_elMinPrice").Value = Me.SearchParam.PriceLow
                  .getElementById("_ctl0__ctl0_elFinancialDetails_elMaxPrice").Value = Me.SearchParam.PriceHigh
                  .getElementById("_ctl0__ctl0_elBuilding_elBeds").Value = "3-0"
                  .getElementById("_ctl0__ctl0_elFinancialDetails_elOwnershipType").Value = 1
                  .getElementById("_ctl0__ctl0_elSortResults_ddlPageSize").Value = 50
                  .getElementById("_ctl0__ctl0_btnSubmit").Click
               End With
            Case "action=details"   'property details
               With pDisp.Document
                  .getElementById("txtMlsNumber").Value = m_item.MLS
                  .getElementById("lnkMlsSearch").Click
               End With
         End Select
      
      Case "http://www.someurl.com/Disclaimer.aspx"
         pDisp.Document.getElementById("_ctl0_lnkAccept").Click
         
      Case "http://www.someurl.com/PropertyResults.aspx"
         Select Case "q=listing"
            Case 0
               ParsePage pDisp.Document.body.innerHTML
               DoCmd.Close acForm, Me.Name
         End Select
      
      Case "http://www.someurl.com/PropertyDetails.aspx"
         Me.Visible = True
         
   End Select
   Exit Sub

handler:
   App.Log.AddError err, Me, "DocumentComplete()", True
   
End Sub


'********************************************************************
'*                                                                                                   Jun 26 2009
'*       Utilities>
'*
'********************************************************************
Private Sub ParsePage(text As String)
On Error GoTo handler
   'string processing here to store data from page HTML to tables
   Exit Sub
   
handler:
   App.Log.AddError err, Me, "ParsePage()", True

End Sub[/SIZE][/FONT]

 

[darbid]

You're method does work for that html doc you provided, along with a few other sites I've tried(this forums search page was also successfully automated on my end). I suspect that since the form name on the webpage is the same name as the combo box, except they're cased different. The form is "report" and the combo box is "REPORT". Im guessing the code is trying to set the form value to 2, not the combo box. This would explain the "object doesn't support this property or method" error.

ok then you have a couple of other options.

1. getelementbyID("select1").Value = 2 (I think)

and then if you really want to use this. We could try to give it at least focus and then use sendkeys to send it a value.

If I were you, to test out your theory that it is getting another object for report set an object to the "report" and then set a break point at this point and see what you can add to it like to see its ID or if it is clickable or what other properties it has, what its parent element is. If this does not make sense I will write something for you.

 

[option]

darbid - that seems to have done the trick! thank you for your help once again!