Screen Scraping to Access (1 Viewer)

[hullstorage]

Hi all,

Is there a way of screen scarping a website to give me the results on my form

this will be based from a postcode to postcode mileage result

thanks all

 

[HiArt]

Well, yes and no. It kind of depends!

I screen scrape regularly using the WebBrowser object in VB.net. So it is possible.

If the website you are looking at is simple-ish HTML, then save the website to a text file and write some code to navigate around the HTML tags.

If you need to enter passwords and such, you may need to either a) download the free version of Visual Studio and and use the WebBrowser object (Visual Studio has database tie-ins), or b) find a nice friendly programmer to cut some code for you.

But, if the site is heavy PHP, Javascript thingy then good luck! Unfortunatly, most sites are going that way to prevent screen-scraping. Still possible (I think, but it defeats me) but orders of magnitude harder!

Art

 

[hullstorage]

http://www.postcode.org.uk/country/uk/_postcode-distance-calculator.asp

 

[gemma-the-husky]

hullstorage

slightly different approach

i have formed a url string to request the info from google (easy to do, i now see), but i need a way of extracting the lat/long from the file.

not sure how to do this last bit - ie without displaying the url target - any thoughts anyone?

 

[gemma-the-husky]

heres what i've got

in this string, replace address_goes_here with the required address
replace my_google_key_goes_here with your google key

now entering this string in a web browser

http://maps.google.co.uk/maps/geo?q=address_goes_here&output=xml&key=" & my_google_key_goes_here

displays a xml(kml) file including the coordinates

and entering

http://maps.google.co.uk/maps/geo?q=address_goes_here&output=csv&key=" & my_google_key_goes_here

displays a oneline csv that looks like

200,8,latitude,longitutde

But how do i get this info into my app, instead of displaying it on a screen.

 

[ByteMyzer]

[COLOR="Navy"]Dim[/COLOR] bResponse() [COLOR="navy"]As Byte[/COLOR]
[COLOR="navy"]Dim[/COLOR] sResponse [COLOR="navy"]As String[/COLOR]
[COLOR="navy"]Dim[/COLOR] XMLHTTP [COLOR="navy"]As Object[/COLOR]

[COLOR="navy"]Set[/COLOR] XMLHTTP = CreateObject("MSXML2.XMLHTTP")

XMLHTTP.Open "GET", "http://maps.google.co.uk/maps/geo" _
    & "?q=address_goes_here&output=csv&key=" _
    & my_google_key_goes_here, [COLOR="navy"]False[/COLOR]
XMLHTTP.send
bResponse = XMLHTTP.responseBody

[COLOR="navy"]Set[/COLOR] XMLHTTP = [COLOR="navy"]Nothing[/COLOR]

sResponse = StrConv(bResponse, vbUnicode)
[COLOR="DarkGreen"]' sResponse now contains the CSV string[/COLOR]

 

[motleyjew]

I use the scripting edition of Imacros. The free version will not give you everything that is needed.

http://www.iopus.com/

Gregg

 

[gemma-the-husky]

brilliant

thanks both, I will check these solutions out forthwith

----------
tested bytemyzer's code, and its done the trick

I dont think i would have got close to that without a lot of work, so thanks a million,
out of interest, is this all part of the web browser (IE) run time, or is this something else

-----------
out of further interest, i read somewhere, that you couldnt use a UK postcode to retrieve this information, as Google wouldn't return UK postcode data

however, i just tried, and it gave me the same mapping with the address and with just a postcode