Screen Scraper - help finding example or tutorial please.

[DiverGuy]

New member here. I've searched this forum and the internet but came up dry and sure could use some guidance.

There are plenty of references to being able to use Access 2003 VBA to open a separate program (in this case it will be Internet Explorer 6.0) and scrape data from the window, importing it into a table. But all of the references only mention that it can be done, not how to do it. Apparently, it is such a common thing that pretty much everyone except me already knows how to do it.

Can anyone please point me toward a tutorial that walks step-by-step through the process of using VBA to open IE and extract data? If there isn't an online tutorial, maybe someone has an example I could use to figure it out.

Any and all help is appareciated.

Thanks,

DiverGuy

 

[boblarson]

New member here. I've searched this forum and the internet but came up dry and sure could use some guidance.

There are plenty of references to being able to use Access 2003 VBA to open a separate program (in this case it will be Internet Explorer 6.0) and scrape data from the window, importing it into a table. But all of the references only mention that it can be done, not how to do it.

I don't remember seeing anything like that here. I do know you can import from Excel, Word, etc. but it's not doing screen scraping. Maybe you could supply a couple of links to some posts that you found so we can see what they're talking about.

 

[DiverGuy]

Hi Bob,

erindyl posted a question on this site about screen scraping from a Rumba session but that was six years ago and no one responded. The link is http://www.access-programmers.co.uk/forums/showthread.php?t=27246&highlight=screen+scrape

Here are several links that mention it:
http://www.experts-exchange.com/Programming/Languages/Visual_Basic/Q_21239696.html

http://www.hummingbird.com/hostexplorer-users/feb2005/0007.html

http://p2p.wrox.com/TopicIndex/51725.htm

http://www.parallax.com/dl/docs/article/FlashingStockAlert.pdf

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1481928&SiteID=1

 

[boblarson]

I thought you were talking about that there were solutions posted here.

The one from Experts Exchange has listed a solution. Click on the See Solution button to see the solution (register if you haven't so you can see it).

 

[DiverGuy]

I'm halfway there. The problem I ran into was an unanticipated one and I would appreciate your thoughts.

The report the customer wants summarized is accessed through an intranet website. Unfortunately, each time you log onto the website, it generates a randomly numbered session code, which becomes part of the URL, and nothing in the URL contains the report number requested. Bottom line, the target report can’t be reached with a preconfigured URL.

The best I can do so far is to programmatically open the web browser to the reports login page. Once the user is logged in and the report is displayed, I thought about using VBA to find the IE window by name and grab the innerText that way. Unfortunately, finding a window by name has to be the exact and entire name in the title bar. Our IE title bars include the service pack and patch info, which varies from workstation to workstation and changes whenever Corporate pushes out a new patch. Coding to find the window by name won’t work for long or for everyone. These are some challenging barriers and, for the moment at least, I’m at a loss on how to get around them. And, of course, the boss is pushing to wrap this one up quickly.

Any advice would be appreciated.

Thanks.

DiverGuy

 

[Moniker]

I write screen scrapers in Rumba all the time. I haven't tried it in IE, but it can't be that difficult. To get to the window you need, you don't need the full name, just enough to make it unique. Take a peek at my post here, starting with response #5.

http://www.access-programmers.co.uk/forums/showthread.php?t=87299

 

[KeithG]

I write screen scrapers in Rumba all the time

Is Rumba terimal elumanation software similar to Attachmate? I write screen scraping macros all the time with EB or VBA for Attachmate. I know you can initaite a session of IE explorer with VBA and send data to the screen and click the buttons on a web page. Search the forum for Web browser control for some examples.

 

[Moniker]

Not used Attachmate, but assuming you're connecting to a mid to mainframe computer of some sort, then they're probably very similar. We emulate a 3270 connection through our Rumba, and use a custom API to navigate around. It's easier to scrape on those old apps because of the 25x80 screen. "Go to row 4, column 20, and grab the next six characters" is a lot easier than parsing through variable width pages, at least for screen scraping.

 

[KeithG]

Yeah they sound similar. Although its sounds like you know alot more about Rumba than I do about Attchmate. So I assume you can write your Rumba macros in VBA? Is it the good old GetString and PutString methods of the screen object you use to screen scrap? Thats what we use in Attchamate anways.

 

[Moniker]

No screen objects -- we directly connect to the Rumba Session and the use custom functions (CopyPSToString and CopyStringToPS, where "PS" is short for presentation space). It's essentially the same thing, but perhaps a little more flexible.

 

[DiverGuy]

Our company has Rumba, Attachmate, and Reflections and they're all pretty easy to write a VBA screen scrape. IE is proving somewhat more difficult.

Just a few minutes ago I found out how to put an ActiveX web browser on the form and will navigate to the correct intranet site. That part was easy enough... and interesting. With a separate window, I could capture the text from the page (using objIE.document.body.innertext) into a variable and then dump the variable into a textbox so the text could be parsed. The ActiveX component is a different animal, though. Anyone know how to reference the innertext in it? Me!Webbrowser.document or .document.body .innertext doesn't work and Intellisense doesn't indicate a method or property that would help. If I can grab the innertext from the page, parsing it shouldn't be too bad.

You guys have been great. Appreciate you not laughing too hard at the simple questions. We're inching toward a solution.

 

[masif]

No screen objects -- we directly connect to the Rumba Session and the use custom functions (CopyPSToString and CopyStringToPS, where "PS" is short for presentation space). It's essentially the same thing, but perhaps a little more flexible.

Hi Moniker.

I have done lots of screen scrapping work with Attachmate. But new for Rumba. I have spend lots of time on internet to find out some help that how to do screen scrapping with VBA (MS ACCESS) and Rumba. Can you please help me that how to connect with session and how to pass sendkey to that connected session. I am badly stuck in it. Please help.. many thanks..

 

[ganeshkumars82]

Hi Moniker,

I am also struggleing to scrap the data from TN3270 Plus, Rumba and Eagle simulators.

Plesse help me on the same

 

[ganeshkumars82]

Hi Moniker.

I have done lots of screen scrapping work with Attachmate. But new for Rumba. I have spend lots of time on internet to find out some help that how to do screen scrapping with VBA (MS ACCESS) and Rumba. Can you please help me that how to connect with session and how to pass sendkey to that connected session. I am badly stuck in it. Please help.. many thanks..

Hi Masif,

Did you get any resolution for this task. If yes please share the same. I am also stuck in this point.

Regards,
Ganesh

 

[darbid]

The ActiveX component is a different animal, though. Anyone know how to reference the innertext in it?

In my opinion there is little difference. The webbrowser control is just like a translator in languages. It simply allows your VBA to talk to IE. It is however old which means that IE might have a function which you cannot call from the control.

Other than that it works identically in that it hosts html documents, and you can get a copy of them and then go through them.