Import dynamic internet data

[JGT]

Hi all,
I have been looking for a solution, still without success, for something very useful, but also quite a big challenge.
On the internet there are a lot of sites which display a follow up for processes, products bought etc. like e.g. the post office.
I want to supply my processes, running under Access, with this data automatically. The idea is scanning regularly the internet for the status of the processes to be supervised.
I got no further than to the point where I can read text on whatever static internet page, however, I need to read dynamic data. The site uses AJAX, asking for a process number and then supplies a table snapshot of its db. The table has not an own www. address. How to read and import this data?

Thank you for any help
Jean

 

[spikepl]

It is not clear to me what "processes" are. I am guessing that you want to do a screen scraper, not only of the first html-page you get into but also after inputting some data. So you need to input the data in the appropriate tag somehting.value=MyValue and then submit (presumably by clicking on a submit button, or perhaps Ajax does it, when you assign a value to a tag). Google screen scraping VBA.

 

[JGT]

Thank you Spikepl for answering.
In the mean time I got somewhat more insight in how to get/put data from/to an internet page. This is fine for a static page; my problem is that the site I am interested in uses AJAX, i.e. the source code changes according to answers given.
To be a little bit more specific the site I want to scan opens with a radio button option. After you click an option, it releases a text box, where you have to put a reference number. After this, you press a button and receive the history of your request. I manage to get into the site, which shows its first form, I click the radio button and start the routine that shows the textbox for the reference number. Now I am in the dynamic session, and do not find the address where to put the reference number. I finally tried all “name=” and “id=” that are on the source file, but no way to get the number into the right place. My conclusion is that being dynamic I just see the first page, there must be a trick to get to the dynamic part. And its very hard to find info about AJAX for beginners.
If you know the trick, please tell me!
Thanks for any help.
Below you can see how far I got, its also shows the site address where the source code can be found.
Sub GetVISAinfo()
On Error GoTo Error_GetVISAinfo
Dim v_check$, v_id1$, v_id2$
Dim ie As InternetExplorer
Set ie = New InternetExplorer
'step 0
ie.Navigate "http://migranteweb.mte.gov.br/migranteweb/publico/consultarProcessoInternet/consultarProcesso.seam"
ie.Visible = True
While ie.Busy
Wend
Do Until ie.Document.ReadyState = "complete"
Loop
'step 1
'click the botton
ie.Document.all.Item("subviewCorpoPaginaesquisarProcessoForm:j_id49").Item(0).Checked = True
'now start the rotine
v_id1 = "j_id51": v_id2 = "j_id49"
v_check = "A4J.AJAX.Submit('subviewCorpoPagina:regionTemplate','subviewCorpoPaginaesquisarProcessoForm',event,"
v_check = v_check & "{'parameters':{'subviewCorpoPaginaesquisarProcessoForm:" & v_id1 & "':"
v_check = v_check & "'subviewCorpoPaginaesquisarProcessoForm:" & v_id1 & "',"
v_check = v_check & "'ajaxSingle':'subviewCorpoPaginaesquisarProcessoForm:" & v_id2 & "'} ,"
v_check = v_check & "'actionUrl':'/migranteweb/publico/consultarProcessoInternet/consultarProcesso.seam',"
v_check = v_check & "'similarityGroupingId':'subviewCorpoPaginaesquisarProcessoForm:" & v_id1 & "',"
v_check = v_check & "'control':this} )"
ie.Document.parentwindow.execscript (v_check)
While ie.Busy
Wend
Stop
'step 2
'proc.nbr = 46094000078201275, then push [Pesquisar]
'until here it works fine
'step 3
'get the tracking info
ie.Quit
Set ie = Nothing
Exit_GetVISAinfo:
Exit Sub
Error_GetVISAinfo:
If Err = 462 Then Resume Exit_GetVISAinfo 'Server not available
Debug.Print Err, Error(Err)
Resume Next
End Sub

 

[spikepl]

At the end of the day Ajax just presents some new HTML. Sometimes Browser.Busy is not enough to determine when it is done. You can put a small loop that loops for say 10 seconds, just as test, and see if you get your code. Alternatively, you can loop looking for en element that you know appears with the Ajax bit - when that element is present then you can proceed. WHich element? Look at the souce code of the page at that specific point in the process.

 

[JGT]

Hi Spikepl, the problem right now is not a delay, but to find the address where to put the register number. I am not a AJAX specialist, last time I used HTML was years ago, sinds then things changed a lot. But it looks to me that the place I need to put the nbr is not in the source code. I always see the same code and I understood AJAX dynamically adapts the code. How to get to this changed code?

 

[spikepl]

Look at the souce code of the page at that specific point in the process.

At the end of the day what you see on the screen at each point in time is what you have at that point in time, no matter whether the entire page was loaded from the server or whether some AJAX-magic replaced parts of it. But the lack of delay can be the problem - if your code tries to locate the spot where to input the register number, but the spot simply is not there yet, because the thing has not finished loading the relevant bits (Ajax or no ajax). I have an app (with some Ajax-updated html) where an ie.busy-loop did not delay the next line of code enough for the new page-bits to be available.

 

[JGT]

Hi Spikepl, got one step ahead, only one, still a long way to go - its terrible. I got to the address thanks o the IE9 F12 developper tool, which shows the hidden code. Now I can put the number at the right place.
MHTL is
'<input name="subviewCorpoPaginaesquisarProcessoForm:j_id57" tabIndex="0" class="form-text"
'onkeyup="mascara(this,processo);" onkeypress="mascara(this,processo);" onblur="mascara(this,processo);"
'type="text" size="25" maxLength="20" accesskey="undefined" value="46094.000078/2012-75"/>

ACCESS:
ie.Document.all.Item("subviewCorpoPaginaesquisarProcessoForm:j_id57").Value = "46094000078201275"
ie.Document.parentWindow.execScript ("mascara(this,processo)")

Now I am struggling to find the next step, the last above command line is not working.
When I put the number manually it changes format automatically. With code it doesn´t, so no script is initiated.

 

[FernandoFonseca]

Olá Jean,

Estou com o mesmo problema, você conseguiu a resolução? Poderia disponibilizar?