Importing multiple uneven sized ranges from .csv file into multiple tables (1 Viewer)

NZArchie

Registered User.
Local time
Tomorrow, 07:44
Joined
May 9, 2011
Messages
84
Hi everyone,
I'm another bloke on a steep learning curve to understand access as a system. I have this as my current problem:

A client is emailed monthly .csv files which contain sales data for each salesman in the company. I want to be able to automate an import this information to a database for later manipulation. The problem is that the information must go across many tables, and that many of the rows/columns in the .csv file are not relevant at all. I don't fully understand the concept of ranges, can I create and import them through vba? even if I can, many columns have to be ignored

I have implemented a file browser in a form, which passes a file name to a macro procedure, now how can I use vba to browse the file, and choose which columns from which rows to import? I gather I can use an ADO connection to create an append query but I have no idea how.

I've searched for what seems like a long time but only came up with an older .csv thread which (can't post link sorry) is for only one table and not for 2007 (different?)

Thanks for any help
 

Guus2005

AWF VIP
Local time
Today, 21:44
Joined
Jun 26, 2007
Messages
2,641
You can link the csv file as a table. Then you can distribute all data over all the tables you want. After that you can disconnect the table.

Only if you want to automate this process you need some VBA code. I am not a fan but i believe you can also do it using Macro's. Don't ask me how.

HTH:D
 

NZArchie

Registered User.
Local time
Tomorrow, 07:44
Joined
May 9, 2011
Messages
84
There are so many options to think about. I think my problem is actually importing many unnamed ranges from an excel file at once.

How do you link to excel in VBA?
 

gemma-the-husky

Super Moderator
Staff member
Local time
Today, 20:44
Joined
Sep 12, 2006
Messages
15,653
you should not do this by inspection.

if the files are standard layout, then you just need to develop a standard routine to handle them.If you are getting csv files, you should definitely not bring them inot excel. At best this will not help, at worst it will change the data.

if each row of data is the same, then import the whole table into access, and concoct a series of queries to handle them. At this point you can be selective about the columns and rows you need to process

if each row is not the same, then there should still be an ordered structure. in which you can achieve a similar thing, by reading in each line sequentially - and processing each row in turn. This way is harder, but still doable.

if the format of the csv is indeterminate - then you need a different csv.
 

NZArchie

Registered User.
Local time
Tomorrow, 07:44
Joined
May 9, 2011
Messages
84
Ok, thanks for the direction. The file is a fairly standard layout so I know i could loop through it sequentially. But how do I get to the point where I can read/sequentially operate on the file.

Do I open it with DAO? Presuming I do find a row I want to import, what is the call to create a new entry in my tables?

Thanks for any help
Tom
 

Rabbie

Super Moderator
Local time
Today, 20:44
Joined
Jul 10, 2007
Messages
5,906
Ok, thanks for the direction. The file is a fairly standard layout so I know i could loop through it sequentially. But how do I get to the point where I can read/sequentially operate on the file.

Do I open it with DAO? Presuming I do find a row I want to import, what is the call to create a new entry in my tables?

Thanks for any help
Tom
The VBA editor Help files are an invaluable source of info for you. Don't be afraid to use it.

Link to the file as a table then open the recordset. You can use the Movenext, movefirst methods to navigate the file. use AddNew to add a new record to the file. remember to use Update to write the changes to the DB.

I would strongly reccommend you to google for a good tutorial on file handling in Access.
 

Guus2005

AWF VIP
Local time
Today, 21:44
Joined
Jun 26, 2007
Messages
2,641
Look at the TransferSpreadsheet command if you want to import multiple rnages into different tables.
Example:
Code:
DoCmd.TransferSpreadsheet acImport, acSpreadsheetTypeExcel12, _
    "Employees","C:\Excel\Newemps.xls", True, "A1:G12"

Share & Enjoy!
 

NZArchie

Registered User.
Local time
Tomorrow, 07:44
Joined
May 9, 2011
Messages
84
Hey again everyone, thanks for the help. I found this template which I am trying to modify for my use on databasejournal.com (can't post link yet sorry) I will post my code here instead
Code:
Public Function importSovereign(sFile As String) As String
   On Error GoTo ProcessFileImport_Error
   
   ' Excel object variables
   Dim appExcel As Excel.Application
   Dim wbk As Excel.Workbook
   Dim wks As Excel.Worksheet
   
   ' Access object variables
   Dim dbs As DAO.Database
   Dim rstRead As DAO.Recordset
   Dim rstWrite As DAO.Recordset
   Dim fld As DAO.Field
   
   ' Declared variables
   Dim bytWks As Byte
   Dim bytMaxPages As Byte
   Dim intStartRow As Integer
   Dim strData As String
   Dim intMaxRow As Integer
   Dim strSql As String
   Dim strMsg As String
   Dim intLastCol As Integer
   Dim intRow As Integer
   Dim intRec As Integer
   Dim strCurrFld As String
   Dim intCol As Integer
   Dim intLen As Integer
   Dim varValue As Variant
   Dim lngErrs As Long
   Const cPassword As String = "xxx999"
   
   DoCmd.Hourglass True
   
   ' Create the Excel Application, Workbook and Worksheet and Database object
   Set appExcel = Excel.Application
   Set wbk = appExcel.Workbooks.Open(sFile)
   Set dbs = CurrentDb
   
   ' Optionally, you can protect / unprotect with a password
   'wbk.Unprotect (cPassword)
   
   ' You could loop through sheets, but for this example, we'll just do one.
   bytMaxPages = 1
   
   ' Sometimes there is header info, so the "Start Row" isn't the first one.
   ' Set this variable to the first row that contains actual data.
   intStartRow = 7
   
   For bytWks = 1 To bytMaxPages
      ' Initialize variables on each pass
      Set wks = Nothing
      Set rstRead = Nothing
      intRow = intStartRow
      
      ' Load current worksheet.  Find used range to determine row count.
      Set wks = appExcel.Worksheets(bytWks)
      
      ' Optionally, you can protect / unprotect with a password
      'wks.Unprotect (cPassword)
      
      ' You need to figure out how many rows this sheet contains, so to know
      ' how far down to read.  That value is saved in intMaxRow
      strData = wks.UsedRange.Address
      intMaxRow = CInt(Mid(strData, InStrRev(strData, "$")))
      'intMaxRow = CInt(Mid(strData, LastInStr(strData, "$")))
      
      strData = ""
      
      ' Go get the list of fields for this worksheet from the Field Map table
      ' In this case I am importing Advisor name fields, Customer Name fields,
      ' all of policy table, Provider company name, and all of payment
      strSql = "SELECT Advisors.FirstName AS Advisors_FirstName, Advisors.LastName " & vbCrLf & _
            " AS Advisors_LastName, Customers.FirstName AS Customers_FirstName,  " & vbCrLf & _
            "Customers.LastName AS Customers_LastName, Payment.PaymentID,  " & vbCrLf & _
            "Payment.DateOfPayment, Payment.PayRunNo, Payment.PolicyNumber  " & vbCrLf & _
            "AS Payment_PolicyNumber, Payment.AmountExGST, Policy.PolicyNumber " & vbCrLf & _
            " AS Policy_PolicyNumber, Policy.CustomerID, Policy.[Start Date],  " & vbCrLf & _
            "Policy.ProviderID AS Policy_ProviderID, Policy.PolicyType, Policy.GSTApplicable,  " & vbCrLf & _
            "Policy.Premium, Policy.InsType, Policy.AdvisorID, Providers.ProviderID AS  " & vbCrLf & _
            "Providers_ProviderID, Providers.[Company Name] " & vbCrLf & _
            "FROM ((Advisors INNER JOIN Customers ON  " & vbCrLf & _
            "Advisors.AdvisorID=Customers.AdvisorID) INNER JOIN  " & vbCrLf & _
            "(Providers INNER JOIN Policy ON  " & vbCrLf & _
            "Providers.ProviderID=Policy.ProviderID) ON  " & vbCrLf & _
            "Advisors.AdvisorID=Policy.AdvisorID) INNER JOIN Payment  " & vbCrLf & _
            "ON Policy.PolicyNumber=Payment.PolicyNumber;"
      
           
         ' The name of the import and destination table should be the same for this
         ' code to function correctly.
         Set rstWrite = dbs.OpenRecordset(strSql, dbOpenDynaset)
         Do Until intRow > intMaxRow
            ' Check first cell to be sure it is not blank.  If so, skip the row
            strData = strData & Trim(Nz(wks.Cells(intRow, 1), ""))
            
            If strData = "" Then
               intRow = intRow + 1
            Else
               intRec = intRec + 1
               rstWrite.AddNew
               Do Until rstRead.EOF
                  ' Loop through the list of fields, processing them one at a time.
                  ' Grab the field name to simplify code and improve performance.
                  strCurrFld = Nz(rstRead!AccessField, "")
                  intCol = rstRead!OrdinalPosition
                  
                  ' Make sure that text fields truncate data at prescribed limits.
                  ' Users may not enter supply more text than the fields can contain.
                  If dbs.TableDefs(sTable).Fields(strCurrFld).Type = dbText Then
                     intLen = dbs.TableDefs(sTable).Fields(strCurrFld).Size
                     varValue = Left(Nz(wks.Cells(intRow, intCol), ""), intLen)
                  Else
                     varValue = wks.Cells(intRow, intCol)
                  End If
                  
                  ' The database schema requires that empty fields contain NULL, not
                  ' the empty string.
                  If varValue = "" Then varValue = Null
                  
                  ' Handle date columns.  Sometimes Excel doesn't format them as dates
                  If InStr(1, strCurrFld, "Date") > 0 Then
                     If Not IsDate(varValue) Then
                        If IsNumeric(varValue) Then
                           On Error Resume Next
                           varValue = CDate(varValue)
                           If Err.Number <> 0 Then
                              ' Can't figure out the date.  Set to null
                              varValue = Null
                              Err.Clear
                           End If
                           On Error GoTo ProcessFileImport_Error
                        Else
                           lngErrs = lngErrs + 1
                           varValue = Null
                        End If
                     End If
                     rstWrite.Fields(strCurrFld) = varValue
                  Else
                     ' If not a date field, then just write the value to the rst
                     ' (you may need to validate numeric values too)
                     rstWrite.Fields(strCurrFld) = varValue
                  End If
                  
                  rstRead.MoveNext
               Loop
               If Not rstRead.BOF Then rstRead.MoveFirst
                              
               rstWrite.Update
               
               ' Reset the variables for processing of the next record.
               strData = ""
               intRow = intRow + 1
            End If
         Loop
         Set wks = Nothing
      
   Next
   
Exit_Here:
   ' Report results
   strMsg = "Total of " & intRow & " records imported."
   ProcessFileImport = strMsg
   
   ' Cleanup all objects  (resume next on errors)
   ' Optionally, you can protect / unprotect with a password
   'wbk.Protect (cPassword)
   'wks.Protect (cPassword)
   On Error Resume Next
   Set wks = Nothing
   wbk.Close True
   Set wbk = Nothing
   appExcel.Quit
   Set appExcel = Nothing
   Set rstRead = Nothing
   Set rstWrite = Nothing
   Set dbs = Nothing
   DoCmd.Hourglass False
   Exit Function
   
ProcessFileImport_Error:
   MsgBox Err.Description, vbExclamation, "Error"
   Resume Exit_Here
End Function
However I don't understand what the rstRead variable is? Is it a link to excel or the database? Also, I have to import to many tables, this is the SQL statement I've made:
Code:
strSql = "SELECT Advisors.FirstName AS Advisors_FirstName, Advisors.LastName " & vbCrLf & _
            " AS Advisors_LastName, Customers.FirstName AS Customers_FirstName,  " & vbCrLf & _
            "Customers.LastName AS Customers_LastName, Payment.PaymentID,  " & vbCrLf & _
            "Payment.DateOfPayment, Payment.PayRunNo, Payment.PolicyNumber  " & vbCrLf & _
            "AS Payment_PolicyNumber, Payment.AmountExGST, Policy.PolicyNumber " & vbCrLf & _
            " AS Policy_PolicyNumber, Policy.CustomerID, Policy.[Start Date],  " & vbCrLf & _
            "Policy.ProviderID AS Policy_ProviderID, Policy.PolicyType, Policy.GSTApplicable,  " & vbCrLf & _
            "Policy.Premium, Policy.InsType, Policy.AdvisorID, Providers.ProviderID AS  " & vbCrLf & _
            "Providers_ProviderID, Providers.[Company Name] " & vbCrLf & _
            "FROM ((Advisors INNER JOIN Customers ON  " & vbCrLf & _
            "Advisors.AdvisorID=Customers.AdvisorID) INNER JOIN  " & vbCrLf & _
            "(Providers INNER JOIN Policy ON  " & vbCrLf & _
            "Providers.ProviderID=Policy.ProviderID) ON  " & vbCrLf & _
            "Advisors.AdvisorID=Policy.AdvisorID) INNER JOIN Payment  " & vbCrLf & _
            "ON Policy.PolicyNumber=Payment.PolicyNumber;"
Assuming I can read from an excel row into some variables, how does the actual addNew vba line work with all those options?

I couldn't see how transferSpreadsheet would work sorry, because I do not want to read all the columns, and I don't know what size the ranges will be.
 

gemma-the-husky

Super Moderator
Staff member
Local time
Today, 20:44
Joined
Sep 12, 2006
Messages
15,653
partial quote

I couldn't see how transferSpreadsheet would work sorry, because I do not want to read all the columns, and I don't know what size the ranges will be.

Archie

transferspreadsheet is not correct. if you have a csv then it's transfertext.

First reading the csv directly is more powerful, as you get control over the data.
Second, you KNOW the data has not been interfered with
Third, just by reading stuff into excel, excel can change things (eg text comes in as numbers) and as a result there is a possibility access will handle some data incorrectly - as it makes assumptions about spreadsheets that you cannot control at all.
Fourth, it saves time anyway. If you have lots of files, you don't wany to spend time having to find them and bring them into excel

Either link to, or import the csv (personally, I import). You THEN concoct a series of queries to discard/ignore the rows/columns you don't want, and process the rows/columns you do want. The transfertext can be set to ignore certain columns anyway.
 

NZArchie

Registered User.
Local time
Tomorrow, 07:44
Joined
May 9, 2011
Messages
84
Ok Dave, that sounds logical in theory then but how does it work? Does transfertext create a temporary table if you put an unkown name into it? As above, I need to somehow put the data across many tables, and the examples I have seen seem to only use one table, and have .csv's full of relevant information, the same as the access table.
 

gemma-the-husky

Super Moderator
Staff member
Local time
Today, 20:44
Joined
Sep 12, 2006
Messages
15,653
general data in a csv will correspond to multiple tables in your database.


try doing it manually. file/get external data/import and see what the table looks like when its imported.

Now you need some code and queries to match it to your existing data, and process it in the way you want. It will take some time to set it up - but once it's done, you (ie - the users) can repeat the process without having to do anything else.
 

NZArchie

Registered User.
Local time
Tomorrow, 07:44
Joined
May 9, 2011
Messages
84
Yep I think that's the ideal way thanks. The leap for me was calling an import specification from the vba, and the checks it needs, like this:

Code:
    ' Check that there is only one period in the filename
    Dim numPeriods As Integer
    numPeriods = Len(Dir(sFile)) - Len(Replace(Dir(sFile), ".", ""))
   
    'if the file path exists and there is one period in the filename
    If Len(sFile) > 0 And (numPeriods = 1) Then
    
        ' If the temp table exists, delete it
        If fExistTable(tempTableName) Then
            DoCmd.DeleteObject acTable, tempTableName
        End If
    
        ' else skip to next step,
        DoCmd.TransferText acImportDelim, "SovereignImportSpecification", tempTableName, sFile, False, ""
   
    Else
        ' There was an extra period in the filename, error
        MsgBox ("Only one period is permitted in the file name, please try again")
   End If

Now I need to work out how to parse my temporary table and append the relevant parts of the records. But that is another thread. Consider yourself helpful :)
 

gemma-the-husky

Super Moderator
Staff member
Local time
Today, 20:44
Joined
Sep 12, 2006
Messages
15,653
well now the new table is in your database, and can be treated like any other table. you can write queries to select certain columns/rows, and add them into the "real" data tables in your database.

obviously it means you can validate the data as well, and pick up duplications, and other errors

good luck
 

Users who are viewing this thread

Top Bottom