VBA Arrays (1 Viewer)

The_Doc_Man

Immoderate Moderator
Staff member
Local time
Today, 08:12
Joined
Feb 28, 2001
Messages
27,169
OK, here is where life can get hinky...

Sometimes you need to denormalize a database slightly to optimize it. If you are looking for speed,you might want to store the information separately for a month-number and year-number. Tack on two integers (INTEGER, not LONG) on your transactions that you are scanning. Update the table to put the year number in one of the items and the month number in the other. Put non-unique keys on those integers. Put a non-unique key on the account number, whatever it is.

Now you should be able to do an aggregate query that gives you your monthly totals for each account with a series of GROUP BY entries, the order of which will depend on you. And this would be a single query, not a loop. How fast will it run? DAMFINO, but it should be faster that your loops.

Your idea about VBA arrays is wrong. VBA isn't compiled. It is interpreted. Running interpretive code will eat your socks in loops like that. You would think that SQL can't be any faster... but it is. When you define the query, Access analyzes it to decide ahead of time the best way to do it, sets up its pointers, and stores all of that somewhere. (No, offhand I don't know where.) When you actually execute the query, Access has already decided how to do it and just starts its own internal loop. Now the SQL processor IS compiled and its criteria are predefined - so THAT loop IS fast. I know this lesson painfully. On a much smaller table, I changed from a VBA loop to pure SQL and saw a tenfold speed increase instantly. I've been working on it ever since, and I can tell you - VBA loops ain't the way to go unless you have to do something inside the loop such as change-logging or something like that.
 

stopher

AWF VIP
Local time
Today, 14:12
Joined
Feb 1, 2006
Messages
2,395
OK, here is where life can get hinky...

Sometimes you need to denormalize a database slightly to optimize it. If you are looking for speed,you might want to store the information separately for a month-number and year-number. Tack on two integers (INTEGER, not LONG) on your transactions that you are scanning. Update the table to put the year number in one of the items and the month number in the other. Put non-unique keys on those integers. Put a non-unique key on the account number, whatever it is..
I agree and generally extract reporting data to separate database, de-normalise/star schema or similar, add extra repoting keys e.g. reporting periods.

Now you should be able to do an aggregate query that gives you your monthly totals for each account with a series of GROUP BY entries, the order of which will depend on you. And this would be a single query, not a loop. How fast will it run? DAMFINO, but it should be faster that your loops.
This is what my first query does. But it's only half the problem. To get a balance for month 3 you need to sum months 1-3. For month 4 it's the sum of months 1-4 etc. Hence the recursive nature. I took the approach of using queries to do all the summing then just used VBA to run the query for each month.

Chris
 

bjsteyn

Registered User.
Local time
Today, 16:12
Joined
May 15, 2008
Messages
113
I notice you have a "Purchase" - that does not count in calculating the account balance?

Purchases are there, you probably just mist it. :eek:
 

bjsteyn

Registered User.
Local time
Today, 16:12
Joined
May 15, 2008
Messages
113
I've attached a example that uses just two queries (although you have to run the queries for each required month). You only need to run the second query (qryCountPosAccounts).

The first query (qryListBalances) creates a balance for a given month for all accounts.

The second query (qryCountPosAccounts) simple counts the number of accounts with a balance >1 that the first query generates and appends the count to the AdminFees table along with the Year/Month reference.

So running the second query is all that is needed to populate the AdminFees table with a record for a given year.

The code (Test) in module 1 is used to loop through the required months and run the update query.

Note that because the first query needs parameters passed to is, I have use query definitions in VBA to pass the parameters.

I stress tested the code and queries with
40,000 accounts with 10 records per account per month over 15 months i.e. 6 million records.

It took about 10 minutes to populate 15 months.

hth
Chris

Thanx Chris, i have tested it and it only takes about a minute to run threw the 270000 transactions. The only problem, although it might be small, but is needed for accuracy is that R1 needs to be deducted from the AccountBalance for a month if it was counted as an Admin Fee Deducted. Any idee how this can be done?

The code i wrote only takes 6 seconds to run and only uses one query and vba, but for some reason is not giving me the right values. I have added debug.print to the code to see what is happening as it is stepping threw the loops, but still cannot find any error, yet the total admin fees are wrong.

Below is my code:
Code:
Public Sub AdminFees4()

Dim db As DAO.Database
Dim rs As DAO.Recordset
Dim x As Integer
Dim strSQL As String
Dim AdminFee("2010" To "2011", 1 To 12) As Currency
Dim AccountBalance As Currency
Dim MonthInLastLoop As Integer
Dim AccountInLastLoop As Double
Dim YearInLastLoop As String
Dim FirstLoop As Boolean
Set db = CurrentDb()
 
strSQL = "SELECT [Acc# No#] , year([Time]) AS TransYear, month([Time]) AS TransMonth, [Tran Type], sum([Tran Amount]) AS TranTotal FROM Transactions GROUP BY [Acc# No#] ,year([Time]),month([Time]),[Tran Type]"
Set rs = db.OpenRecordset(strSQL)


rs.MoveLast
If rs.RecordCount > 0 Then

    rs.MoveFirst
    FirstLoop = True
    Do While Not rs.EOF
        
        If FirstLoop Then
        AccountBalance = 0
        AccountInLastLoop = rs![Acc# No#]
        MonthInLastLoop = rs![TransMonth]
        YearInLastLoop = rs![TransYear]
        FirstLoop = False
        Debug.Print "First Account in Loop: " & AccountInLastLoop & "," & MonthInLastLoop & "," & YearInLastLoop
        Else
        Debug.Print "Account In Previous Loop: " & AccountInLastLoop & "," & MonthInLastLoop & "," & YearInLastLoop
        End If
        
        
        
        If Not (rs![Acc# No#] = AccountInLastLoop) Then
        Debug.Print "Finalize Account and Month in Previous Loop"
            If AccountBalance >= 1 Then
                If YearInLastLoop = "2010" Then
                    AdminFee("2010", MonthInLastLoop) = AdminFee("2010", MonthInLastLoop) + 1
                    Debug.Print "Admin fees:2010," & MonthInLastLoop & ": +1"
                ElseIf YearInLastLoop = "2011" Then
                    AdminFee("2011", MonthInLastLoop) = AdminFee("2011", MonthInLastLoop) + 1
                    Debug.Print "Admin fees:2011," & MonthInLastLoop & ": +1"
                Else
                MsgBox "What the f!"
                End If
               
            End If
            
            AccountBalance = 0
            AccountInLastLoop = rs![Acc# No#]
            MonthInLastLoop = rs![TransMonth]
            YearInLastLoop = rs![TransYear]
        Debug.Print "New Account: " & AccountInLastLoop
        Else

            If Not (rs![TransMonth] = MonthInLastLoop) Then
            Debug.Print "Finalize Month in Previous Loop"
                If AccountBalance >= 1 Then
                    If YearInLastLoop = "2010" Then
                        AdminFee("2010", MonthInLastLoop) = AdminFee("2010", MonthInLastLoop) + 1
                        Debug.Print "Admin fees:2010," & MonthInLastLoop & ": +1"
                    ElseIf YearInLastLoop = "2011" Then
                        AdminFee("2011", MonthInLastLoop) = AdminFee("2011", MonthInLastLoop) + 1
                        Debug.Print "Admin fees:2011," & MonthInLastLoop & ": +1"
                    Else
                    MsgBox "What the f!"
                    End If
                    AccountBalance = AccountBalance - 1
                    Debug.Print "Acc Balance: -1"
                End If
                AccountInLastLoop = rs![Acc# No#]
                MonthInLastLoop = rs![TransMonth]
                YearInLastLoop = rs![TransYear]
            End If

        End If
        Debug.Print "Current Transaction: " & rs![Acc# No#] & "," & rs![TransMonth] & "," & rs![TransYear] & "," & rs![Tran Type] & "," & rs![TranTotal]
        If rs![Tran Type] = "Deposit" Then

            AccountBalance = AccountBalance + rs![TranTotal]

        ElseIf rs![Tran Type] = "Payment" Or rs![Tran Type] = "Withdrawal" Then

            AccountBalance = AccountBalance - rs![TranTotal]
        End If

        Debug.Print "Acc: " & rs![Acc# No#] & " Balance: " & AccountBalance
        Debug.Print "----------------------------------------------------------"
        If rs.AbsolutePosition = rs.RecordCount - 1 Then
        
            If AccountBalance > 1 Then
        
                    If rs!TransYear = "2010" Then
                        AdminFee("2010", rs![TransMonth]) = AdminFee("2010", rs![TransMonth]) + 1
                    ElseIf rs!TransYear = "2011" Then
                        AdminFee("2011", rs![TransMonth]) = AdminFee("2011", rs![TransMonth]) + 1
                    End If
        
           End If
        End If
        
        rs.MoveNext

    Loop

End If
rs.Close
Set rs = Nothing

strSQL = "SELECT * FROM AdminFees"
Set rs = db.OpenRecordset(strSQL)

For x = 3 To 12
rs.AddNew
rs![MonthF] = x
rs![YearF] = 2010
rs![AdminFees] = AdminFee("2010", x)
rs.Update
Next x

For x = 1 To 5
rs.AddNew
rs![MonthF] = x
rs![YearF] = 2011
rs![AdminFees] = AdminFee("2011", x)
rs.Update
Next x

rs.Close
Set rs = Nothing
Set db = Nothing

MsgBox "Complete"
End Sub
Here is a sample of the debug output (A line shows a new loop ):
Code:
Current Transaction: 3000098619948,2,2011,Payment,69.47
Acc: 3000098619948 Balance: 51.47
----------------------------------------------------------
Account In Previous Loop: 3000098619948,2,2011
Finalize Month in Previous Loop
Admin fees:2011,2: +1
Acc Balance: -1
Current Transaction: 3000098619948,3,2011,Payment,39.4
Acc: 3000098619948 Balance: 11.07
----------------------------------------------------------
Account In Previous Loop: 3000098619948,3,2011
Finalize Account and Month in Previous Loop
Admin fees:2011,3: +1
New Account: 3000098684421
Current Transaction: 3000098684421,4,2011,Deposit,30
Acc: 3000098684421 Balance: 30
----------------------------------------------------------
Account In Previous Loop: 3000098684421,4,2011
Current Transaction: 3000098684421,4,2011,Payment,30
Acc: 3000098684421 Balance: 0
----------------------------------------------------------
Account In Previous Loop: 3000098684421,4,2011
Finalize Account and Month in Previous Loop
New Account: 3000098936640
Current Transaction: 3000098936640,4,2011,Deposit,50
Acc: 3000098936640 Balance: 50
----------------------------------------------------------
Account In Previous Loop: 3000098936640,4,2011
Finalize Account and Month in Previous Loop
Admin fees:2011,4: +1
New Account: 3000098984534
Current Transaction: 3000098984534,4,2011,Deposit,20
Acc: 3000098984534 Balance: 20
----------------------------------------------------------
Account In Previous Loop: 3000098984534,4,2011
Current Transaction: 3000098984534,4,2011,Payment,15.98
Acc: 3000098984534 Balance: 4.02
----------------------------------------------------------
Account In Previous Loop: 3000098984534,4,2011
Finalize Account and Month in Previous Loop
Admin fees:2011,4: +1
New Account: 3000099087967
Current Transaction: 3000099087967,2,2011,Deposit,20
Acc: 3000099087967 Balance: 20
----------------------------------------------------------
Account In Previous Loop: 3000099087967,2,2011
Finalize Account and Month in Previous Loop
Admin fees:2011,2: +1
New Account: 3000099302966
Current Transaction: 3000099302966,12,2010,Deposit,600
Acc: 3000099302966 Balance: 600
----------------------------------------------------------
Account In Previous Loop: 3000099302966,12,2010
Current Transaction: 3000099302966,12,2010,Payment,136.9
Acc: 3000099302966 Balance: 463.1
----------------------------------------------------------
Account In Previous Loop: 3000099302966,12,2010
Current Transaction: 3000099302966,12,2010,Withdrawal,400
Acc: 3000099302966 Balance: 63.1
----------------------------------------------------------
Account In Previous Loop: 3000099302966,12,2010
Finalize Account and Month in Previous Loop
Admin fees:2010,12: +1
New Account: 3000099511035
Current Transaction: 3000099511035,12,2010,Deposit,500
Acc: 3000099511035 Balance: 500
----------------------------------------------------------
Account In Previous Loop: 3000099511035,12,2010
Current Transaction: 3000099511035,12,2010,Payment,430.58
Acc: 3000099511035 Balance: 69.42
----------------------------------------------------------
Account In Previous Loop: 3000099511035,12,2010
Finalize Account and Month in Previous Loop
Admin fees:2010,12: +1
New Account: 3000099545730
Current Transaction: 3000099545730,10,2010,Deposit,50
Acc: 3000099545730 Balance: 50
----------------------------------------------------------
Account In Previous Loop: 3000099545730,10,2010
Finalize Account and Month in Previous Loop
Admin fees:2010,10: +1
New Account: 3000099588676
Current Transaction: 3000099588676,9,2010,Deposit,100
Acc: 3000099588676 Balance: 100
----------------------------------------------------------
Account In Previous Loop: 3000099588676,9,2010
Current Transaction: 3000099588676,9,2010,Payment,42.96
Acc: 3000099588676 Balance: 57.04
----------------------------------------------------------
Account In Previous Loop: 3000099588676,9,2010
Finalize Month in Previous Loop
Admin fees:2010,9: +1
Acc Balance: -1
Current Transaction: 3000099588676,1,2011,Payment,28.22
Acc: 3000099588676 Balance: 27.82
----------------------------------------------------------
Account In Previous Loop: 3000099588676,1,2011
Finalize Month in Previous Loop
Admin fees:2011,1: +1
Acc Balance: -1
Current Transaction: 3000099588676,5,2011,Deposit,100
Acc: 3000099588676 Balance: 126.82
----------------------------------------------------------
Account In Previous Loop: 3000099588676,5,2011
Finalize Account and Month in Previous Loop
Admin fees:2011,5: +1
New Account: 3000099647158
Current Transaction: 3000099647158,4,2011,Deposit,2
Acc: 3000099647158 Balance: 2
----------------------------------------------------------
Account In Previous Loop: 3000099647158,4,2011
Finalize Month in Previous Loop
Admin fees:2011,4: +1
Acc Balance: -1
Current Transaction: 3000099647158,5,2011,Deposit,50
Acc: 3000099647158 Balance: 51
----------------------------------------------------------
Account In Previous Loop: 3000099647158,5,2011
Finalize Account and Month in Previous Loop
Admin fees:2011,5: +1
New Account: 3000099838102
Current Transaction: 3000099838102,5,2011,Deposit,30
Acc: 3000099838102 Balance: 30
----------------------------------------------------------
Account In Previous Loop: 3000099838102,5,2011
Current Transaction: 3000099838102,5,2011,Payment,23.57
Acc: 3000099838102 Balance: 6.43
----------------------------------------------------------
Account In Previous Loop: 3000099838102,5,2011
Finalize Account and Month in Previous Loop
Admin fees:2011,5: +1
New Account: 3000099931983
Current Transaction: 3000099931983,2,2011,Deposit,1
Acc: 3000099931983 Balance: 1
----------------------------------------------------------
Account In Previous Loop: 3000099931983,2,2011
Finalize Account and Month in Previous Loop
Admin fees:2011,2: +1
New Account: 3000100056540
Current Transaction: 3000100056540,4,2011,Deposit,530
Acc: 3000100056540 Balance: 530
----------------------------------------------------------
Account In Previous Loop: 3000100056540,4,2011
Current Transaction: 3000100056540,4,2011,Payment,527.21
Acc: 3000100056540 Balance: 2.79
----------------------------------------------------------
Account In Previous Loop: 3000100056540,4,2011
Finalize Month in Previous Loop
Admin fees:2011,4: +1
Acc Balance: -1
Current Transaction: 3000100056540,5,2011,Deposit,500
Acc: 3000100056540 Balance: 501.79
----------------------------------------------------------
Account In Previous Loop: 3000100056540,5,2011
Current Transaction: 3000100056540,5,2011,Payment,499.92
Acc: 3000100056540 Balance: 1.87
----------------------------------------------------------
Account In Previous Loop: 3000100056540,5,2011
Finalize Account and Month in Previous Loop
Admin fees:2011,5: +1
New Account: 3000100520986
Current Transaction: 3000100520986,5,2011,Deposit,100
Acc: 3000100520986 Balance: 100
----------------------------------------------------------
Account In Previous Loop: 3000100520986,5,2011
Finalize Account and Month in Previous Loop
Admin fees:2011,5: +1
New Account: 3000100552996
Current Transaction: 3000100552996,1,2011,Deposit,550
Acc: 3000100552996 Balance: 550
----------------------------------------------------------
Account In Previous Loop: 3000100552996,1,2011
Finalize Month in Previous Loop
Admin fees:2011,1: +1
Acc Balance: -1
Current Transaction: 3000100552996,2,2011,Deposit,400
Acc: 3000100552996 Balance: 949
----------------------------------------------------------
Account In Previous Loop: 3000100552996,2,2011
Finalize Month in Previous Loop
Admin fees:2011,2: +1
Acc Balance: -1
Current Transaction: 3000100552996,3,2011,Payment,921.16
Acc: 3000100552996 Balance: 26.84
----------------------------------------------------------
Account In Previous Loop: 3000100552996,3,2011
Finalize Month in Previous Loop
Admin fees:2011,3: +1
Acc Balance: -1
Current Transaction: 3000100552996,5,2011,Deposit,600
Acc: 3000100552996 Balance: 625.84
----------------------------------------------------------
Account In Previous Loop: 3000100552996,5,2011
Current Transaction: 3000100552996,5,2011,Payment,107.02
Acc: 3000100552996 Balance: 518.82
----------------------------------------------------------
Account In Previous Loop: 3000100552996,5,2011
Finalize Account and Month in Previous Loop
Admin fees:2011,5: +1
Here is the result i get from running my code:
AdminFees
MonthF YearF AdminFees
3
2010 53
4
2010 564
5
2010 826
6 2010 1015
7
2010 961
8 2010 1719
9
2010 2187
10
2010 2395
11
2010 3482
12 2010 4645
1
2011 5802
2
2011 8365
3
2011 9333
4
2011 11432
5
2011 9596

Here is the result i get from running your code:

AdminFees
MonthF
YearF AdminFees
3
2010 53
4
2010 584
5
2010 1073
6
2010 1630
7
2010 1924
8
2010 2853
9 2010 3936
10
2010 5055
11
2010 7116
12 2010 9568
1
2011 12532
2
2011 16914
3 2011 21833
4
2011 28124
5
2011 31190

The admin fees for May 2011 was around 29700. There are a few accounts that i might need to delete starting with "2000" and not "3000" like normal accounts which whould bring your value much closer to the actual. Just the above mentioned problem that R1 needs to be deducted from account balance when a admin fee has been deducted.
 

bjsteyn

Registered User.
Local time
Today, 16:12
Joined
May 15, 2008
Messages
113
I can't see any mention of Purchases in your code either. Does that mean you want to ignore them?

Purchases are "Payments". It is in my code: :) :) :)

Code:
 If rs![Tran Type] = "Deposit" Then              
AccountBalance = AccountBalance + rs![TranTotal]          
ElseIf rs![Tran Type] = "Payment" Or rs![Tran Type] = "Withdrawal" Then       AccountBalance = AccountBalance - rs![TranTotal]         
End If
 
Last edited:

bjsteyn

Registered User.
Local time
Today, 16:12
Joined
May 15, 2008
Messages
113
OK, here is where life can get hinky...

Sometimes you need to denormalize a database slightly to optimize it. If you are looking for speed,you might want to store the information separately for a month-number and year-number. Tack on two integers (INTEGER, not LONG) on your transactions that you are scanning. Update the table to put the year number in one of the items and the month number in the other. Put non-unique keys on those integers. Put a non-unique key on the account number, whatever it is.

Now you should be able to do an aggregate query that gives you your monthly totals for each account with a series of GROUP BY entries, the order of which will depend on you. And this would be a single query, not a loop. How fast will it run? DAMFINO, but it should be faster that your loops.

Your idea about VBA arrays is wrong. VBA isn't compiled. It is interpreted. Running interpretive code will eat your socks in loops like that. You would think that SQL can't be any faster... but it is. When you define the query, Access analyzes it to decide ahead of time the best way to do it, sets up its pointers, and stores all of that somewhere. (No, offhand I don't know where.) When you actually execute the query, Access has already decided how to do it and just starts its own internal loop. Now the SQL processor IS compiled and its criteria are predefined - so THAT loop IS fast. I know this lesson painfully. On a much smaller table, I changed from a VBA loop to pure SQL and saw a tenfold speed increase instantly. I've been working on it ever since, and I can tell you - VBA loops ain't the way to go unless you have to do something inside the loop such as change-logging or something like that.

What i meant with the array is that I believed if I could load/stucture the data into an array and then loop threw it whould run faster than 400000 queries, as i could not get the data structured using just 1 query ( i thought). I have 1 query know and am looping threw it but my values are wrong. Don't now why. :eek: .

I will take the advise to go rather with SQL than looping, but how can i solve my problem with the R1 needing to be deducted from the balance if an admin fee was counted.
 
Last edited:

bjsteyn

Registered User.
Local time
Today, 16:12
Joined
May 15, 2008
Messages
113
I have attached a db with my code module. Thanx BJ
 

Attachments

  • AdminFeesByMonth (2).accdb
    368 KB · Views: 66

bjsteyn

Registered User.
Local time
Today, 16:12
Joined
May 15, 2008
Messages
113
I used your account balance query, Chris, in my final code. Thanx Stopher, Spikepl and The Doc Man for your help.

Code:
Public Sub calcAFees()
Dim db As DAO.Database
Dim rs As DAO.Recordset
Dim rs2 As DAO.Recordset
Dim strSQL As String
Dim iMonth As Integer
Dim iYear As Integer
    
Dim AdminFee("2010" To "2011", 1 To 12) As Currency
Dim AdFee As Integer
Set db = CurrentDb
    
iMonth = 2
iYear = 2010

Do

    
    iMonth = iMonth + 1
    If iMonth > 12 Then
        iMonth = 1
        iYear = iYear + 1
    End If
    
    strSQL = "SELECT [Acc# No#]," _
             & " Sum(IIf([Tran Type]=""Deposit"",[Tran Amount],IIf([Tran Type] In (""Withdrawal"",""Payment""),-[Tran Amount],0))) AS Total FROM Transactions" _
             & " WHERE (((DateValue([Time])) Between DateSerial(2010,3,1) And DateSerial(" & iYear & "," & iMonth & "+1,0)))" _
             & " GROUP BY [Acc# No#];"

    
    Set rs = db.OpenRecordset(strSQL)
    rs.MoveLast
    'MsgBox rs.RecordCount
    If rs.RecordCount > 0 Then
        rs.MoveFirst
        Do While Not rs.EOF
            strSQL = "SELECT count(tAccount) AS AdminFees FROM temp WHERE tAccount = " & rs![Acc# No#]
            Set rs2 = db.OpenRecordset(strSQL)
            rs2.MoveLast
            If rs2.RecordCount > 0 Then
                rs2.MoveFirst
                AdFee = rs2!AdminFees
            Else
                AdFee = 0
            End If
            
            If (rs!Total - AdFee) >= 1 Then
                If iYear = "2010" Then
                    AdminFee("2010", iMonth) = AdminFee("2010", iMonth) + 1
                ElseIf iYear = "2011" Then
                    AdminFee("2011", iMonth) = AdminFee("2011", iMonth) + 1
                End If
                strSQL = "INSERT INTO temp (tMonth,tYear,tAccount) VALUES (" & iMonth & ",'" & iYear & "'," & rs![Acc# No#] & ")"
                DoCmd.SetWarnings False
                DoCmd.RunSQL strSQL
                DoCmd.SetWarnings True
            End If
            rs.MoveNext
        Loop
    End If
               
               
               
    Debug.Print iYear & " " & iMonth
Loop Until iYear = 2011 And iMonth = 5   '(stop processing at this year/month)

rs.Close
Set rs = Nothing

Dim x As Integer
strSQL = "SELECT * FROM AdminFees"
Set rs = db.OpenRecordset(strSQL)
    
For x = 3 To 12
    rs.AddNew
    rs![MonthF] = x
    rs![YearF] = 2010
    rs![AdminFees] = AdminFee("2010", x)
    rs.Update
Next x
    
For x = 1 To 5
    rs.AddNew
    rs![MonthF] = x
    rs![YearF] = 2011
    rs![AdminFees] = AdminFee("2011", x)
    rs.Update
Next x
    
rs.Close
Set rs = Nothing

Set db = Nothing

MsgBox "Complete"
End Sub
 

Users who are viewing this thread

Top Bottom