Does a filtered mdb recordset still bring down the whole recordset?

Premy

Registered User.
Local time
Yesterday, 23:59
Joined
Apr 10, 2007
Messages
196
Hi guys, I'm a bit confused: I know that opening a bound form will bring down the whole recordset. But if I create a parametered query as the recordsource, will it still bring down the whole recordset first and only afterwards cut down to the records matching the parameters?

Example: I have a clients table with 5000 clients on a backend mdb, and my client lookup form in my frontend mdb. If I open the form looking for just client "Jim Jones" (via the parameter query), will Access bring down all 5000 clients from the backend to my frontend first, and only then seek out "Jim Jones" before discarding the rest of the recordset?

For if this is true then this whole Access business is rather unsuited for networks, even a small one. I hope you guys prove me wrong.

Premy
 
Yes I believe you are correct. Access is a file Sever RDBMS you will need to migrate to something like SQL Server for better perfomance.
 
Premy,

To the best of my knowledge, Access will only "bring down" the whole
recordset if you base your form on a table or query that DOESN'T restrict
the data.

If your form's query has parameters, that should greatly cut down on
the amount of network traffic.

Wayne
 
Premy,

To the best of my knowledge, Access will only "bring down" the whole
recordset if you base your form on a table or query that DOESN'T restrict
the data.

If your form's query has parameters, that should greatly cut down on
the amount of network traffic.

Wayne

I don't think that is correct.

Think about it this way... if the backend MDB file is sitting on a file server, what process on that server would be able to filter out the unneeded records, and then only return the requested records? The answer is that there is nothing - no "Microsoft Access Jet Database Engine" service, or anything like that, which could handle the query.

Instead, the computer that has the Access installed (and thus the Jet database engine) links into the backend database, and it has to "see" each record in order to know which ones match the criteria of the query.

With SQL Server, there IS a service on the server where the actual data is stored - and any queries that are sent to that service are then processed by that service, and then the results only are sent back to the client application.
 
Thanks for the reply Keith/Wayne.

As I suspected, based on other readings, there does not seem to be a real consensus on the subject. Some say it doesn't matter how much you parametrize, Access will still bring down the whole rs, and some say proper parametrizing/filtering will bring down only the records wanted...
Still leaves me with questions, for I have an app which works great on a single machine, but since there's a need now for it to allow multiple users I need to know if it's really necessary to make the sql leap. If it were a small DB I would readily do it, but it's not that small (at least I don't consider it small: over 40 tables, over a 100 forms, over a 100 queries, and almost a 100 reports and also a few dozens of modules and dll's) this migration would take me months. I would still want to move it to sql eventually, but on a more relaxed pace. But right now it has to be up and running in a multi-user environment (5-7 users) in a few weeks at the most, and the easiest for me would be really a backend on the server and a front on each workstation, all mdb of course.

Let's see if some other people with experience on the subject shed some more light.

regards,
Premy
 
Thanks BrettStah, just seen your reply

Well it seems it'll have to be the hard way for me hey!
 
Thanks BrettStah, just seen your reply

Well it seems it'll have to be the hard way for me hey!

You can verify it for yourself by using a network monitoring tool to "sniff" what's going to/from your computer's network card, by the way (or ask one of your network gurus if you have one).
 
If you search here on network+backend+performance, you will pick up some threads like
this one.
 
Thanks guys, for your replies.

I think I'd grab this opportunity to warn all beginning Accessers: If you know beforehand that your database will have to be "multiuser on network" at some point, start looking into Access Projects/SQL Server as soon as possible, and don't waste too much time on MDB/JET, for the latter is clearly intended for a 1 database on 1 machine scenario (which I really don't see as sustainable anymore nowadays, to be honest).

I wish someone had given me such an unambiguous warning when I started out with Access.


Regards,

Jaime
 
Sorry to confirm the issue but if there is no copy of something like SQL Server or another DB product on the location where the MDB is located, then that server is just a file server. All the work is done by the Access image on the workstation. The WHOLE TABLE must be moved to be filtered.

There ARE ways to improve performance, though.

Something called a front-end/back-end (FE/BE) split makes it more efficient to load the queries,forms, reports, macros, and modules. Put the FE on the local workstations. Only the remotely located tables have to hit the network.

There is also a strategy for splitting tables that normally have long text fields. If you store a lot of codes to be translated for reports but they don't matter for other transactions, you can shorten the size of the records, which means that each buffer load holds more records, which reduces the number of I/O calls required to transfer the whole table.

For this next one, I think there is some truth but it is not something I have researched that closely. If you have good indexes on the main fields of the table and a stored query, it is possible that a query filtering solely on an indexed field WILL pull down less than the entire table. The down side is that you can have too many indexes for efficiency in STORING data. It is a balancing act to get the right number of indexes.
 
Last edited:
Thank u for your reply, DocMan, I appreciate.

Regards,
Premy
 
Unless you do something to force client side processing, see for instance #9 here http://www.fmsinc.com/tpapers/faster/index.html#Client/Server Performance Tips, Jet will examine the indices, and bring over only the records matching your criteria.

This is not the same as Client/Server, where the whole process takes place on the server, but it is Jets way of using our proper design. First examine the indices, then fetch only the relevant records. Jet, afterall is a database, and the Jet engine is a database engine.

See for intance the help file (this is from A2000, topic "Optimizing performance", subtopic "Improve performance of linked tables")

"Use filters or queries to limit the number of records that you view in the form or datasheet. This way, Microsoft Access can transfer less data over the Network" [highlighting by me]

Whether or not using a filter or a query reduces the initial load time of a form, is however a different matter. The Jet engine has two strategies of fetching information.
- Complete the query, then show the data
- Display data, then complete the query

We cannot control which method Jet uses, but the latter one, is often the reason why we will rather quicly see the first record(s) in a form, but the number of records won't display in the "current record" box by the navigation buttons until the whole query is run (and hitting the "last record" button, might give us a coffee break).

Say you want only the last couple of hundred records available initially, using a query with a criterion on an indexed field (date field?), utilizing the index, will give you significant performance gain over binding the form to the whole table and move to the end.

This should demonstrate or verify that Jet will NOT move the whole table over the network to filter it, unless we do something <insert appropriate term here> to force client side processing, which means WayneRyan is correct.
 
Wow, Roy thanks for your reply, this re-launches the debate. Like I said at the beginning there's still seems to be quite some confusion concerning this topic, even among experienced users.

I'm glad to see though that Jet isn't altogether unsuited for stuff other than an inventory of one's DVD collection.

Regards,
Premy
 
Note that Roy's reply explicitly depends on indexes being available. If you have no indexes and it is an Access database with the tables on a remote machine (regardless of where the form/report/query is located), Access MUST pull in the whole table.

With an index and a stored query, you have a chance of doing less work. Without the index, you will read the whole table. Without the query being stored, you will read the whole table (because Access doesn't have a clue beforehand on the best method to use whatever the table has.) The stored query CAN be a parameter query and still count as "stored" for this purpose.

Although I did use Jet/Access for my CD collection, many small businesses are indeed using Access for their shop. For a small business it is still quite capable. When the business ceases to be small (and that determination is solely the domain of the business owner/staff), Access is STILL relevant as a front-end interfacing tool to other database tools like SQL Server or ORACLE.
 
> Note that Roy's reply explicitly depends on indexes being available.

Yes, this is one of the reasons indexing is usually recommended for columns you wish to filter or sort on, and also why it is particularly important in a FE/BE setup. See #119 and #120 in the link I gave.

Putting an index on the foreign key, though, isn't necessary, as that is done by Jet when creating the relationship, though if you join on non-key/non-indexed columns, indices on both sides will be beneficial.

> If you have no indexes and it is an Access database with the tables on a remote machine (regardless of where the form/report/query is located), Access MUST pull in the whole table.

Gee, you sound like you don't like indexing, and/or don't expect people to use it ;)

> Without the query being stored, you will read the whole table (because Access doesn't have a clue beforehand on the best method to use whatever the table has.)

Nope, Jet will optimize also dynamic SQL. Fact is, dynamic SQL can even be faster than precompiled querydefs, if there's been a while since they were compiled.

Try test whether it loads the whole table or not, try the following simple test in a form, remove the recordsource, and have one button for filtered and one button for All, with the code similar to the following

Me.RecordSource = "SELECT <the columns> FROM myTable"

and

Me.RecordSource = "SELECT <the columns> FROM myTable WHERE <SomeINdexedField> = SomeValue"

To diminish the importance of buffers and caches, one might at least close Access between each try. You should see significant difference between retrieving a couple of hundred records and retrieving the whole table.

I don't know how to time the opening of forms, but the filtered version, which gave 100 records, used a bit less than one second, the non filtered version, about five seconds. There is a total of 360 000 records).

Had your assertion been correct, that without a stored query, it would read the whole table, then I doubt anyone would be using dynamic SQL when opening recordsets, but it turns out the Optimizer does a fine job also when you don't have precompiled querydefs.

It is very easy to verify, try a dynamic string of the type

SELECT <the columns>
FROM myTable WHERE
SomeINdexedField = SomeValue

vs

SELECT <the columns>
FROM myTable WHERE
Some_NOT_idexedField = SomeValue

With my setup, and 360 000 records, the Time function is enough to tell the difference.

dt = time
set rs = currentdb.openrecordset(theSQL)
if rs.recordcount>0 then
rs.movelast
debug.print rs.recordcount
end if
debug.print format$(time-dt, "hh:nn")

This takes 26 seconds on the non indexed table, and I was finally able to make the query on the indexed table return 1 second one or two times, by repeatedly hitting F5 several times, but the Time function had problems recording this efficiency ;)

Even without traversing the recordset, it uses 21 seconds just to open a recordset on the non indexed table with the criterion.

Opening the table and looping to find the same records, takes 28 seconds, using .FindFirst/.FindNext takes the same (28 seconds)

This again stresses the need to do proper indexing!

Another interesting thing, which is also indicated by Pat Hartman here http://www.access-programmers.co.uk/forums/showthread.php?t=70952, you might even experience that if you join a local table with a ODBC linked table, Jet will only bring the relevant records over. Jet will at least try to optimize, so that it doesn't have to bring it all over.
 
Thanks again guys, for enlightening this debate

Regards,
Jaime
 
As I suspected, based on other readings, there does not seem to be a real consensus on the subject.
Wow! Still no consensus. eh?

I got a chuckle out of that because I was asking the same question back in 2001 on the old Compuserve Forum. Basically my question was this:
Is Access - and I'm talking about the back-end here - act more like a file server or a client server? Specifically, here are three queries:

1. SELECT * From Clients;
2. SELECT * FROM Clients WHERE ClientID = 123;
3. SELECT * FROM Clients WHERE Address Like "123*"

Let's assume the Clients table is pretty big - a million or so records. Let's also assume that ClientID is the PK and thus indexed, and Address is not indexed.

According to some, the Access back-end will behave like a file server and return the entire Clients table for each of the three of the queries and the Access front-end will have to do all the work to figure out which records to present.
The general consensus was that #2 would run faster because of the index, while the other two would return the entire Clients table. Still there were a lot of unanswered question about how the JET engine worked.
 
Jet is good but it is not a mind-reader.

I believe there IS some consensus here.

1. No indexes? Jet pulls down everything.

2. Indexes and the query can use them? Jet works with indexes first, which is typically smaller.

3. FE/BE split? Jet works better because the FE components are local.

4. BE isn't Access but instead is something like SQL Server? Jet works best when it has help at the remote end.
 
I'm happy to know that Jet does do some pre selecting, for it saves me the trouble of having to move all my backends to sql server. I now see it like this: for small to medium business applications jet will do very fine. It's only when creating big corporate/e-commerce/critical apps that it would be worth the trouble, time and money to look for a solution like sql server. Even then access can still act as front end, even though MS seems to have stopped further ADP developpement (which I think is a pity for it has obviously been dumped to "force" developpers wanting to code against sql server, to purchase Visual Studio 2005).

Jaime
 
...to "force" developpers wanting to code against sql server, to purchase Visual Studio 2005)
Not so! MDB's and MDE's handle SQL Server as a BackEnd just fine, just use queries and not the tables directly.
 

Users who are viewing this thread

Back
Top Bottom