MajP gave you the short answer. Here is the more detailed answer.
When you have an active reference and the program gets launched, what happens during the launch is that the image loader (part of Windows task activation services) "diddles" with the memory management tables to assign a chunk of virtual memory to be use for that referenced module, which is usually a .DLL or Dynamically Linked Library. This allocation is permanent in the sense that Windows does the same image diddling every time you launch the app with that reference still checked. Of course you can uncheck it to get back memory space, but your question is "what if you DON'T uncheck it?"
What is rarely appreciated is the structure of your virtual task image in Access. Every forum member sees posts discussing how Windows imposes a 2 GB size limit on the data in the app; as for example if you split the DB into 2 files, the BE has a 2 GB limit. But so does the FE, and that part of your task structure is where memory space is allocated for the referenced libraries. If you have a LOT of libraries that COULD expose their internal structures for automation / COM operations, you have a lot of drains on your FE virtual memory. That section of memory also has to hold Access itself, your program stack, and your program heap. Every checkmark on a library reference takes statically allocated memory out of that 2 GB limit for the FE leaving less space for the dynamically allocated (run-time) working memory of the FE.
Depending on just how many of those .DLL references you have, each one chewing up some amount of 32-bit virtual memory, you could start to see "Out of Memory" or "Resources Exceeded" or "Stack Overlaps Heap" or "Virtual Memory Too Large" (all of which have been reported on this forum over the years.)
Going to 64-bit Excel DOES increase the potential virtual size of the image ... but going to 64-bit Access does not appear to do so. The bitness-change for Access was more for compatibility with other Office elements, but it had little or no effect on the addressing structure within Access. So your addressing in a 64-bit Access is still 32-bit for all of the object pointers within Access itself. And besides that, most of us recommend staying with 32-bit Office unless you absolutely need Excel spreadsheets with a million rows, mostly because of compatibility with many of the library files that were NOT converted to 64-bit anyway.
The one error that usually comes up first is the one about resources, particularly if you are doing something that involves a lot of files, because a file requires buffer structures associated with file handles. This file activity includes IMPLIED files - such as doing Excel import or export operations, each of which opens a file. When you open a file in a given process, new buffer structures get created. However, even if you close the structures, the Windows file system is going to allocate a NEW file handle rather than re-use the old one. (It's an implied system call in the WIN32 API.) This means that the buffers aren't dissolved or re-used until the process exits or a LOT of files have been opened. So a lot of file ops in the situation you describe would lead to one of the many possible memory errors, the most likely of which is about Resources.
I have one minor correction for MajP's comment. You can choose the order of appearance of a reference and it CAN alter the nature of the entry-point search. It is true that the best way to define objects associated with these references is to fully qualify them. It is also true that, left alone, the libraries appear alphabetically. But for a while a decade ago, MS had both ADO and DAO libraries that needed to be activated, and because they were alphabetic, the ADO libraries were always giving us ADO recordsets when we meant to use DAO recordsets. So the #1 solution was to fully qualify all potentially ambiguous references, but the #2 solution was to change the order of references to make DAO come before ADO in the internal search. And yes, that worked.
Which leads me to the last possible effect... reference "collisions." Here's the simplest case: ADO vs DAO collisions on .Recordset objects. But now multiply all of the possible names used in all of the possible libraries, and realize that you have to QUALIFY each reference, as we had to do for DAO.Recordset vs. ADO.Recordset, and realize how much more typing you have to do, because the odds are that everything will have to be fully qualified in code.
THAT is what is involved when you check every possible reference.