I'm going to express doubt. Not certainty of failure but doubt of success.
Are you talking thumb drive or external USB-connected hard disk? One thing for sure is that you would have to have a reliable URS path for that USB device. If it is a removable device, you probably will always have to run it from the slot that was in use when the original installation took place; changing connections would probably bollix up all of the linkages. In order to make Access run off of a router, you have to take into account whether the registry links to the other separate modules of Access (like the Ace DB engine and the reference libraries) are consistently linked.
In theory, if you installed Access as an original installation to a disk-like device (I'm counting rotating and solid state equally here), it might be possible. But the problem is likely to be the "baggage" that goes with such an installation. For instance, I don't think you can easily make a copy of the disk installation and make it work very easily. Not all files installed by Access go in .Windows.System32 or .Windows.System folders. Some will go in .ProgramFiles, for example.
You probably would also want to use a fast USB circuit so you could use USB 3.0 standards. USB 2.0 is far slower regardless of the physical medium.
Peter, I don't want to rain on your parade because I know better than to say this is impossible. But I would rate this as an extremely difficult goal to reach. In terms of software engineering, I would count this as probably requiring a few days of installation prep and at least two full weeks of hard testing with things like Application Object and OLE testing. In engineering terms, this would class as a high-risk project with high risk of failure.