A single building can have multiple units, and a single unit can exist in different buildings.
If this is true then indeed you have found a situation requiring a junction table (many-to-many).
To set up a junction table, create it with not less than two fields that will be FOREIGN KEYS; one of them matched up to the PRIME KEY of BUILDINGS and the other to the PRIME KEY of UNITS. You would have one-to-many relationships with the "many" side on the FKs in the junction table and the "one" side on the respective PKs. For those cases where you have a unit split between buildings, you will have (at least) two records in the junction table, both with the same unit number but different building numbers. But where you DON'T have a split unit you still must have ONE entry in the junction. The side effect of this situation requires that you can no longer look at the BUILDING or UNIT table for location information EVER, because sometimes you have two answers. Even if for most cases you only have ONE answer, Access requires you to always look in the same place for BUILDING/UNIT relationships. If you HAVE a junction table, THAT is where you must always look.
I said "not less than two" fields because it CAN occur that there would be something to be stored that is neither specific to the building nor specific to the unit. As a
contrived example of such "extra" data, suppose that when you have a split unit, you need two different keys and thus need to record two different key numbers. And as part of the contrivance, the keys are different from each other AND not necessarily related to the building they are in. Thus you would need a place to store key IDs that neither match up to the building nor match up to the single unit. That would be an example of extra fields in the junction.
As to how you deal with them on forms, they can be a pain - but the good news in your case is that if you DON'T have a split unit, you just make one entry with the BUILDING ID and the UNIT ID as the two FKs and you are done. So maybe if you have a split, you would have a check-box on a form that says "SPLIT UNIT" and if you check it, a second control opens up to allow you to define that second element of the UNIT. But you might prefer some other way to control this and, I have to say it this way, the GUI is up to you since there are so many ways you COULD want to do this.
In reports, MOST of the time you would either group by BUILDINGS or group by UNITS. So your query would be based on a JOIN between the junction and either BUILDINGS (if grouping by buildings) or UNITS (if grouping by units). This can be an INNER JOIN because from the description, there will never be a case of a "dangling" UNIT or a "dangling" BUILDING. You use LEFT/RIGHT JOINS only when you can have "danglers." In the detail section of your reports, there will be TWO detail records for split cases and one for non-splits.
You can always post more questions, of course. Hope this gives you an idea of what a junction looks like.