Don't unload reference files?

So i accept that Connect is slower then v8i when i open a big layout or a GA with 15 references it takes a solid 1 or 2 minutes to open if i'm lucky this buys me a bit of time to clean up my email inbox.  But then i considered i'm often shifting from one drawing or model into another with the same or many similar reference files.  So not only do i watch the software seemingly dump these files but then go to the exhausting effort to re-load them all.

So i guess does the software really do this?... and if so why?  If i click file-open and open a new file without closing the one i'm in first, why can't the software do a quick look and see what references and for that mater shared cells are also in the next file and avoid dumping all of it to just re-open all of it.  Maybe i'm crazy but i feel like this could improve file load times so long as i don't actually "close" the file i'm in before opening another.

Yes i know this likely would serve better as an IDEA but lets be honest who goes and digs thru the IDEA's section every week to find new ones, also there isn't a category for "make software more efficient" so i'm throwing it to the wolves here if a moderator wants to move it feel free.  I also genuinely don't know how MS handles opening files so it might be already trying to do this or an impossibility.

-Grant