I'm assuming that once join -i opens a window it gets taken off the list
Yes, exactly, that is how it works. Once it is used, or the channel is joined in any other way, the last invite channel is cleared. That was the design decision at the time it was implemented.
