I am currently testing this with an IRC bouncer and the Slack IRC bridge, whenever the server reconnects after being disconnected, it will open all windows. This may be due to the servers sending rejoin messages.
I also have the setting "keep channel windows open" enabled (but not rejoin on connect)
