While a /whowas may work, if you're going to be doing this on a regular basis, you might want to start doing a /whois on joining the channel, then storing the IP for a few minutes so that those that stay have the IP removed automatically, but those that leave quickly have the information stored permanently.
This would require more of a code than the /whowas command, but does have advantages.
BTW: Personally, I never would've started putting sensitive information into the channel topic, as the channel topic is displayed in a /list (unless the channel is marked Secret or Private)