Add this to your ON *:XXX event used to reply or other event handler. Since you are light on details, that's about all I can do for the moment.

if ( $nick !ison $chan ) { reply }