That's how it was previosuly setup.
Replace the invite, and only users on the !add list can use the invite command.
on
invite:TEXT:!invite &:?: ...
invite means those on your add list. Any command you want only those users to be able to use, use the invite user level.
[edit]
As far as joining, they will be able to. If you want the script to ban the user, replace the /kick command with the following:
/ban -k $chan $2 2 $iif($3,$3-,Kicked by $nick $+($chr(40),!remove command,$chr(41)))