Why not use a snipper for a temporary fix? i.e:
Code:
on *:invite: { /* Not sure if i used this wrong, my PC broke few days ago :(*/
 if (#* !iswm $1) ignore -u3600 $address($nick,2) /* Not sure if i used this right eather :S*/
 else join $1
}

this should let your autojoin work, and ignore the user that invited you if the name of the room isnt a channel.
i used somthing like this, just i made a list of all my frends nicks, and it checked if they were identified with nickserv(better than host, because they went to school/work/ect (chenged their hostname))