Legitimate? There is no illegitimacy when it comes to ops banning people from their own channel.
I have banned people for being in bad channels before, mainly because a channel is very well known for harbouring flooders, or channels which facilitate child pornography. Some channels mainly set up for younger people (irrespective of who thinks younger people shouldn't be on IRC) don't want to have people who are in #horsesex on #irc4kids.
Another example, following on from the last, is that some channels are for 18+ or for other certain age groups, and so will detect 'kid' channels, or vice versa.