mIRC Homepage
Posted By: NoMiaus Duplicated ban detection - 20/03/17 08:28 PM
Bug when using mIRC 7.47 and the latest UnrealIRCd version.
1. /mode #Channel +b ~r:xyz
2. /mode #Channel +b ~R:xyz
3. Only the last ban set is shown on the ban list window. I've to detele the second ban added to see the first ban added.
Posted By: Raccoon Re: Duplicated ban detection - 20/03/17 11:36 PM
Hmm, that'd be an interesting one to fix. Ban masks are expected to be case-insensitive, and UnrealIRCd decided to started adding case-sensitive flags to case-insensitive ban masks.

I'm tempted to say this is a bug with UnrealIRCd. They -should- fix it on their end. I'm curious how other clients handle this.
Posted By: Khaled Re: Duplicated ban detection - 21/03/17 09:47 AM
Thanks for your post. There seem to be two issues with UnrealIRCD's banlist changes here:

1) The ban list is meant to consist of non-case-sensitive addresses. So this will break case-insensitive handling throughout mIRC of anywhere that these addresses are used both by mIRC and scripts.

2) The "r:" prefix character will break use of these addresses if a client/script assumes that they consist of nick!user@host.

Basically, the ban list cannot be treated as a list of addresses any longer.
Posted By: Raccoon Re: Duplicated ban detection - 21/03/17 11:01 AM
Of note, Charybdis (and ircd-seven aka freenode-seven), also permits flags and extraneous info in banmasks. Here are some examples.

nick!user@host$#redirect-channel
$x:nick!user@host#gecos
$r:gecos
$a:account
$~a
$j:#borrow_from_our_banlist

$x:raccoon!fuzzy@woodland.critter.net#I'm?a?little?teapot.$#you_are_banned_go_away

... but in every instance these are case-insensitive.
Posted By: Khaled Re: Duplicated ban detection - 21/03/17 12:53 PM
Thanks for the clarification. Would you know if each ircd uses a different method for formatting banmasks? Or is there an agreed format? Even so, case-sensitive masks would require special handling, so a numeric 005 token would be needed to identify networks that use them.
Posted By: Raccoon Re: Duplicated ban detection - 23/03/17 03:54 AM
In the Charybdis family tree of IRCd's (which is a fork from Ratbox, la Hybrid) they call this extension EXTBAN.

Code:
:hobana.freenode.net 005 Raccoon EXTBAN=$,ajrxz 

-> Server: help extban
-
extban MODE <channel> <+|-><b|q|e|I> $[~]<type>[:<data>]
extban  
extban Extended bans (ban conditionals) allow different checks than the usual
extban nick!user@host or nick!user@ip match to determine whether someone should
extban be banned, quieted, exempted or invited.
extban  
extban Extended bans are of the form $[~]<type>[:<data>]. The <type> is one
extban character (case insensitive) and determines the type of match. Most types
extban allow or require an extra field <data>. If the tilde (~) is present, the
extban result of the comparison will be negated, unless the ban is invalid in which
extban case it will never match. Invalid bans are ones where <data> is missing but
extban required or where <data> is otherwise invalid as noted below.
extban  
extban Unless noted below, all types can be used with +b, +q, +e and +I.
extban  
extban  extb Type  - DESCRIPTION
extban ------------------------------------------------------------------------
extban      $a     - Matches all logged in users
extban   $a:<mask> - Matches users logged in with a username matching the mask
extban               (* and ? wildcards)
extban   $c:<chan> - Matches users who are on the given channel; this is only
extban               valid if the channel exists and is not +s or +p. (The ops
extban               of the channel the ban is on cannot necessarily see whether
extban               the user is in the target channel, so it should not
extban               influence whether they can join either.)
extban      $o     - Matches opers (most useful with +I)
extban   $r:<mask> - Matches users with a realname (gecos) matching the mask
extban               (* and ? wildcards); this can only be used with +b and +q
extban   $s:<mask> - matches users connected to a server matching the mask
extban               (* and ? wildcards); this can only be used with +b and +q
extban   $j:<chan> - matches users who are or are not banned from a specified
extban               channel
extban   $x:<mask> - Bans all users with matching nick!user@host#gecos
extban      $z     - Matches all SSL users
extban  
extban End of /HELP.

I understand this help may be missing one or two.

Also, there have been no breaks or faults in mIRC for years and years, and nobody honestly expects you to add support for these. I don't think any client scripting language automatically does. I mean, maybe $a:<account> and $a and $~a would be doable. Seems a bit far out there.
Posted By: Khaled Re: Duplicated ban detection - 23/03/17 02:53 PM
Thanks, that really helps. I am sure I have seen the EXTBAN token before but never needed to cater for it in any specific way.

The help reply you listed states that the type character is case-insensitive on freenode. I cannot see a way of supporting case-sensitive type characters on UnrealIRCd without breaking the ban/except/invex/quiet lists on other networks, since users can issue /modes using different cases for both the type and address on case-insensitive networks.

UnrealIRCd would need to add another numeric 005 token or EXTBAN flag to identify that it uses case-sensitive type characters. A client/script would then have to split the address at the : character and perform a case-sensitive comparison on the type and a case-insensitive comparison on the address.
Posted By: Sat Re: Duplicated ban detection - 23/03/17 03:45 PM
I suppose that a simple workaround would be to check for presence of both the uppercase and lowercase variant of any letters in the EXTBAN= token. UnrealIRCd would have both 'r' and 'R', which would never be the case on a network that treats them as case-insensitive. Not that I'm advocating supporting all these weird UnrealIRCd features..
Posted By: Khaled Re: Duplicated ban detection - 23/03/17 04:10 PM
Yes, I was considering that. It seems like a reasonable assumption, as long as a network never sends type characters that it has not listed in the EXTBAN - which should be the case.
Posted By: Raccoon Re: Duplicated ban detection - 23/03/17 04:34 PM
LOL. "Should be the case."

I learned that many/most of these IRCDs have their 005 hard coded like an MOTD. Freenode disabled MONITOR about a year ago and still leaves it in their 005. It's static decoration, and not actually pulling from a config file. crazy

But yeah, I came to the same conclusion as both you did. Check for case alternation in the 005 flags.
Posted By: Khaled Re: Duplicated ban detection - 23/03/17 05:13 PM
Right. Just when you think you've figured it all out, along comes:

-> card.freenode.net MODE #thisisatest +b ~R:xyz
<- :user!nick@host MODE #thisisatest +b *!*@~R:xyz

Whereas on UnreadlIRCd:

-> irc1.unrealircd.org MODE #thisisatest +b ~R:xyz
<- :user!nick@host MODE #thisisatest +b ~R:xyz

Not too sure what freenode is doing there.

Update: A little more testing on UnrealIRCd shows that it allows:

-> irc1.unrealircd.org MODE #thisisatest +b *!abc@~r:def.org
<- :user!nick@host MODE #thisisatest +b *!abc@~r:def.org

-> irc1.unrealircd.org MODE #thisisatest +b ~r:*!~r:abc@~r:def.org
<- :user!nick@host MODE #thisisatest +b ~r:*!~r:abc@~r:def.org

So the $[~]<type>[:<data>] format can appear anywhere inside an address and more than once.

Update: I've implemented the case-sensitive/case-insenstive dual-comparison split based on the first : in the address. This will only be enabled for networks that contain two different cases of the same character in EXTBAN. This will be in the next beta.
Posted By: Sat Re: Duplicated ban detection - 23/03/17 08:32 PM
Well, the EXTBAN prefix on FreeNode is '$', not '~'. Also, are you sure that UnrealIRCd actually interprets those non-prefix occurrences as extended bans and not simply as part of the ban address mask?
Posted By: Khaled Re: Duplicated ban detection - 23/03/17 09:27 PM
Yup, you're right. I was assuming that : wouldn't be allowed as part of a hostname but that depends on the server, IPv6, etc. So looking for the first : will not be enough. In order to check for a type prefix, mIRC will have to check that the prefix character specified in the EXTBAN is at the start of the address and only then look for the : character.

Update: Drats. Looking at this document on the UnrealIRCd website, it says that you can stack extended bans (see bottom of page). So looking for the first : will not be enough. An IRC client will have to scan through all stacked type prefixes ~q:~c: to make sure it has found the whole of the case-sensitive section of the address. This is assuming all servers implement this the same way.
Posted By: Raccoon Re: Duplicated ban detection - 23/03/17 10:51 PM
Not all EXTBANs on Freenode will contain a ':' colon, only those that carry parameter data with it. And though extban flags can't be stacked in Freenode, a forwarding '$#channel' can amend any ban, and colon can be used in those channel names as well as IPv6 hosts.

Code:
MODE +b $a          ... ban all registered users.
MODE +b $~a         ... ban all nonregistered users.
MODE +b $a$#c:olon  ... send registered users to channel #c:olon
MODE +b $z          ... ban all SSL users.
MODE +b $~z         ... ban all non-SSL users.
MODE +b $~z$#bu:tt  ... send non-SSL users to channel #bu:tt
Posted By: Khaled Re: Duplicated ban detection - 24/03/17 01:45 PM
Well, this is getting increasingly complicated and the odds are that I will implement an approximation to how different ircds parse these lines, which might work in some cases but not all.

One solution to all of this is to just avoid performing comparisons on these addresses. The original issue is that mIRC is not seeing all of the addresses in the list. This is because mIRC is maintaining/synchronizing its own internal lists when it sees mode numerics, to avoid excessive /mode requests to the server when channel central is used. This is what requires address comparisons.

I will remove this feature so that mIRC no longer needs to maintain internal ban/except/invex/quiet lists and instead requests them every time you use channel central. That seems to be the only practical solution at this time.

This will not solve everything - the auto-unban feature needs to compare addresses in various situations. However, I can make it case-sensitive and the odds are low that someone will use different case text when adding/removing bans. If they do, the worst that can happen is that mIRC will fail to remove a ban at some point.
Posted By: Sat Re: Duplicated ban detection - 24/03/17 02:21 PM
That sounds reasonable but what about $ibl etc then though?
Posted By: Khaled Re: Duplicated ban detection - 24/03/17 02:40 PM
These would return the most recently looked up list. And .ibl/etc. would return $true/$inmode/$false as usual, however this would be reset to $false the moment a new mode is seen.
Posted By: Sat Re: Duplicated ban detection - 24/03/17 03:04 PM
Hm, but then the contents of $ibl may still end up being "wrong" (in the sense of the original post here) if mIRC retains its fully case-insensitive ban tracking based on MODE +b/-b changes?

Also, that means that after a ban list has been requested already, half the ban list may still be missing from the $ibl list during a new channel-central-incurred banlist update (ie during the $inmode state) from a script's point of view?

I think this does have the potential to break some scripts, especially those implementing their own channel central kind of thing..

I don't know, is telling UnrealIRCd to get their shit together not a better option after all? smirk
Posted By: Khaled Re: Duplicated ban detection - 24/03/17 03:26 PM
Quote:
Hm, but then the contents of $ibl may still end up being "wrong" (in the sense of the original post here) if mIRC retains its fully case-insensitive ban tracking based on MODE +b/-b changes?

mIRC would not be tracking or storing the contents of mode changes - all it would see is that eg. a new mode +/-b had arrived, so it would change the state of .ibl to $false.

Quote:
Also, that means that after a ban list has been requested already, half the ban list may still be missing from the $ibl list during a new channel-central-incurred banlist update (ie during the $inmode state) from a script's point of view?

This has always been the case though. That is why $inmode was added, so that scripts would know if it was being looked up.

Quote:
I think this does have the potential to break some scripts, especially those implementing their own channel central kind of thing.

I am concerned about that, however it seems okay so far... if a script is checking for the .ibl value, it will continue to work as before. If it isn't checking for the .ibl value, then it never had a guarantee that the list was filled anyway.

Quote:
I don't know, is telling UnrealIRCd to get their shit together not a better option after all?

It sure would be :-)
Posted By: Sat Re: Duplicated ban detection - 24/03/17 03:49 PM
Originally Posted By: Khaled
mIRC would not be tracking or storing the contents of mode changes - all it would see is that eg. a new mode +/-b had arrived, so it would change the state of .ibl to $false. [..] This has always been the case though. That is why $inmode was added, so that scripts would know if it was being looked up.

Oh I see. Yes, I do think that is very likely to break scripts then: as far as I know -correct me if I'm wrong- it is currently the case that after the the ban list has been requested once (after joining the channel), a script could assume that the IBL would accurately reflect the bans set on the channel. As such, a script could issue a "/mode # b" on join and, after the final numeric reply, assume that the IBL would be in good shape for as long as the client is on the channel. This change would break that assumption and as such even make $ibl nearly useless, I think..
Posted By: Khaled Re: Duplicated ban detection - 24/03/17 04:52 PM
Good point. So it looks like this would break all existing scripts that use these features. I am going to avoid making any changes for now since there does not seem to be a viable solution. The best solution, that would avoid breaking clients/scripts, would be for UnrealIRCd to ensure that the ban/etc. lists are case-insensitive.
Posted By: Raccoon Re: Duplicated ban detection - 24/03/17 06:29 PM
If you are needing a way to identify conventional bans, versus bans prefixed with flags, I would check if the first character of the banmask matches none of: A-Z a-z ][}{\|^`_- ?* 0-9 and possibly: . : /

In theory, ban flag prefixes are not going to use any of the following: Qualified nickname characters nor Wildcard characters. Possibly wouldn't use qualified DNS characters '.' and ':' (though '~' is found in ident). Probably wouldn't use '/'.

Charybdis uses '$', and UnrealIRCd uses '~' for its flag prefixes. Let's just start by assuming these are the only possibilities for abnormal ban address masks that might require special handling or case sensitivity.
Posted By: Khaled Re: Duplicated ban detection - 24/03/17 09:25 PM
The issue is that I would need to split the extended ban address into the case-sensitive and case-insensitive parts in order to compare them separately. For example, on UnrealIRCd, these would be identical:

~r:abc!def@ghi.org
~r:abc!DEF@ghi.org

case-sensitive comparison: ~r and ~r
case-insensitive comparison: abc!def@ghi.org and abc!DEF@ghi.org

And these would be different:

~r:abc!def@geh.org
~R:abc!def@geh.org

case-sensitive comparison: ~r and ~R
case-insensitive comparison: abc!def@ghi.org and abc!def@ghi.org

So I would need to parse the extended ban prefixes all the way to the data parts in both addresses, so I can compare the prefixes case-sensitively and the data parts case-insensitively.

In order to do that, I would need to know:

1) The exact method that the ircd itself uses to parse the prefixes, to make sure I do it the same way.

2) That all ircds use the same method. If they do not, it will fail on different ircd implementations.

This is why I wanted to avoid the above by not performing tracking. But that would break features/scripts that have worked for 15+ years.

I have already implemented a routine that extracts:

EXTBAN=$,ajrxz (on irc.freenode.net)
EXTBAN=~,SOcaRrnqj (on irc.unrealircd.org)

And uses the specified prefix. It then stores and scans everything after , and determines if extban types are case-sensitive or not. If case-sensitive, a banmatch routine parses the ban addresses to be compared, checking for a prefix (required), a ~ negation (optional), an extban type character (optional), and a : colon (optional), repeatedly (to handle stacked types) until no more prefixes are available. It then assumes whatever is left is the data part. It then compares the extban section case-sensitively and the data part case-insensitively. However, there is no way to know if this will work across ircd implementations.

I am also going to add a $ibl() .extban and .extdata property that return the split information. That might help debug the routine to see if it is working correctly with different extbans.
Posted By: Khaled Re: Duplicated ban detection - 24/03/17 10:52 PM
Okay, it looks like the parsing method needs to be more sophisticated than the one I described above.

If the extbans are stacked, they can contain multiple extban characters and data, all of which need to be parsed and split separately and compared case-sensitive/case-insensitive per extban:data combination.

-> irc2.unrealircd.org MODE #thisisatest +b ~q:~xyz!abc@lmn.org~c:#pumpkin~T:censor:*badword*
<- :nick!user@host MODE #thisisatest +b ~q:~xyz!abc@lmn.org~c:#pumpkin~T:censor:*badword*

Notice that the ~ prefix can also be used in data, so it is not as simple as spltting the line into parts based on the prefix character. We will need to go character by character, validating each character, and its formatting, against the EXTBAN list, and splitting them into multiple extban:extdata parts.

Really, the only way to be sure we are doing this correctly is to look at the ircd source code. The odds are that UnrealIRCd and Freenode will do this differently, which means that an IRC client cannot parse this information without using a different method for each ircd.

I am going to leave this issue for now, since I cannot see a practical solution.
Posted By: Sat Re: Duplicated ban detection - 24/03/17 11:08 PM
Originally Posted By: Khaled
I am going to leave this issue for now, since I cannot see a practical solution.

That does sound like the best option. What a mess..
© mIRC Discussion Forums