It seems you already know the answer.
The help file is not incorrect, it's just incomplete, but that's not a new accusation as the help file is rarely that detailed. $and() *does* return binary AND of A and B, it just does it with a 32-bit operation. That part is simply undocumented (because mIRC is not a language that needs to concern itself with bits and bytes).
The expected result should take into account this 32-bit limitation.
Maybe at some point mIRC will support 64-bit numbers, you can suggest this, but there are other ways to handle that.
