Well I think you may have figured out what's happening yourself, but I'll break it down anyway just in case:
The problem is that your $replacex() call is replacing "[username]" with the contents of %username - not the text "%username" - so when the whole contents of the %reply_message is evaluated mIRC sees the line $iif( == $null, ... which is of course invalid syntax.
I think you got that far, in which case we just need to explain why this isn't a bug but rather the desired behaviour. Simple really: If I type the code if ( == $null) ... into a script file directly, what do you think I meant there? There's really any number of ways to interpret that and none of them are obviously the right choice, what you mean in your case is probably the least likely to be the correct choice because in any typical situation (ie. where that script was hardcoded) you obviously wouldn't type if ( == $null) ... because it would be redundant. You're expecting the script parser to guess what you meant, and that's a bad idea - far better for you as the scripter to provide clear code. In your case that means $replacex(%reply_message, [username], %[color:red] $+ username)[/color]
