When you do $regex(test,expression), mIRC passes that expression to pcre_compile2(), and if compilation fails, mIRC store in $regerrstr a string corresponding to the error (given by pcre).
When you do $regex(test,??) pcre_compile2()/$regerrstr returns "nothing to repeat", which is correct.
After a pattern is compiled, it has to be executed via pcre_exec().
The PCRE demo code shows that after an error from pcre_compile (or pcre_compile2 really), you should not be trying to execute with pcre_exec(), which mIRC seems to be doing in this case, because $regex(test,??) returns -14, an error from pcre_exec().
Note that -14's error description does not match this context and that mIRC reports -14 for some others error reported by pcre_compile2(), ex: $regex(test,(?<=a*))
While I believe this is wrong from a programmer point of view, in practice, nobody uses $regerrstr, (which was undocumented and only found recently) to invalidate a regex match where $regex would have returned a non negative value (including 0 for no match, which doesn't mean there was an error).
There's no real relationship between pcre_compile() and pcre_exec; in practice, I believe that $regex returning a negative value in this case is much more useful and unlikely to surprise scripter and break scripts checking the returned value of $regex.
I was wondering if mIRC was smartly doing this on purpose or if this was just the result of using pcre_exec()?
Again, remember that returning 0 for $regex() would mislead scripters to think there was no match when an error actually occured, because scripters have not been trained to use $regstrerr, so I don't think $regex should return 0, I think it should return a negative value because the compilation failed and $regex kind of has the responsibility to provide an error for both compile/exec functions. Error codes for the exec() function go up to -32, pcre is a stable project with no new feature added to it (only in pcre2) so maybe returning a negative value larger than -32 (well, smaller then) could be a way to indicate to scripters that an error occured at compile time, in which case we could check $regerrstr anyway. Again this is according to the fact that people are not using $regerrstr currently.