Forums Bug Reports Continue/While/Bracket bugs

The 1st 3 aliases are identical commands, except /testcontinue-1 has everything in 1 long line, while /testcontinue-2 and /testcontinue-3 use different places to split it into 2 lines. I include a 4th alias /testbraket having a bug that seems unrelated, except that it appears to have begun at the same time as the /testcontinue-2 behavior, so including the 4th alias might help track down the issue.

/testcontinue-3

This has the correct behavior in all versions I've tested, including the current one. It continues the loop all the way through %i == 7

---

/testcontinue-2

This has the correct behavior in all versions I've tested up through v7.38, but in v7.41 it fails, probably due to the changelog 7.39 item:


The /testcontinue-2 bad behavior is leaving the while() loop at i=2, i assume caused by the check for i=3, but it doesn't even display the 'goto next' message before leaving the loop.

---

/testcontinue-1

This has the correct behavior up through 7.67 but fails after that, so is probably related to changelog 7.68 item:


The bad behavior is is that it executes the same /var command twice in a row even though there should be a 'before varset' and 'after varset' echo that should display between each execution of that /var command. I used the $$ to call attention to the repeat error rather than let it continue to the end.

* * *

The 4th alias /testbraket is the other bug which began at what I assume is the same time as the /testcontinue-2 bug, but I can't test the 7.39 behavior. The bad behavior now is that /testbraket ignores all following commands on the same row as the "text 2" echo, allowing it to continue showing the 3rd and 4th text echo instead of halting.

However, if /testbraket is edited to make the "text 2" echo be on the line below the curly bracket that precedes it, then there's no fail.

---

Thanks for your bug report. In your last example, your use of { } in that context is technically not a valid use of those brackets, which were meant to be used with if/while. These really should report an error and halt the script but were left as-is for backward compatibility reasons long ago.

As for your other examples, you are certainly pushing the scripting language beyond its limits. As I have explained many times before, the scripting language was not designed, it grew organically over 25 years. It is amazing that the parser works as well as it does. It has many quirks, which have been discussed many times before. As much as I would love to see it working perfectly, the odds are that it will need a complete redesign to make it work correctly in the examples that you have provided, where you are mixing a large number of if/while/goto/brackets/separators/splits/etc. on one, long single line.

That said, I have added these examples to my to-do list.

This is not a criticism of those who report bugs, but I just want to give a show of support and a thank you to @Khaled for what he has achieved with mSL. It may have its quirks and bugs, but you can achieve miracles with it. Whilst I think we all whinge at times about the constraints of backwards compatibility that gives preference to decades old, unsupported scripts, I think that we would all moan much, much more if Khaled did a complete redesign and suddenly broke a lot of old, unsupported and unreadable scripts that somehow have become essential.

My wife and I have both used mIRC for literally decades for a very modest one off license fee each and Khaled's support over these decades has been consistently fantastic. I may not have liked some of his decisions about bugs, and I may have a different view on backwards compatibility to him, but it is his software and I absolutely respect his consistent policy and decisions in support of this.

That said, here are a few thoughts on how this might be addressed...

Firstly here are two open-source counter examples to Khaled's backwards-compatibility-first policy.

1. The CMS Joomla has a history of multiple internal re-architectures requiring repeated rewrites of extensions/plugins - and it certainly gets the community riled, and quite a lot of people quit Joomla for Wordpress as a consequence. The Windows IRC app "market" is not dissimilar, and Khaled's approach is definitely better than Joomla's IMO.

2. In order to switch PHP from similarly organic origins to being a more rounded-out object-oriented language, the PHP scripting language has similarly deprecated a lot old syntax over the years, making people update their scripts - and as a webmaster it has been a right pain (which is still ongoing, but deprecations are definitely reducing as this goal is almost realised). But, A) they have always given a lot of advance notice of deprecations, and B) PHP has such a dominant position that IMO they can get away with more. However, the community has weathered the pain this has been accepted because of the overall benefits and the PHP 8.1 language is a pretty fully featured OOL as a consequence.

So how are these relevant to mSL?

A decade or so ago, I did a major remediation of a substantial but ancient & unreadable mSL script into something more maintainable, so I know just how difficult working with such historical scripts can be. The mediation was successful, and I would like to think that my code is much more readable and maintainable as a consequence. And I think that this has given me a good insight into two areas where improvement might be possible.

A. mIRC could (under the covers) have two script engines - the existing "legacy" engine and a new "clean" engine.

B. The "legacy" engine could have deprecation functionality added to help people to identify where scripts would break if they switched to the "clean" engine. This might be i.e. a "deprecation" switch in preferences, which when turned on would create a custom window (or a log file or both) that collected details of where scripts executing scripts had syntax that would break under the "clean" engine. (I am not sure whether this would analyse code at load time or at runtime, but either way it would be a help to switch).

C. Part of the problem with code unreadability and opaqueness is the need to write code using (relatively) low level mSL primitives rather than use advanced pre-written functionality provided by "libraries" or "frameworks" or "packages". dotNET, Python, Java, PHP etc. all have libraries/packages which deliver standardised advanced functionality that build upon the basic language primitives. These help greatly in avoiding developers from reinventing the wheel, and because they (generally) have sensible names they encourage writing more readable code, and because they have many users, they tend (on average) to be better supported. These languages also have a package infrastructure i.e. a repository for libraries/packages and a package manager (e.g. for PHP Packagist/Composer, for Python Pypi/Pip etc.) that help with distributing code to users and help them to update their code, and they also have some coding standards (PEPs for Python, PSRs for PHP) to help developers write readable code which will have longevity, and linters to help identify coding issues. They also have unit testing packages to help with regression testing.

A decade ago I did some thinking around coding standards, packages, package manager, unit testing etc. and even started developing some prototype code. Unfortunately real-life priorities overtook this and I stopped work on these. I still don't have the time to do work on these, but if there is interest from the community I would be happy to be involved in such a team effort.

Just an explanation for how I encountered these, and comments about the deprecation topic.

I'm not intentionally trying to go to extremes to break things. The one-liner of /testcontinue-1 is based on a //command I was doing in the editbox. In the editbox you can use $& to join 2 lines together with the pipe symbol ignored, like ...

//echo -a test $& | message

... but there's no way to 'trick' the editbox into thinking part of your //command is on a 2nd row, and normally there would be no need for it, just as there's no need to use $& there. And my //command eventually ran into an infinite loop that had been caused by the impossibility of a /var command executing twice in a row between its neighbors.

As for the not-needed bracket, that was created from a normal script where I'd used copy/paste to to put the a series of commands inside an if() condition, and at first I thought this issue was because it was inside the if() but it's not, though it did happen here:

alias testbraket2 {
if (1 == 1) {
{ echo -ag text 1 | echo -ag this doesn't halt or show this message | halt }
}
echo -ag text 2 should be halted
}

If this had generated an error, I would've been fine with that and removed the extra curly brackets, as the glitch happens regardless if the end-brace following the 'halt' is on the same line or on the next row. But eventually this ended up being the result of a trouble-shooting to figure out why the script continued to execute even though it had branched to show the echo, and then to determine what's the least edit that would make it behave as expected.

* *

As for Protopia's command about preserving backwards compatibility, a lot of people have different ideas of what should be preserved, and what can be discarded.

https://forums.mirc.com/ubbthreads.php/topics/269731/re-readini-vs-writeini#Post269731

The #B in this link was an example of this, where the cause of $readini and /writeini handling "item name" differently is because of supporting the legacy usage of identifiers like $read etc where their parameters are not enclosed inside the parenthesis.

While I personally have never used the no-parenthesis format, I do rarely see scripts still going around which use them. You can often get an idea of how old a script is by how many obsolete commands it uses. Such as a recent script I was helping with where they were using $ifmatch instead of $v1, $hregex instead of $hfind, or trying to use a .dll to add functionality to a script that was added to the language a couple decades ago.

As a possible solution toward deprecating things, there could be a new section in /help where it lists the deprecated commands/identifiers/syntax, and gives examples of usage that's been taken away and what should be done to fix it. It's easiest to do with identifiers, where with $read it could halt the script with an error that displays a message pointing them to /help for the fix, rather than triggering an identifier warning. The no-parenthesis identifier shouldn't be simply removed, because most scriptors don't immediately understand the difference between $identifier and $identifier(), so using a rare identifier like $sha1 without parenthesis showing an error message saying that $sha1 is not an identifier threw me for a loop the 1st time I saw it, but if a script suddenly showed an error saying that $read did not exist, lots of users, especially non-scriptors, would think it meant that $read() was kaput.

So the deprecated section of /help could give simple examples of old-vs-new syntax, like

old: //echo -a $read -ntw *mir* mirc.ini
new: //echo -a $read(mirc.ini,ntw,*mir*)

For some commands it can be appropriate to do like the deprecated /finger does, behaving the same as noop does

//finger $findfile($mircdir,*.ini,1,echo -a $1-)
//noop $findfile($mircdir,*.ini,1,echo -a $1-)

... but the 'deprecated' section of /help could show like

old: /finger nick
new: /ctcp nick finger

Regarding the advanced notification that Protopia mentioned, one way to do that would be to have a future version include a message during the installation that deprecations were being planned, and to invite users (and especially scriptors) to test the beta version to see how their scripts behave without the deprecated command, which gives them a chance to get things tidy for when the next version-number is officially released.

It may not be sufficient to simply make beta versions available on the forum, because there are a lot of users who never run the beta. Possibly the reason is because the forum post for it paints a dire picture as if the beta is bug infested, when that's rarely the case. The last time I remember a beta version becoming unusable was the time several years ago where a common syntax for $regex was broken and Khaled rushed out a new beta the next day, but typically the difference between $beta and regular is restricted to fixing bugs, and any problems that would be encountered are only using new syntax for existing command/identifiers or new ones.

If there is a beta for deprecated things, it could be something releasd shortly after an official new version, and the only change would be the changes due to deprecation.

--

Or, have a temporary checkbox in options within an official release, that is unchecked by default, where checking the box would deprecate things like $readini without the parenthesis. Then, commands/identifiers planned to be deprecated would respond to that checkbox for 1-2 releases before the deprecation becomes permanent even if the box weren't checked, at which point the language could 'move forward' without being inhibited by that aspect of deprecation. The same options box could continue affecting a sliding series of items if there's more than just a single Mass Extinction Event.

If commands or identifiers do start to be deprecated, then scripts could be helped out by an $iscommand() and $isidentifier() to see if something is part of the internal language, and is not the same as $isalias() seeing scripted aliases. For $isidentifier to tell the difference between $readini and $readini(), it could be $isidentifer(readini) vs $isidentifier(readini,1)

--

As for deprecating the scripting syntax in this thread, that might be difficult to attach to a legacy message, and an options checkbox would require having 2 copies of the scripting engine loaded, so would probably be only in the things-to-be-deprecated beta that doesn't support the things in the scripting syntax that is rarely used, and has a simple alternative which would be just as good if not better. Khaled would know better than we do what kind of support for edge cases there is in the parsing engine which would needlessly slow things down for the 99.99%, and which has a better alternative.

There are probably as many different views about backwards compatibility as there are people expressing them, but in the end someone has to make a decision, and hopefully make them in a consistent, thought through way over time. Whilst many of us may have different views about how we would handle it if we were making these decisions, Khaled is the developer and the choices are his to make and his to own the responsibility for the consequences - and I fully support him in this regardless of whether I would make the same decisions myself.

However it appears that I need to apologise for not having explained myself clearly.

1. The "legacy" script engine would maintain backwards compatibility in the exact way that Khaled has maintained it for all these decades - when using the "legacy" engine, $read would not "halt the script with an error that displays a message pointing them to /help for the fix" as tat would break the script and be backward incompatible. Instead it would log a message to the custom window/log file detailing the issue and what would need to be changed to fix it so that the code would work in the "clean" engine. It might even actually be possible to write a script that when run by the user goes through this log and makes the relevant changes to the code (though that would probably be down to the community to create once the "legacy" engine had the deprecation-identification functionality).

2. This is completely different to having beta versions before delivering a full version. IMO it would be several years (if not decades) before you could remove the "legacy" script engine - but clearly any new functionality would have no need for any backwards-compatibility, and so the "legacy" engine would effectively be stabilised code, with all new functionality being delivered by the "clean" engine - which I assume would be easier to deliver because that part of the code would be much cleaner.

Of course, this is only theory - only Khaled can say whether the existing code base is amenable to supporting both "legacy" and "clean" script engines.

Thanks for your comments. There are no plans to make different versions of the scripting language :-)

As for the { } brackets, I actually found a unit test case that checks for these in a slightly different format to your example, so it is clearly something that I've looked into before. I have made a change that handles your example as well. This change will be in the next beta.

Okay. I have made changes to the script parser that fix the issues in your bug report. This was about as arduous as I expected. There are layers of changes and fixes and tweaks here, implemented over decades. I have a script that runs around 100 unit test cases for if/while/brackets/etc. and it took a lot of tweaking to make them all pass with the fixes. This change will be in the next beta. Let's see how it works out - the main issue will, of course, be maintaining backward compatibility.

Thanks, I know this kind of thing can boomerang into other problems, as evidenced by the fact that these 3 examples seem to have been triggered by 2 different parser fixes listed in versions.txt, and the final examples didn't involve a while() loop yet the description of the fix showed the change was because of while() loops.

Hopefully there will be more people willing to test the beta so anything that might be caused by this would be noticed before any official release.

Hey, I found a case that doesn't work:

//var %j 2 | while (%j) { var -s %c 3 | while (%c) dec -s %c | dec -s %j }

This is an infinite loop:

on the second iteration of %j (when it reaches 1), when the loop on (%c) runs, it executes the /var -s %c 3 as well instead of once before the loop runs, which it shouldn't.

I hope this bracket-less nested while loop won't be as difficult to fix as the previous ones.

I found something related to fixing testbraket, as it affects v7.72 but not v7.71

As a side-effect of this, I find that my $window font jumps into being some proportional font instead of Consolas, except returns to normal for the duration of highlighting text with the mouse.

It took me a while to figure out the cause since the syntax error happened at the top of the script and the { } orphan block was way down at the bottom where it could not possibly ever be executed. And sometimes the line number being reported wasn't actually an error, but was a line holding something like "inc %variable"

But Wims pointed out it's happening due to trying to look for an :error label, and happens when the orphan block occurs prior to finding the label, regardless whether the label is there or not.

There's no long delay, and the mem alloc error doesn't happen if I put above the orphan { } block...

:error | echo -a $error | halt

... but does still happen if that's placed below the orphan block because the :error label can't be found, so it keeps sucking up memory looking for the :error which hopefully it never finds. Calling it this next way does confirm that the error is not caused by the displaying of the error message, but happens prior.

//var %start $ticksqpc | error_allocating_memory_if_error_message | halt | :error | echo -a pause after $calc($ticksqpc - %start) ms : $error

pause after 43907 ms : * Error allocating command memory (line NNN, aliases.ini)

Found another case where the parsing fails. Even though this is not a nested while() and fails in method2 where the while() open/close braces are on 2 lines, it may be related to the changes that affected the above testcontinue-2, as all 5 methods work up through v7.38, but beginning v7.41 thru the current beta, method 1 and 2 fail while the other 3 keep working.

Yes, it will be and is and was :-]

I have added a fix for this but... it could easily have side-effects.

The recommended method of formatting if/while statements is to use () and {} brackets. These are really important due to the lack of quoting/escaping in the scripting language. To make matters more complicated, all kinds of tweaks have built up over decades to overcome ambiguities and limitations in different contexts. So fixing your above issue could very well affect existing code in other contexts.

Ideally, the scripting language would reports errors when () or {} are not used but this would break all existing scripts at this point, so cannot be changed.

If there are any side-effects, I will have to revert this change. Let's see how it works out in the next beta.

Thanks fixed in the next beta.

This is similar to the above issue, ie. not using () and {} with if/while statements.

The behaviour was actually due to a very old tweak that handled ambiguous contexts.

I have implemented a fix and added your examples to my unit tests. But, as with the above fix, if it has side-effects, it will be reverted.

This change will be in the next beta.

I've done some searches under "Comment" "Bracket" Brace" and "{" and "}" and not found this being discussed anywhere.

I've just spent many hours trying to debug a script only to discover that the a comment at the end of the line that starts with a ; character after the code does not stop the parser from reading and processing a } character and therefore unintentionally closing a loop/event. Running v7.76

e.g.
Will output Command1 and Command2.

The "AlwaysCommand" will never execute as mIRC will treat the last } after "TestCommand3" on the IF line as the closing bracket for the ON TEXT event. You can see this instantly in the way the script window will indent the code.

Remove that supposedly commented text " | TestCommand3 }", or even just the "}" and everything will ident and execute as expected.

Having read this post and the replies I don't expect this is critical enough to warrant a serious "fix", it sounds like it might be very difficult to fix.

I'm capturing here so some future fool like me can find an answer before banging their head off the desk for several hours.

Khaled: I am constantly in awe of the power and scope of the script engine "as-is". Thanks for everything.

You have multiple issues in your example.

1. The "IF" cannot touch the "(".
2. The "==" cannot touch the numbers.
3. The "{" cannot touch the "Command1".

Fixed example:

Granted....all typos I'll fix. None of which have anything to do with or impact the issue being identified.

Talking about the fixed code:

You cannot comment code reliably with the ';' if it's not starting the line. You can check this thread for more informations about how ; can in fact be used to comment only a single statement out of multiple statement separated by a |, but is has a quirk and you cannot comment out the last statement if it ends with a } or the } is parsed incorrectly, but if no } are involved on the line, you can comment a single statement, even the last :
alias test {
echo -ag ok | echo -ag ok1 | ;echo -ag ok2
}
Will only display ok and ok1, it's also worth noting that the else statement does not have this problem and you can always comment out the last statement of a single line else statement with multiple statements in it as is the case in the above link.


You also still have one extra } in this code, making 'alwayscommand' outside on the on text event because spaced out } cannot be commented out if ; isn't starting the line.

Focusing only on the line "IF (1 == 1) { Command1 | Command2 } ; | TestCommand3", what's happening there is that } in itself acts as a statement separator:
//if (1 == 1) { echo -ag ok | echo -ag ok1 } echo -ag ok2
is equivalent to
//if (1 == 1) { echo -ag ok | echo -ag ok1 | echo -ag ok2 }
If you change the condition to 1 == 2 for example, 'ok2' is not echoed
It's known that } act as a statement separator, same as |, and you don't need prior { } involvement to see this:

//echo -ag ok } echo -ag ok1
is equivalent to
//echo -ag ok | echo -ag ok1

So then ';' is seen as starting a statement but you see, the quirk with ; and commenting out single statement is that it correctly works stopping at a | but not a }, so this is the same as
IF (1 == 1) { Command1 | Command2 | ; | TestCommand3 }
Where you just have an empty commented out statement, and /testcommand3 is executed 'normally'

Although the problem is as simple as "; starting a statement which is not starting a line but is the last statement in a line doesn't stop at } correctly", given Khaled's answer it's likely that fixing it would require more work than just adding that logic once somewhere. But of course I would like to be able to comment out single statement like that so I wouldn't mind a fix.

I wouldn't recommend anyone trying to comment out single statement out of multiple statements separated by | on a single line or trying to commentate anything with ; if it's not starting the line anyway because of all of these, but if you're aware of them, you can actually workaround them. For example you can solve the issue reported in the mentioned url by adding a fake last empty statement, just by adding a spaced out pipe character, which works for your case as well, coming from this code:
and trying to get testcommand3 not to be executed without changing much in the code, your best bet is

But trying to use ; when it's not starting a line and when it's not starting a statement properly either (properly aka after the } in your original example isn't proper) is just not possible.

Found an issue with /break:

1) in single line, /break isn't stopping a loop when it should, /return works.
2) but in multiline, /break works.

alias show1 {
var %a 5
while (%a) { dec -s %a | break }
;decrease down to 0 instead of stop immediately after decreasing once to 4
var %a 5
while (%a) { dec -s %a | return }
echo -sg here
;echo -sg here isn't executed and %a was decreased to 4, so /return works.
}

alias show2 {
var %a 5
while (%a) {
dec -s %a
break
}
;also works:
;var -s %a 5
;while (%a) {
; dec -s %a | break
;}
}


Thanks this issue has been fixed in the latest beta.