mIRC Home    About    Download    Register    News    Help

Print Thread
literal inc or dec in an $identifier() breaks vars #269302 22/08/21 11:12 PM
Joined: Dec 2002
Posts: 224
T
Talon Offline OP
Fjord artisan
OP Offline
Fjord artisan
T
Joined: Dec 2002
Posts: 224
I was working on an identifier for Raccoon to return the ISO 8601 standard week-of-the-year from a specific date. I needed to take advantage of $ctime() to parse a date string to unix timestamp. I discovered trying to construct my date string, some portions weren't evaluating thus returning a blank timestamp.

I was doing something along the lines of $ctime(%month %day %year %time)

With the help of maroon, we discovered it was because of the shorthand month december "Dec" was causing the issue.

maroon's rigorous tests have found that multiple strings cause this issue that are also mIRC commands. So far, inc, dec and set cause the same exact issue. It seems to only affect variables that follow directly after that text if the first parameter inside the ()'s will evaluate to one of these three words, or is literally one of these three words.

Below is an alias to show what's happening for the following examples ran on mIRC 7.66 on Windows 10.
Code
testme { return $qt($parms) }


//var %x = 1 | echo -a $testme(Dec %x)
"Dec %x"
//var %x = 1 , %y = Dec | echo -a $testme(%y %x)
"Dec %x"
//var %x = 1 , %y = Dec | echo -a $testme(%y %x %y %x)
"Dec %x Dec 1"

The unusual part, this doesn't happen with evaluation brackets:
//var %x = 1 | echo -a $testme( [ Dec %x ] )
"Dec 1"
//var %x = 1 , %y = Dec | echo -a $testme( [ %y %x ] )
"Dec 1"

Re: literal inc or dec in an $identifier() breaks vars [Re: Talon] #269303 23/08/21 09:37 AM
Joined: Dec 2002
Posts: 4,999
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 4,999
Thanks for your bug report. This is the intended behaviour. This is related to the inc/dec/set handling that has been discussed several times in the past. The last time a change was attempted relating to this, it broke existing scripts, so it was reverted. At this time, there is no way around it that does not break existing scripts.

Re: literal inc or dec in an $identifier() breaks vars [Re: Khaled] #269305 23/08/21 12:35 PM
Joined: Jan 2004
Posts: 1,617
maroon Offline
Hoopy frood
Offline
Hoopy frood
Joined: Jan 2004
Posts: 1,617
//var -s %day 31 | echo -a $ctime(Oct %day 2021 00:00) vs $ctime(Dec %day 2021 00:00)

* Set %day to 31
1635656400 vs

Yes, it's related to the issue reported here:
https://forums.mirc.com/ubbthreads.php/topics/266592/timer-and-variable-assignment

So basically, the unexpected behavior happens for strings passed to an identifier which happen to begin with a word which matches some of the commands which don't evaluate %word in some situations, such as when the 1st parm is either INC or DEC or SET or UNSET or SOCKREAD, But not VAR. And the link shows it's also happening when a timer commandline is evaluated, and in that case even when the "inc %word" isn't at the beginning of the string.

The evaluation routine doesn't know whether it's handling a string that's actually a command or if it's handling a string that's just a string being passed to an identifier like $Identifier(string)?

//var %a 123 | echo -ag $upper(set -s %a 2 + 2)
//var %a 123 | echo -ag $upper(var -s %a 2 + 2)
//var %a 123 | echo -ag foo $upper(sockread 1234 %a ) bar
//var %a 123 | echo -ag foo $eval(inc -s %a bar,$rand(99,999) )
//var %a 123 | echo -ag $upper(%a = 1 )

returns:
SET -S %A 2 + 2
VAR -S 123 2 + 2
foo SOCKREAD 1234 %A bar
foo inc -s %a bar
SET %A 1

Re: literal inc or dec in an $identifier() breaks vars [Re: Khaled] #269306 23/08/21 12:49 PM
Joined: Apr 2004
Posts: 862
Sat Offline
Hoopy frood
Offline
Hoopy frood
Joined: Apr 2004
Posts: 862
For completeness: /unset is also affected. However, /var is not.

This is the first time I hear about this issue, and I'm honestly a bit shocked. I would surmise that lots of scripts out there are already broken, in the sense of not behaving as intended under all realistic legitimate circumstances. That is in particular due to the fact that it is possible to use any of those four-or-so special keywords as nicknames. That means that while this issue will not come up often by accident, any instances of it may be triggered intentionally by people with malicious intent. As such, I for one would argue that it is better to break a few (I'd guess old and badly written, but anyone please prove me wrong) scripts clearly and overtly, than to have many other scripts that misbehave under very specific but realistically unpreventable conditions. Therefore, I'd say this is definitely worth another look/try! In that sense, I think that the fact that /var is not affected, is a pretty good sign..


Saturn, QuakeNet staff
Re: literal inc or dec in an $identifier() breaks vars [Re: Sat] #269307 23/08/21 06:13 PM
Joined: Dec 2002
Posts: 4,999
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 4,999
Quote
For completeness: /unset is also affected. However, /var is not.

This was indeed discussed last time.

As I mentioned before: /var was implemented, around 1999, in a completely different way to /set, /dec, /inc, etc. which cannot, in any way, be implemented in the same way as /var, at least, not without breaking the way the parser works for all existing scripts.

The current behaviour of /set, /dec, /inc, etc. has been in place for about 20 years. They were implemented in this way because users requested this specific behaviour. There is actually no way to change this particular behaviour without breaking existing scripts. In my previous attempt, I changed them in a way that I thought would only break some scripts but a beta tester immediately reported an issue, so the change was reverted. No matter how they are changed, they will definitely break scripts. The question is, is it worth breaking all existing scripts in a subtle, unpredictable way in order to fix this specific issue?

In any case, as I said, this has already been discussed and is on my to-do list. Thanks for your comments.

Re: literal inc or dec in an $identifier() breaks vars [Re: Sat] #269310 27/08/21 11:58 AM
Joined: Dec 2002
Posts: 4,999
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 4,999
I have taken another look at the code I wrote previously to try to resolve this isue (see the post linked to in maroon's reply above) and I am going to enable it again in the next beta.

However, I can only test the code so far - it is impossible for me to know every possible way that scripters can make their scripts invoke this issue.

Last time, only TECO provided feedback when the beta was released.

Since we have a few more eyes on this issue this time (Talon, maroon, Sat), hopefully we'll be able to find a working solution that maintains as much backward compatibility as possible.

This change will be in the next beta.

Re: literal inc or dec in an $identifier() breaks vars [Re: Khaled] #269311 27/08/21 12:33 PM
Joined: Jul 2006
Posts: 3,807
W
Wims Offline
Hoopy frood
Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 3,807
Glad to see you are looking into this.

!what's the reason the scripting engine is doing this check on variable command etc, inside identifier's parameters? I don't understand how it could be the case, especially with the fact that it can be anywhere, not necessarily at the beginning of the parameter.
I understand that those commands must not evaluate the variable name passed to the command while evaluating the rest of the parameter but I still can't see how that would result in the current behavior being reported here or why it would be tricky to fix this issue, in other words, would it break the scripting engine to remove this check on identifier's parameter? it seems it wouldn't, from a naive/external point of view.


Looking for a good help channel about mIRC? Check #mircscripting @ irc.swiftirc.net
Re: literal inc or dec in an $identifier() breaks vars [Re: Khaled] #269313 27/08/21 02:34 PM
Joined: Dec 2002
Posts: 224
T
Talon Offline OP
Fjord artisan
OP Offline
Fjord artisan
T
Joined: Dec 2002
Posts: 224
I don't know if it would help at all, but it appears how you handle []'s doesn't have this flaw. If I understand correctly, brackets are supposed to force parsing in that nested region first, then jump back to the beginning and then parse the entire string. If the same parsing routine is invoked for it all then [ ]'s should suffer the same fate, yet they don't.

Eval doesn't behave the same as [ ]'s

//var %a = dec , %b = 1 | echo -a $eval(%a %b,99)
dec %b
//var %a = dec , %b = 1 | echo -a $+(%a %b)
dec %b

You would assume you'd need to eval the 2nd parameter 1st by using []'s to avoid the issue, which works...
//var %a = dec , %b = 1 | echo -a $+(%a [ %b ] )
dec 1

But so does just nesting the entire thing...
//var %a = dec , %b = 1 | echo -a $+( [ %a %b ] )
dec 1

This last one I would think should suffer the same flaw as it's shorthand $eval(%a %b,2) yet it works perfectly.

Re: literal inc or dec in an $identifier() breaks vars [Re: Talon] #269315 28/08/21 11:20 AM
Joined: Jan 2012
Posts: 171
Epic Offline
Vogon poet
Offline
Vogon poet
Joined: Jan 2012
Posts: 171
What if we try to solve it this problem way, separating 2 variables with a code for space $chr(32) or using $() - enclosing the variable between parentheses:

Code
//var %x = 1 , %y = Dec , %z = $chr(32) | echo -a $qt($+(%y,%z,%x))
//var %z = Dec , %x = 2 , %y = %z %x | echo -a $eval(%y,99)
//var %z = Dec , %x = 3 , %y = $+(%z,$chr(32),%x) | tokenize 32 %y | echo -a $1 $2
//var %a 123 | echo -ag $upper(set -s $(%a) 2 + 2)
//var %a 123 | echo -ag foo $upper(sockread 1234 $(%a)) bar
//var %day 31 | echo -a day = %day -> $+($chr(40),$ctime(Dec $(%day) 2021 00:00),$chr(41))

Return1: "Dec 1"
Return2: Dec 2
Return3: Dec 3
Return4: SET -S 123 2 + 2
Return5: foo SOCKREAD 1234 123 bar
Return6: day = 31 -> (1640898000)


πŸ…ΈπŸ†πŸ…² - 𝔦'𝔱𝔰 𝔸 π•‚π–ŽΙ΄π™™ ᴏ𝙛 π•„π™–π–Œπ™žπ™˜
Re: literal inc or dec in an $identifier() breaks vars [Re: Talon] #269316 28/08/21 11:32 AM
Joined: Dec 2002
Posts: 4,999
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 4,999
Quote
I don't know if it would help at all, but it appears how you handle []'s doesn't have this flaw. If I understand correctly, brackets are supposed to force parsing in that nested region first, then jump back to the beginning and then parse the entire string. If the same parsing routine is invoked for it all then [ ]'s should suffer the same fate, yet they don't.

This is the same observation that Sat made for /var. My reply is the same as that for /var. [] brackets work in a completely different way, at a completely different point in the parser.

I keep saying this: the scripting language grew organically. Every feature in it was patched on, over a period of 25 years, with no over-arching grand plan or design.

I cannot make feature X work like feature Y, not without rewriting the parser, shfiting code around to different points in the parser, breaking backward compatibility, etc.

It works the way it does because that is how it was implemented X years ago.

Re: literal inc or dec in an $identifier() breaks vars [Re: Khaled] #269317 28/08/21 12:38 PM
Joined: Apr 2004
Posts: 862
Sat Offline
Hoopy frood
Offline
Hoopy frood
Joined: Apr 2004
Posts: 862
Originally Posted by Khaled
I have taken another look at the code I wrote previously to try to resolve this isue (see the post linked to in maroon's reply above) and I am going to enable it again in the next beta.

Just to be sure: I might be reading it wrong, but it appears to me that the previous thread (and the potential fix) was addressing the behavior of strictly and only the /timer command, where the problem was with the occurrence of any of the five (thanks maroon!) reserved keywords anywhere in the command to be evaluated and executed. The current thread covers a much wider scope for evaluation in general, where the problem occurs only if the first word is any of those five keywords. Thus, these sound like two somewhat different (if overlapping) issues to me, with different potential fixes. Could you please elaborate on the fix that you're planning now?

I'd be happy to help with testing to the degree that I can; the more specific you can be about the intended effects of the changes, the better we will be able to find any issues..


Saturn, QuakeNet staff
Re: literal inc or dec in an $identifier() breaks vars [Re: Sat] #269366 14/09/21 03:08 PM
Joined: Dec 2002
Posts: 4,999
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 4,999
Quote
Just to be sure: I might be reading it wrong, but it appears to me that the previous thread (and the potential fix) was addressing the behavior of strictly and only the /timer command

The issues are different but they are actually present in the same small, shared function. I am going to make a change to address the issue in this thread in the next beta.

Re: literal inc or dec in an $identifier() breaks vars [Re: Talon] #269431 30/09/21 08:55 PM
Joined: Dec 2002
Posts: 4,999
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 4,999
This change is now in the latest beta.

If you could please test it, compare its results to v7.66, and confirm that it is working as expected, that would be appreciated.

This change affects how %vars are evaluated if a line is considered a command and begins with a /set/unset/inc/dec/timer/sockread command.

Re: literal inc or dec in an $identifier() breaks vars [Re: Khaled] #269433 30/09/21 09:18 PM
Joined: Jul 2014
Posts: 238
TECO Online
Fjord artisan
Online
Fjord artisan
Joined: Jul 2014
Posts: 238
Hi Khaled

Even with all the changes made in the last beta version, everything is working with my scripts wink


TECO
irc.PTirc.org (Co-Admin)
Re: literal inc or dec in an $identifier() breaks vars [Re: Khaled] #269434 30/09/21 10:49 PM
Joined: Dec 2002
Posts: 224
T
Talon Offline OP
Fjord artisan
OP Offline
Fjord artisan
T
Joined: Dec 2002
Posts: 224
So far so good, my initial attempts to break the change has failed to produce a problem, and I also haven't noticed any breakage in any previous scripts I have loaded due to the changes. I haven't had a chance to see what happens with sockets as per TECO's issue (all my previous socket scripts sockread to &binvar), nor timers from Wims initial thread besides his test example.

Re: literal inc or dec in an $identifier() breaks vars [Re: Khaled] #269435 01/10/21 03:58 PM
Joined: Jan 2012
Posts: 171
Epic Offline
Vogon poet
Offline
Vogon poet
Joined: Jan 2012
Posts: 171
For the test, I used "mIRC v7.66 (Beta 272)" - all my scripts that use commands "/set/unset/inc/dec/timer/sockread" work good, no crashing and errors. All the problems described above have been fixed.


πŸ…ΈπŸ†πŸ…² - 𝔦'𝔱𝔰 𝔸 π•‚π–ŽΙ΄π™™ ᴏ𝙛 π•„π™–π–Œπ™žπ™˜
Re: literal inc or dec in an $identifier() breaks vars [Re: Talon] #269436 01/10/21 08:15 PM
Joined: Dec 2002
Posts: 4,999
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 4,999
Great, thanks for testing.

I plan to release a new version soon due to the issue with the Let's Encrypt certificates, so will keep this change in the release.