/tokenize [prefix] <c> <text>
Let me explain that a bit with some examples then the reasoning why:

/tokenize N 32 1 2 3 4 5
$N1 = 1
$N2 = 2
$N3 = 3
$N4 = 4
$N5 = 5

The idea is, consider this
ON *:TEXT:*:*:{
tokenize 45 $1
echo -a $nick typed $1 $+ . $+ $2
}
Once that tokenize is performed, everything else sent as text ($2-) just disappeared. To continue using original params you'd have to do something like:
ON *:TEXT:*:*:{
var %text = $1-
tokenize 45 $1
echo -a $nick typed $1 $+ . $+ $2
tokenize 32 %text
echo -a The third word of that line was $3
}
With my idea:

ON *:TEXT:*:*:{
tokenize N 45 $1
echo -a $nick typed $N1 $+ . $+ $N2
echo -a The third word of that line was $3
}

Basically it just prevents it from overwriting what is currently in $1..N and also allows you to easily use many tokenize calls in the same event/alias/etc.

what if u have such alias? alias n1 ?
anyway, u can store it before u /tokenize like:
on *:text:*:#:{
var %x = $1-
tokenize 45 $1
}
i know that u know it but it will be a problem with your suggestion if u have such alias

That completly void the point of tokenize. The idea he was saying what he shouldn't have to gettok one string, and have the luxury of using the $1, $2, $..., $N aliases. Its pointless to have to do $gettok(%x, 1, 32) just to have the please ur tokenizing the next string. You may have just kept the "to-be-0tokenized" string the same and use gettok.

didn't you write a l-o-v-e-r-l-y array addon that'd do all this nicely?
i don't believe you wouldn't have written an arr_explode 45 $1- command...

still i agree with your suggestion

Well you are defining the prefix, so you set it to something you don't use. If you have $N1 as an alias, then you specify M as the prefix, then it is $M1, and I never specifically said it has to be one char.

/tokenize this-is-my-tokenize-prefix 45 $1

$this-is-my-tokenize-prefix1
$this-is-my-tokenize-prefix2
...

So conflicts shouldn't be a problem since you can define the prefix. Plus there is the issue of scope. Arguing this would be the same as saying /var shouldn't have been added because what if I /set %somevar, then /var %somevar, now I can't use the global one. Well thats because of scope. In the same respect, if I have:

alias N1 { return test }

alias blah {
tokenize N 45 $1-
echo -a $N1
}

That would return the first parameter as the result of tokenize.
If I have :
alias blah2 {
echo -a $N1
}
That prints "test". Since $N1 is not defined in the scope of blah2 as being a result of tokenize.

An arrayexplode type of thing is already in my next release (I already have the reverse $arraytok), but I'd still like to see it as a built in feature of /tokenize. IMHO it would make life easier. If anyone here is a C programmer who has ever used strtok you know the exact kind of thing I'm talking about. Using strtok on two sets of tokens at the same time doesn't return fun results.

lol--i hate strtok in c, thus i made an almost duplicate library in c to make available the mirc $gettok, $deltok,$reptok, etc functions. >:D If anyone is interested in a copy, let me know. Realise its not gone through extensive tests, nor is it all the $*tok* identfyers in mirc. I also release it as-is, any improvements must be made by the owner >:D

This would be difficult/impossible because /tokenize uses existing identifiers $1 - $99. To create new identifier names on the fly, I'm not sure what kind of new variable scope Khaled would have to invent. Your suggestion is much like asking to /var variables that begin with a dollar sign $.

Personally, I'm satisfied with $gettok. Through speed tests I've conducted, referencing a %var is much faster than referencing $1. Not positive why, but %variables evaluate faster. My guess is that $1- identifiers are simply $gettoks behind the scenes and each evaluation is an entirely new string comparison. Whereas a %var is a defined value that needs no parsing to obtain. It would be much faster to perform your $gettoks and save them to individual %var's.

Code:

/mytokenize {
  var %name = $1
  var %delimit = $2
  var %s = $3-
  if (!%s) || ( %delimit !isnum ) return
  var %i = 1
  WHILE ( $gettok(%s,%i,%delimit) ) {
    set -u0 % $+ [ %name ] $+ [ %i ] $ifmatch
    inc %i
} }

//mytokenize test 32 a b c d | echo -a %test1 - %test2 - %test3 - %test4
a - b - c - d

- Raccoon

Well you would still be free to use $gettok if you don't like the interface.

Additionally saying it is "difficult/impossible" is not really a call you can accurately make. Perhaps this is a feature Khaled had already started, and completed 80% and then thought maybe no one would want it so decided to halt his efforts. You have not seen the mIRC source code, therefore your just making a guess at how mIRC's internals work. Perhaps adding new levels of scope involves:
ScriptParser.Aliases(thealiasname).AddScope("$N*", LOCAL_TO_ALIAS);
You are making assumptions to the fact that it would be difficult. As I just showed I can just as easily make an assumption that adding it is a single line of code. But rather than make assumptions about something neither you nor I can accurately conclude, I would much rather leave it up to Khaled about whether it is difficult/feasible/etc, seeing as he is the only one who can accurately make that call.

As for your speed issues, I used to think speed was 100% everything when it came to programming. Then I got introduced to the concept of OOP. Basically the design ideas of OOP is "ease first, speed second." If something is easier to do (and using /tokenize would be easier than $gettok), and doing it the easier way doesn't cause a dramatic speed loss then do it the easier way, why make more work for yourself to save 2ns? After all, you could just as easily argue that /tokenize should not have been added because it's function can already be emulated with $gettok, but apparently Khaled disagreed.

Firstly, when you've been using mIRC and scripting and emailing Khaled and using these forums and being a general #mIRC nuisance for as long as I have, you get an idea of how mIRC works internally, especially the parser's general order of operations. This is based simply on suggestions that Khaled says is not possible, and he explains why. After a while, these reasons accumulate to form a pretty good understanding of mIRC and one can being to determine what is probably difficult or impossible.

We know how identifiers and variables are stored and evaluated. We know the scopes that these different items can exist in, and the priority in which items of similar names get chosen first. Yes, it is probably possible to do by emulating the creation of many Local Aliases (Alias -l) so that $N1- /returns a processed $gettok value on demand. There are other ways that, yes, I'm not aware of... but the idea seems inconsistent with how mIRC works in general. %variables are best suited for returning stored data, $identifiers for processing a function and returning results. Anything else means adding a new layer of special handling to the parser.

About "OOP", this sounds more like a joke you caught the butt end of. "OOPs!" Good programmers consider speed and Nice use of resources first, and ease second. Unless you work for Microsoft and you have some impossible deadline to meet and writing secure clean code doesn't mean much to you. If ease means not declaring your variables or making all variables global, speed means minimizing the use of variant/global variables. This alone saves you as much as 5%, not "2ns". If you have trouble reading "speed enhanced" code, then you shouldn't be programming.

- Raccoon

I've been using mIRC for abut 5 years, I've been using these forums since they day they opened, I used to be a helper in DALnet's #helpdesk, and I've been programming for the last 8 years. If the qualifications you mentioned are enough to "prove" your argument, then why wouldn't the qualifications I mentioned be enough to "prove" my argument? I'll tell you why, because yours nor my experience/qualifications tells us how mIRC works; your making assumptions.


If you are so sure it is impossible with the current system, how does mIRC do it with $1, $2, etc? If it is impossible to add $N1, $N2, etc why is it possible to add $1, $2, etc? Your argument doesn't make much sense. As I said in my other post, it's all supposition. Instead of making suppositions of things I can't know, I'd again rather let Khaled decide.


Would you mind telling that to everyone of my computer programming professors and the computer programming professors of everyone else I know? Because they would seem to disagree with you. Oh and for reference OOP is Object Oriented Programming, which is known by everyone to be slower than simple functional composition coding.


Well then I suggest you quickly start a petition for the abolishment of C, C++, Java, C#, and every other high-level language, because they don't even come close in comparison to the speed you can attain with good assembler skills. But guess what? Due to the fact that those high-level languages make porting EASIER, make programming EASIER, they are used 1000x more frequently than ASM. So that alone pretty much disproves your argument. Speed is NOT the primary issue when it comes to coding. Programming has become a lazy man's field, CPUs are getting faster and faster, and simple code no longer has to be optimized as much to get the same effect. Again I'm not saying don't try and make your code fast, but if you have to write the code in such a way that it becomes confusing, uses "tricks," etc, then you're not going to do too well in the field of programming.

If it is impossible to add $N1, $N2, etc why is it possible to add $1, $2, etc?
mIRC is capable of doing $1, $2, etc because they are hardcoded into the language. What you are asking is a variable identifier named on demand.

Oh and for reference OOP is Object Oriented Programming, which is known by everyone to be slower than simple functional composition coding.
Yes, I know what OOP is, and yes I can deduce that you program in Visual Basic. Visual Basic is inharently slow, which is especially why VB coders try using as many speed enhancing "tricks" as possible."

[rant about killing all programming languages in favor of machine code that is inherently faster, and how programmers who use "tricks" to make their programs faster are failures]
I have no gripe against a programming language layer because it's easier to code in than Assembly. But I'll have you know that only serious coders embed ASM routines ("tricks") into their programs. They seem to be doing pretty well for themselves. (Incidentally, I have a friend who even wrote a VB compiler addon which allows you to add an ASM routine in the form of a Sub... if this interests you). If you code in VB, you really ought familiarize yourself with all the tricks out there. A little MoveMemory or other API can make your program run 100x faster than with native functions... and with little effort too.

How do You know $1 $2 $3, etc is hardcoded? YOU DON'T. You are guessing.

I'm a VB programmer? I HATE VB. I won't even go near it! I do C++ and C where the FIRST thing they teach you is speed is NOT everything.

Serious coders do NOT embed ASM into their code. What are you talking about??? Seriously, you sound like you are making it up as you go along. Why would I embed ASM code? So that my program can only run on an x86 processor? what if someone is using an alpha-dec? an ia64? a sparc? a ppc? and any other processor out there? My "asm tricks" just made it so they can't use my program. And guess what, my saying "use ASM" was a trick to see if you really knew what you were talking about, your answer proved that you did not. A good C compiler can optimize your C code to something that is 10x better than anything 99% of programmers could ever write in ASM. NO ONE recommends using ASM, the compiler has the collective knowledge of hundreds of skilled programmers, the rule thumb follows from literature, "William Faulkner could write a sentance that was one page long and still manage to keep it interesting. You are not William Faulkner, don't do it." Same holds true, 99% of programmers are NOT better than the knowledge of ALL of the programmers who wrote the compiler combined. Their knowledge of ASM is going to be far greater than yours and therefore you shouldn't even attempt to try and surpass theirs. ASM is only still used on the very few systems which C has not been ported to (z80 comes to mind), on systems that support high level languages, NO ONE uses asm.


Seeing as how your "deduction" that I'm a VB programmer couldn't be further from the truth, why should I believe that your _assumption_ that you know how mIRC's internals work is the truth? From everything you've said you seem to not have the slightest clue what you're talking about, and I hope Khaled notices that when viewing this thread so he doesn't immediately dismiss this suggestion due to your "deductions." In any case I'm done posting on this thread since I'm sure all that will happen as a result of this post is you will go and make more "deductions" and they won't even be worth replying to.