I've never heard anyone say that tokenize shouldn't be used. Is there some reason you say it's not recommended? Tokenize works very well for splitting words in locations that are not already split. Maybe you're referring to the $* as I've seen some say that isn't recommended because it's not documented. But that isn't needed when using tokenize. It's just a simple way to display each token on another line once you have the tokens.
To the OP, if you're monitoring what is typed by someone, all you really need to do is use $1, $2, etc. in your ON TEXT event as mentioned in the middle of asdfasdf's post.
Example:
on *:text:*:#:{
echo -a Word 1: $1 ~ Word 2: $2 ~ Word 3: $3 ~ All words: $1- ~ All words except the first: $2- ~ Words 2-3: $2-3
}
As you can see, all you need to do is use $ and the word number to get that word. Use - at the end to get that word through the end of the sentence. And to get a range, add the final word number after the -. There are a variety of things you can do, but this can get you started. Use of token identifiers like $gettok() are very useful when you want the "words" to be separated by something other than a space, such as splitting a list of words/phrases at every comma. Tokenize also works for splitting a sentence by other characters. Unlike using $gettok(), the use of tokenize gets you back to using $1, $2, etc. instead of having to use $gettok(sentence, "word" #, chr to split on) every time.
