I dont think so, he mentiones not talking about spaces, looking at it again i now think he screwed up the example, i think he ment to use this
//tokenize 44 1,2,3 | echo -a $1 - $2 - $3 == 1 - 2 - 3
//tokenize 44 1,,3 | echo -a $1 - $2 - $3 == 1 - 3 -
I think hes saying the $null field between the 2 ,, should be seen as $2, and i can see where hes coming from with that (
not that i aggree).
I used 46 over 44 becuase there easier to insert into a $+()
//var %a = 1, %b = 1, %c = 3 | tokenize 46 $+(%a,.,%b,.,%c) | echo $!1= $1 $!2= $2 $!3= $3
$1= 1 $2= 2 $3= 3
.
//var %a = 1, %b = $null, %c = 3 | tokenize 46 $+(%a,.,%b,.,%c) | echo $!1= $1 $!2= $2 $!3= $3
$1= 1 $2= 3 $3=
He wants to maintain the location of his values but insert them into $N values, with the posablility that any $N value maybe a $null string.
As im sure you know you can call an alias as an identifier and manage the effect of creating $null $N values.
alias example ( echo $!1= $1 $!2= $2 $!3= $3 )
//var %a = 1, %b = $null, %c = 3 | .echo -q $example(%a,%b,%c)
$1= 1 $2= $3= 3
I personally wouldnt like it, i have depended on the exact opersite effect , tokenize 32 %a %b %c, to get $1=1 and $2=3