Thanks to Raccoon for these aliases. Obviously 32 is the codepoint for the space delimiter.

If you're not concerned with the order in which the tokens appear, you can just do:

//var %a ab cd ef gh ij ab ab ef | echo -a $dedupetok(%a,32)

which returns: cd gh ij ab ef

... which is the order after deleting the leftmost dupes. If you want to make it get rid of the rightmost dupes, you have to flip the word order, then kill the leftmost dupes, then flip it again:

//var %a ab cd ef gh ij ab ab ef | echo -a $revtok($dedupetok($revtok(%a,32),32),32)

returns: ab cd ef gh ij

alias revtok { var %a = $1, %b = $2 | return $regsubex(_revtok,%a,/[^ $+ $chr(%b) $+ ]+/g,$gettok(%a,-\n,%b)) } ; by Raccoon 2017
alias dedupetok { return $regsubex(_dedupetok,$1,$replace(/(?<=^|x)x*+(?:([^x]++)x++(?=(?:.*?x)??\1(?:x|$)))?|x+$/g,x,\Q $+ $chr($$2) $+ \E),) } ; by Raccoon Ouims Saturn 2017