DaveC:
Thanks! Thats exactly what I needed. Oh and hash tables do stay in the index you add them, I'm pretty efficent with them now. From your description of tokens I can totally see that a mixture is what I need, I can see really good uses for both.

Thanks for the great post I really appreciate it!!

One last question, is there anyway to use other things then ASCII values like regular expressions as your separator?