I have a large hash table I use as a !seen database (records information useful for stalkers).
In my 1st, generation, I actually tried to trim it by having a timer running in the background trying to clean it up.
Then about 4 years ago, I gave up and just let it grow to whatever size.
It's been hovering around half a million entries and 50 megs now, and I'm wondering what I should do with it. Saving it takes 6 seconds I'd rather not waste and it's just been slow to use (but REALLY informative with that much data).
How you do maintain such a huge hash file?
One option I've been contemplating is to split it up. Put more important data like when they were last seen in one table, and in another table put the descriptions (kick messages, quit messages), but that really wouldn't' solve my problem.
Ideally there'd be a large datastructre of a defined size with a least recently used tag for each entry.
