10000 is just a soft limit, the same way mIRC limits lines of text to 4096 bytes. There's no technical reason besides the fact that making an assumption about an upper-bound allows you to optimize a few things.
Loading a 35mb text file is going to take time. It's not going to be instant, and it should "eat RAM" (specifically at *least* 70mb ram, if not much more). It could easily take anywhere from 10 seconds to a minute or so to load 35mb from disk into memory, since you're performing 4 million hash operations on that data as well. Why is this surprising you?
You shouldn't try to load large files into memory all at once, you should create a better way to query it either from disk using an in-memory cache or offload all that work to a DB system (SQL, BDB, etc.).
