Forums Bug Reports /hsave problem

Hello, i found a error probably, when my mirc hsave my hashtables, it's gonna be lagging, you can see screenshot under
(yes i check it, lagging only when hsave)



first settings:


2nd:




with 2nd Settings, there no lags more, so probably it's about hsave to .Ini file, i hope u will fix it in next version.

Regards.
sry for my bad english.

if you have a really big data set it can be expected, because saving it are disc writes.

what you are trying to do is write 4 sets of data to files at approximately the same time, which means your harddisc will have to switch around 4 different locations, trying to write all the data. if your harddisc is really heavy on usage, it can cause lag on your whole system.

one tip would be, that when it's not critical that it has to be done at the same time, to delay some of the saves so they are more sequential instead of simultanius.

ini files will take more processing aswell since it depends on windows API's which are most likely slower compared to normal writes.

this doesn't seem like a bug at all, bit is simply a hardware/OS limitation. but it highly depends also on what you are writing and what the actual difference is between the two attempted methods.

mIRC isn't a multi-threaded application (Apparently it is) mIRC scripts aren't multi-threaded anyway, so it would save them in sequence, one after the other.

mIRC 7 does seem to take considerably longer to save/load hash tables though. Here's a measurement I took of loading a hash file (36315 items):
v7.0 - 3500ms~
v6.35 - 1600ms~

However, v7 seems to save hash files differently. After saving the same data to a file, it gains 72630 bytes compared to v6.35.
Edit: This appears to be due to v7 appending CR as well as LF at new lines when saving, even without using the -b switch.

If I load the file saved by v7 on v6.35, it takes about 5100ms~ to load, but after saving it again on v6.35, it reduces file size and loads at 1600ms~ again. The load time is barely changed using either file on v7.0.
So what's going on here?

Edit: v6.35 also seems to be up to 20x faster at saving, 100ms compared to 2000ms (not averages)

khaled did state he made a change to the INI file handling. maybe this has negatively impacted /HSAVE more than expected.

it's good to maybe have some more tests of this on different systems? (XP, Vista, 7)

Any update on this? Preferably from Khaled? It takes so long to start mIRC now! (Loading a hash file on load)

I have not been able to reproduce this issue here so far. Can you provide me with a small, short, minimal script that runs under v6.35 and v7.x that reproduces the issue for you?

(Presuming you asked me, either way:) I made a script available at http://dl.dropbox.com/u/234274/mIRC/htlcmp.mrc (script info in file)
I used the following versions for testing: mIRC 6.35, 7.01 and 7.02
I used the hash table generator (which creates and saves a table) to create the test file. (Side note at bottom of post)
The file I tested with: http://dl.dropbox.com/u/234274/mIRC/test702.hsh
I ran the generated file through each version and obtained these results:

Note: I created a hash table using each version of mIRC because I remember that 7.0 files took forever to load on 6.35 (or possibly vice versa), although I couldn't test 7.0 without getting a "Beta expired" notice. Between the versions I tested with, there was no noticeable difference in load time between files made in different versions. HOWEVER, 7.xx does save files using CRLF instead of just LF as in 6.35 (no switches). The only difference this makes is file size. According to the help file, this should only happen when using the -b switch.

Thanks for the script. The issue you are describing is different from the original post in this thread (although it might be related), where the user reported delays when saving hash tables as ini files.

I was able to reproduce your issue. It is taking longer because mIRC has to convert each line to/from Unicode/UTF-8 when loading/saving. Older versions of mIRC saved files as single-byte character data and did not require conversion. Unicode uses two bytes to store text, so it needs to be converted to UTF-8 before it is saved to a file. The UTF-8 then needs to be converted back to Unicode when it is loaded.

mIRC v7.02 is taking even longer because of the BOM handling. I have optimized that and it will be back to the same speed as v7.01 in the next version.

Regarding the CRLF difference: mIRC now uses a standard set of file-handling routines that are used consistently throughout mIRC. Older versions of mIRC were not consistent across routines in whether text ended with an LF or CRLF. The /hsave entry in the help file is referring to the difference between saving as text or as binary data, which is not related to this change.

What about storing binary hash table dumps (non .ini files) out as UTF-16 from now on? That would make files /hsave'd in 7.x unusable in 6.x but that shouldn't be such a big deal. If you read the BOM you can just grab the contents without any conversion (which I suspect mIRC already does with the new BOM support anyway).

Or at the very least, give us an option when we save/load them to save/load the as Mirc 6.x has. If unicode is slowing them down a significant amount, allowing us to save them in non-unicode may be helpful.

It's not "unicode" that's slowing it down. It's the fact that the data is non-UTF16 that's slowing it down, so an option wouldn't be helpful besides for compatibility sake. Even if the data was ASCII, mIRC would still need to convert it to UTF-16 when reading it, since mIRC stores all internal data as UTF-16. This is why I suggested storing the data as UTF-16, so mIRC would not need to perform any conversion.