Forums Bug Reports hload -i vs hadd -b

The /help /hload says "$cr and $lf characters are stripped from text when saving as plain text." but they're not being stripped when using -i as does happen by default or with -n.

This alias demonstrates this bug, but also demonstrates a quirk that appears to be difficult to demonstrate without this -i bug being present: It appears that hfree doesn't make mIRC completely forget about the contents of the table that's been hsaved to disk then hfree'ed. The alias shows that $cr and $lf within the &binvar are written to disk by the -i switch. It also shows that, even when the hsaved data is written in the format where hload-i should see there's 4 items on disk, the data gets hloaded with only 3 items off the disk, with one of the items containing $cr $lf, which could happen only if it remembered the state of the hashtable that's already been hfree'ed.

The last line of the alias shows the hsave'ed disk file in binary mode, with reversed colors at the position prior to the $cr $lf within the binary item.

/hloadi_test
/hloadi_test -n

Using no switch or using -n writes to disk with that $cr $lf stripped from disk.

/hloadi_test -i

Using the -i switch writes to disk including the $cr $lf, allowing the remainder of the binary data to simulate a 4th item, but hload won't put that 4th item into the hashtable until you hload that data after it's been copied to another file then hloaded from there.

--

And no, I'm not trying to do strange tricks like in the alias to create modified disk files. I came across this due to unexpected content of a file that had been created by hsave using the -i switch. And then, when I was creating the demo alias, I was finding that hload was 'seeing' something completely different than what was on disk.

--

Thanks for your bug report.


These are not stripped when saving to INI or binary files. This behaviour with INI files was established a long time ago, eg. /writeini will not strip $crlf characters from any of the INI parameters either. It is up to the scripter to ensure that the lines they are writing to INI files are in the correct format.

In the case of INI files, if you use /hsave or /writeini to write text that contains $crlf characters, this will result in an INI file that may not save in the format you are expecting.

I will update the help file for /hsave to make this clear.

One possible solution would be to add a switch to /hsave and /writeini that strips out $crlf from all INI parameters before writing to the INI file.

A switch like that would be good. The ini format seems like it's designed for text lines containing item=data which end with $crlf, so I'm thinking that most users would not expect $crlf within data to be written to disk. I was tempted to write from -default mode for saveini to -i because -i is easier to read, without having to figure out which line in a big file is the itemname and which is the data.

Looking at the wikipedia page was a little confusing since there doesn't seem to be a unified INI format, so I'm not yet sure which all variants are being supported by hsave/writeini. Based on what I read there, I did find something unexpected related to hashtable or writeini itemnames beginning with semi-colons, as I hadn't realized that INI supported optional comments.

In this example, $ini and $readini can see the itemname beginning with semicolon when looking at the file they wrote to, but can't see it when that file is copied to another filename. Same happens with /hload using a cached version of a non-existant hashtable when reading from that same file, but can't see the same thing from another file written from /writeini.

And just like with the prior alias, once the client restarts and the cache is gone, the data following that embedded $crlf moves to a new itemname if it contains an embedded '=', or becomes data that's ignored. And the itemnames beginning with semicolon which formerly had been seen are now invisible.

//echo -a === | hfree -w test* | var %file1 test1.ini , %file2 test2.ini , %file3 test3.ini , %i 1 , %j 1 | remove %file1 | writeini %file1 hashtable ;not-a-comment words words | writeini %file1 hashtable item data | flushini %file1 | while ($ini(%file1,hashtable,%i)) { echo 4 -a %i : $v1 : $readini(%file1,hashtable,$v1) | inc %i } | echo -a now let's again from another filename where count changes 2->1 | copy -o %file1 %file2 | while ($ini(%file2,hashtable,%j)) { echo 3 -a %j : $v1 : $readini(%file2,hashtable,$v1) | inc %j } | hadd -m test item data | hadd test ;not-a-comment words words | hsave -si test %file3 | hfree -s test | hload -sim test %file1 | noop $hfind(test,*,0,w,echo 3 -a $1 $hget(test,$1) ) | hload -sim test3 %file3 | noop $hfind(test3,*,0,w,echo 4 -a $1 $hget(test3,$1) )

As /writeini was implemented so long ago, I can't remember the exact reason that $crlf was allowed. It could be that the Windows INI API itself allowed/allows $crlf in parameters, so it was decided that /writeini should match its behaviour. Or it could be that some scripters at the time were using $crlf to write multiple ini entries with one /writeini call.


There is no consistency in INI file implementations. mIRC used to use the Windows INI API but there were persistent issues with it so, after years of trying to mitigate all of the issues, and testing the available custom INI libraries at the time (all of which had their own issues and limitations), I was forced to implement my own that worked in a similar way to the Windows API. It was a difficult and time-consuming implementation. You will certainly find quirks and edge cases if you look for them. As far as I am concerned, it works well enough.


I have added this to my to-do list and may look into it at some point.

For when looking at the todo item. Just an observation about the possible reason for not stripping $crlf from writeini being that people might have wanted to write 2 items at the same time. If that were happening in the beginning, the change from API to internal routine would have made that more difficult, because of how the cache has been causing the entire disk write to be treated as if a single item until after the cache has been cleared either from copy to new file or after client restart. In my most recent alias, when replacing:

writeini %file1 hashtable item data

with:

writeini %file1 hashtable item $+(data,$crlf,item2=data2)

... $readini sees the entire string as the same item until the cache is cleared, and while the cache is active /remini against the item preceding the $crlf also deletes the item structure following the $crlf.

Yes, there were other issues and compromises to consider as well.

One of the issues with the Windows INI API is that although it interprets/interpreted semi-colon prefixed lines as comments (and at some point may have interpreted trailing semi-colons in lines as comments) when reading an INI file, it does/did not support writing comments to INI files. If the INI API was used to write items beginning with semi-colons, they would then be treated as comments when reading. Since scripters, and some of the default INI files, used semi-colons for comments, these had to be interpreted as such. In addition, since comments are not tied to a section or item, whenever sections or items are/were deleted, re-created, etc. the comments would end up in the wrong place in the INI file.

I have made a change to the next beta that should fix the caching issue you are seeing.