mIRC Homepage

/fseek and file handling vs $read

Posted By: Wims

/fseek and file handling vs $read - 22/12/19 06:05 PM

Thanks for the improvement of file handling.

This script illustrates how /fseek -l seems to always recount the lines from the beginning, /fseeking -l to the same line number consecutively should be a fast operation, doing it 100 times should not take 10 secs.

Could /fseek be improved once again to look for the position in the file and do some smart check in order to avoid recounting from the beginning of the file?

//var %start 10000 , %count 100,%c %count | if ($fopen(foo)) .fclose foo | .fopen foo versions.txt | var %t $ticks | .fseek -l foo %start | echo -ag takes $calc($ticks - %t) ms to do one /fseek -l foo %start from pos=0 | var %t $ticks | while (%count) { .fseek -l foo %start | dec %count } | echo -ag takes $calc($ticks - %t) ms to do $qt(/fseek -l foo %start) %c times from the correct position already.
Posted By: Khaled

Re: /fseek and file handling vs $read - 22/12/19 08:50 PM

1) This script illustrates how /fseek -l seems to always recount the lines from the beginning

That is intentional. The /fopen feature allows scripts to access and update files, possibly while these files are being accessed and updated by other applications. /fseek -l needs to count lines from the beginning every time to ensure it is taking account of all of the CR/LFs in the file. There are no plans to change this. If you want to count lines yourself, you can use /fseek -n and take the chance that none of the lines in the file have been updated since your last /fseek -n.
Posted By: Wims

Re: /fseek and file handling vs $read - 27/12/19 03:14 PM

Ok, I must say I didn't know about this, it is not clear from the help file.

I also believe that unless you're the one updating the file from an external application at the same time, it would be impossible to know how a third party updated your file. However maybe this is good for communication between two mIRCs?

It seems to me that most scripts using file handling currently don't want external application to update their content (probably not even read it) and would want to use the recent -x switch of /fopen.

In any case, it looks like the optimization of /fseek I mentioned is possible to do when -x is used with /fopen, what do you think?
Posted By: Khaled

Re: /fseek and file handling vs $read - 29/12/19 11:55 AM

Right, so it sounds like you are asking for this behaviour:

If /fopen -x is used to open a file exclusively, you want /fseek -l to create a cached index of lines in the file and to maintain this cache for as long as the file is open, to allow /fseek -l to jump to that line quickly. In addition, if the script uses /fwrite to write to the file at any time, this will clear the cache, requiring /fseek -l to rebuild the cache.

As this is a feature request, I have moved this thread to the feature suggestions forum.
Posted By: maroon

Re: /fseek and file handling vs $read - 29/12/19 05:24 PM

Just an observation that the index wouldn't need to hold every line position for this to be effective. A couple dozen pointers stored for each file would be enough to divide the file into chunks which are each 4% of the line count. From there, it would be easy to divide the file similar to how the buckets divide the hashtable, so fseek could limit the search within the 400 lines within that chunk, instead of the 10000 lines of the entire file.
© 2020 mIRC Discussion Forums