mIRC Home    About    Download    Register    News    Help

Print Thread
/fseek and file handling vs $read #266531 22/12/19 06:05 PM
Joined: Jul 2006
Posts: 3,656
W
Wims Offline OP
Hoopy frood
OP Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 3,656
Thanks for the improvement of file handling.

This script illustrates how /fseek -l seems to always recount the lines from the beginning, /fseeking -l to the same line number consecutively should be a fast operation, doing it 100 times should not take 10 secs.

Could /fseek be improved once again to look for the position in the file and do some smart check in order to avoid recounting from the beginning of the file?

//var %start 10000 , %count 100,%c %count | if ($fopen(foo)) .fclose foo | .fopen foo versions.txt | var %t $ticks | .fseek -l foo %start | echo -ag takes $calc($ticks - %t) ms to do one /fseek -l foo %start from pos=0 | var %t $ticks | while (%count) { .fseek -l foo %start | dec %count } | echo -ag takes $calc($ticks - %t) ms to do $qt(/fseek -l foo %start) %c times from the correct position already.


Looking for a good help channel about mIRC? Check #mircscripting @ irc.swiftirc.net
Re: /fseek and file handling vs $read [Re: Wims] #266533 22/12/19 08:50 PM
Joined: Dec 2002
Posts: 4,691
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 4,691
Quote
1) This script illustrates how /fseek -l seems to always recount the lines from the beginning

That is intentional. The /fopen feature allows scripts to access and update files, possibly while these files are being accessed and updated by other applications. /fseek -l needs to count lines from the beginning every time to ensure it is taking account of all of the CR/LFs in the file. There are no plans to change this. If you want to count lines yourself, you can use /fseek -n and take the chance that none of the lines in the file have been updated since your last /fseek -n.

Re: /fseek and file handling vs $read [Re: Khaled] #266562 27/12/19 03:14 PM
Joined: Jul 2006
Posts: 3,656
W
Wims Offline OP
Hoopy frood
OP Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 3,656
Ok, I must say I didn't know about this, it is not clear from the help file.

I also believe that unless you're the one updating the file from an external application at the same time, it would be impossible to know how a third party updated your file. However maybe this is good for communication between two mIRCs?

It seems to me that most scripts using file handling currently don't want external application to update their content (probably not even read it) and would want to use the recent -x switch of /fopen.

In any case, it looks like the optimization of /fseek I mentioned is possible to do when -x is used with /fopen, what do you think?


Looking for a good help channel about mIRC? Check #mircscripting @ irc.swiftirc.net
Re: /fseek and file handling vs $read [Re: Wims] #266579 29/12/19 11:55 AM
Joined: Dec 2002
Posts: 4,691
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 4,691
Right, so it sounds like you are asking for this behaviour:

If /fopen -x is used to open a file exclusively, you want /fseek -l to create a cached index of lines in the file and to maintain this cache for as long as the file is open, to allow /fseek -l to jump to that line quickly. In addition, if the script uses /fwrite to write to the file at any time, this will clear the cache, requiring /fseek -l to rebuild the cache.

As this is a feature request, I have moved this thread to the feature suggestions forum.

Re: /fseek and file handling vs $read [Re: Khaled] #266584 29/12/19 05:24 PM
Joined: Jan 2004
Posts: 1,282
maroon Offline
Hoopy frood
Offline
Hoopy frood
Joined: Jan 2004
Posts: 1,282
Just an observation that the index wouldn't need to hold every line position for this to be effective. A couple dozen pointers stored for each file would be enough to divide the file into chunks which are each 4% of the line count. From there, it would be easy to divide the file similar to how the buckets divide the hashtable, so fseek could limit the search within the 400 lines within that chunk, instead of the 10000 lines of the entire file.