Forums Scripts & Popups Faster file writing or faster sorting?

Okay here's the setup. I have a list of data that needs sorted alphabetically, channel nicks actually, as well as some other data. So I write it to a file and use /filter thing is the /write is really slow. Normally the data is small so it isn't noticeable, however in large channels it causes mirc to pause when ran...

lag! D:

I don't want to use an @window. Any dlls for faster write? Faster ways to code data writing? Better ways to sort a hash table?

The main thing limiting your write speed for a file is your hard drive speed. The only way of increasing that is to get a faster hard drive. I've got a feeling that this is not a reasonable option at this point.
Hash tables are stored in an unsorted format, and are (realistically) unsortable. Regarding DLL's, there might be, but I'm unaware of any.
Realistically (if not literally) using an @window would be your fastest option, so I have to wonder why you don't want to use an @window?
If coded properly, you could use the @window to do the sorting, and (probably) not even notice the window (ie: using a hidden window to do the sorting, and then close it automatically when complete)

Well, hash tables can easily be sorted, but it requires writing it to a file first. The information on how is located in the forum if anyone feels like trying to find it. It was a long time ago and I *think* it may have been written originally by DaveC, though after so long, I could be wrong.

Anyhow, RusselB is right that windows are the best option if you really have that much trouble writing the information and they can be hidden like he said. I'm wondering, though, why it's giving you so much trouble to write it to a file. It could be that your script needs to be improved. Writing even a few hundred lines shouldn't really lag you any.

Yes, I didn't have any test information, but I just did a quick script to see how long my system is lagged for how many lines written, and the results were a 2 millisecond delay (using ticks to calculate) when writing 10,000 lines. Now I know I have a fast system, but even if you multiplied that by a factor of 100, that would still mean a 2 millisecond delay for 100 lines.

Cool, so it's the multiple write operations overhead. Not the actual writing it's self.

I'll just put it all in a big variable, how do I put the line break in between the variables in a single write operation?

you can do that by appending $crlf to the end of each line, but note that variables have a maximum length of about 950 characters (which includes the variable name, the = sign used to assign the variable and all of the characters in the variable). To append the $crlf you have to put like <line1> $+ $crlf $+ <line2> $+ $crlf

So if you have 4 lines at 200 characters each, and you call the variable %a, then that's going to give you a variable that's about 839 characters in length (still under the 950, but hopefully it gives you an idea as to how quickly that 950 character mark can be reached)

I still have to recommend that you consider using @windows for your sorting, especially as you've not said why you don't want to use them.

Thank you both much appreciated.

The main reason is I'd have to recode the data reloader. Also I'm kinda worried about the window stealing the focus. It's not an issue except in very large channels (2000+). That and I'm still learning @windows.

Hide the window and it can't steal focus and you won't even know it is even there.

This makes the @info window a "sorted list box" and since it's "hidden" to start off with, it allows your text to be written to the window a bit faster, AND prevents it from stealing focus. Especially if you choose to close the window after you're finished instead of making it the active window.

If you still prefer to actually *write* to a file, you can use:

fopen myfile the_file_here.txt
fwrite -n myfile %data
fwrite -n myfile %data2
fwrite -n myfile %data3
fclose myfile

The above works much faster then the standard /write.

This code can be used to filter a #channel to a file.txt.



Syntax: /listfilter #channel file.txt

The /listfilter alias can only be called once at a time. A variable ensures that the alias is not called again before the first occurance has completed.

I tested it on a channel of 1000+ users, and it took less than 1 seconds to complete.

-genius_at_work

What about dumping all the lines of a text file in $sorttok dividing them by some oddball character that you know noone uses. then while loop it back into writing it sorted line by line into the file? Since your just looking to do this with nicks,

Seriously, read the original post.

He wanted to avoid writing to a file in the first place, because "WRITE" takes too long. The "example" you gave is just a bit too process intensive.

This is why my above post suggests the use of using a sorted list window /filter the nicklist into the sorted @window and then do whatever he needs to based on the @window. As far as writing to files goes, it's either /filter OR /fopen, /fwrite, and /fclose.

In his original post he doesn't mention not writing to a file. Yes he says it's slow, but his post wasn't just about writing to a file, but also sorting too. My post was just to display some $sorttok info not actually telling him to write to a file. Get the info from something else and send it to $sorttok for %varing then do something with it. I have noway of knowing the posters scripting experience that's why I just made something simple.

However, yeah you're right I have no business replying to someone else's post even if it is to offer some type of help. Touch you and your godlyness. Bow down before the mighty scripter. Me and my crappy "example" might as well stay away from here if we are just going to be put down.

Getting right back to the first thing, I would assume your doing something drasticly wrong with the /WRITE operation, are u just adding lines to the file, or are you inserting them?
becuase as Russel said, just writing 10,000 lines takes stuff all time. And if its not the write I wonder if its the loop your using to fetch each line for writing, maybe its flawed.
I have seen several reasons a script that must loop well start falling on its butt, one of the biggest and simplest mistakes someone well make looks like this

var %i = 1 | while (%i <= $fline(#channel,*matchtext*,0)) { .......code using $fline(#channel,*matchtext*,%i)....... | inc %i }
^ that goes and searches for every matchtext getting the total every single time it loops!

var %i = 1, %m = $fline(#channel,*matchtext*,0) | while (%i <= %m) { .......code using $fline(#channel,*matchtext*,%i)....... | inc %i }
^ thats much better since it need only check the total once

window -h @window | filter -wwc #channel @window *matchtext* | var %i = 1, %m = $line(@window,0) | while (%i <= %m) { .......code using $line(@window,%i)....... | inc %i } | window -c @window
^ that should be faster again (but uses a @window, i however assume you didnt want to use one from lack of knowledge on them as you said)

alias filter.alias.name { .......code using $1....... } | ; note $1 contains the WHOLE LINE! see tokenize to seperate!
&
filter -wk #channel filter.alias.name *matchtext*
^ that is about as fast as i can think of looping with code that you have to execute

filter -wf #channel filename.txt *matchtext*
^ and thats about as fast as fast as you can get selected lines out to a file

---

Without sample code to show us what your doing i doubt you well find a solution. thats well make your problem go away.
The best i can suggest if its JUST write speed is that you switch to using /FWRITE whcih requeres you to open, write and close the file your writting to.

Instead of the second /filter to a file, could he use /savebuf of the custom listwindow if it is opened with the -s auto-sort parameter?

I'm not sure if it's quicker or if /savebuf applies to a listwindow, and if the set window buffersize in options -> other is too small this method may fail.

I see what your meaning however...
Using a list window has overheads likely to do with the selectability functions of a list window, even more so if your sorting the list.
It would have been better to use a plain window in your example, also using -hlS does not work as the -l alone makes the main window area the listbox whcih means u would have needed the -s option to sort it, i however assumed you had ment this rather than the -S.

Heres a small alias and the results i got using it on different window types.

Using " /window -h " took 5078 ticks. result unsorted
Using " /window -hs " took 5110 ticks.
Using " /window -hl " took 10828 ticks. result unsorted
Using " /window -hls " took 73547 ticks.

As you can see a main text window if sorted or not takes near the same time
A listbox took twice the time while unsorted and over 12 times the time when sorted

From what i gather if you are sorting a window it is likely sorted to a memory buffer before being output to its destination.
I expect that this well make it faster than having to construct a hidden window and write to it letting it sort and then savebuf it to a file.

Bit of nitpicking, but when you write the file, and sort the file, you are not sorting the hash table. You're simply sorting the file which holds the information from the hash table. Your hash table is unchanged. Hash tables by nature aren't supposed to be sorted, that would defeat their purpose.

I know you meant it differently, but I have seen many people state they sort a hash table (fex with /filter) which is simply a silly thing to say.

Sweet deal. No lag with that. Thanks heaps.