|
|
|
|
|
Joined: Nov 2004
Posts: 842
Hoopy frood
|
OP
Hoopy frood
Joined: Nov 2004
Posts: 842 |
This morning my laptop had a BSoD, and upon trying to run mIRC when I logged on, I was surprised to discover that my mirc.ini had completely wiped itself and restored itself to a default mirci.ini (which is even more odd because I always delete the default files), and that my servers.ini had deleted all servers.
Other files like control.ini, popups.ini and toolbar.ini were fine.
Is there any reason why this would have happened?
What do you do at the end of the world? Are you busy? Will you save us?
|
|
|
|
|
Joined: Dec 2002
Posts: 5,493
Hoopy frood
|
|
Hoopy frood
Joined: Dec 2002
Posts: 5,493 |
This is a common issue that happens with running applications when windows crashes. It will especially happen with applications that are constantly reading and writing to a file at the time of the crash. mIRC will always recreate mirc.ini, servers.ini, and any other settings files if it is unable to load them for some reason. Other common reasons for file loss are anti-virus/security software incorrectly thinking that mIRC and/or its files may be trojan/virus-related and deleting them without telling you. This can also happen when Windows restarts and the anti-virus/security software performs a startup scan.
|
|
|
|
|
Joined: Nov 2004
Posts: 842
Hoopy frood
|
|
OP
Hoopy frood
Joined: Nov 2004
Posts: 842 |
Ah, so it's not a mIRC issue.
I was just taken aback since this laptop crashes at least three times as a week and this has never happened before...
What do you do at the end of the world? Are you busy? Will you save us?
|
|
|
|
|
Joined: Oct 2003
Posts: 3,918
Hoopy frood
|
|
Hoopy frood
Joined: Oct 2003
Posts: 3,918 |
I should point out that never in my 20+ years of using Windows have I run into another application (besides mIRC) that has corrupted a configuration file during a system reset / crash / etc. -- on the other hand, I've had mIRC corrupt the ini file on at least 5+ known occasions (the most recent a few weeks ago.
Although this is technically not specific in theory to mIRC, it is very specific in practice to mIRC's implementation alone. This is is really not common in other applications (the linked Google search targets generic data loss, most likely for unsaved work, which is very different from this specific use case. You'll note that none of those search results actually refer to corrupted applications, just user data, and many of those results are specific to corruption at the hardware level), and if other applications did this, they should expect to receive bug reports about it and eventually fix the root cause. That is to say, mIRC could be doing more to at least mitigate this issue if not outright resolve it. The sad part here is that all the hand-waving about this being a "common Windows issue" completely glosses over how easy it is to mitigate/fix the issue altogether, regardless of how innate it is to the OS (though I disagree with the premise).
Although I don't know the exact technique mIRC uses, I can make educated guesses that part of the problem comes from the likelihood that mIRC is constantly opening mirc.ini for writing, rewriting the entire ini file in one shot (i.e., opening the file in truncate mode) and then holding on the handle for a "while" without flushing written contents to disk. If that's the case, a simple fsync() (or FlushFileBuffers() in WINAPI) would mostly solve the problem in most cases except where Windows crashes between writing and flushing, which should be exceedingly rare. If mIRC flushes as soon as it writes, this should never be a problem. If mIRC is not already doing this, it should. File contents are never corrupted if mIRC is only opening a file for reading (or not modifying the buffer). Again, I am probably wrong about some of the details, but I do know that this has never been an issue for any other program I've used that writes configuration (namely, all of them).
I personally think this issue deserves more attention and legitimacy as a real problem. Users in various support channels complain about this fairly often-- often enough that it should be resolved or at least mitigated by changing mIRC's file handling routines to be more robust.
- argv[0] on EFnet #mIRC
- "Life is a pointer to an integer without a cast"
|
|
|
|
|
Joined: Dec 2002
Posts: 5,493
Hoopy frood
|
|
Hoopy frood
Joined: Dec 2002
Posts: 5,493 |
If you have been using mIRC for many years and you have only seen it corrupt the ini file on so few occasions during that time, and it is one of your most used applications, that seems to imply that it is pretty reliable. At such a low rate of incidence, statistically those few occasions where the files were lost could be attributable to almost anything eg. Windows issues, background applications, anti-virus/security software, and so on.
mIRC opens, writes, closes, and flushes all files to disk immediately.
mIRC saves files in a two step process. It creates a temporary file first, saves all settings to it, and only if that is successful, with no errors, does it then delete mirc.ini and rename the temporary file to mirc.ini. If there is an error of any kind during the process, it leaves the original mirc.ini untouched. This is the safest way to update files.
In this case, both the mirc.ini and servers.ini were corrupted which implies that something else has happened. When Windows is in a low resource state and Windows and/or applications are unable to allocate memory or save files to disk, and end up freezing/crashing because of the low resource state, almost anything can happen unfortunately.
That said, as I have never been able to reproduce this myself, if someone can find a set of steps that reproduce this reliably while forcing a Windows BSOD (note that this really could result in file corruption in Windows or other applications), I will try to follow your set of steps to reproduce it.
|
|
|
|
|
Joined: Apr 2004
Posts: 871
Hoopy frood
|
|
Hoopy frood
Joined: Apr 2004
Posts: 871 |
Even if there may be additional problems at play in some cases, after a quick look we can only conclude that mIRC is not doing what it should be. Collective pulled a system call trace which quite clearly shows that mIRC leaves a "hole" during which an unclean system shutdown can result in the loss of mirc.ini. In particular, mIRC's procedure to update mirc.ini can be represented in pseudocode as follows: tmp1 = genMircIniTmpName();
// ..create and fill tmp1 with new mirc.ini contents here, and close it..
tmp2 = genMircIniTmpName();
MoveFile("mirc.ini", tmp2);
DeleteFile(tmp2);
MoveFile(tmp1, "mirc.ini");Why mIRC renames the old file before deleting it is anybody's guess, but between the two MoveFile calls there is a brief period of time during which there is no mirc.ini on the file system - there will be one or two temporary files, but as far as we know, mIRC does not try to recover those files on startup if it finds no mirc.ini file. Instead, what mIRC should be doing, is use something like ReplaceFile(), or at least MoveFileEx() with MOVEFILE_REPLACE_EXISTING, so that the old mirc.ini is atomically replaced with the new one. At least in that case, the file system can do its job of ensuring that in all cases, either the old or the new mirc.ini will be present on disk after a system crash. Edit: as for reproducibility, just let mIRC crash right before the second MoveFile() call. No BSOD needed. For completeness, below is Collective's system call trace. The comments are mine. 11:39:14.9233914 mirc.exe 11000 CreateFile E:\mIRC\mirc82882.tm_ SUCCESS Desired Access: Generic Read/Write, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0, OpenResult: Created 00:00:06.9969309
11:39:14.9243209 mirc.exe 11000 ReadFile E:\mIRC\mirc82882.tm_ END OF FILE Offset: 0, Length: 2, Priority: Normal 00:00:06.9978604
11:39:14.9243583 mirc.exe 11000 WriteFile E:\mIRC\mirc82882.tm_ SUCCESS Offset: 0, Length: 3,760, Priority: Normal 00:00:06.9978978
11:39:14.9244117 mirc.exe 11000 WriteFile E:\mIRC\mirc82882.tm_ SUCCESS Offset: 3,760, Length: 1 00:00:06.9979512
11:39:14.9252286 mirc.exe 11000 WriteFile E:\mIRC\mirc82882.tm_ SUCCESS Offset: 3,761, Length: 4,144, Priority: Normal 00:00:06.9987681
11:39:14.9256091 mirc.exe 11000 WriteFile E:\mIRC\mirc82882.tm_ SUCCESS Offset: 7,905, Length: 2,114 00:00:06.9991486
11:39:14.9256325 mirc.exe 11000 CloseFile E:\mIRC\mirc82882.tm_ SUCCESS 00:00:06.9991720
11:39:14.9258079 MsMpEng.exe 976 CreateFileMapping E:\mIRC\mirc82882.tm_ FILE LOCKED WITH WRITERS SyncType: SyncTypeCreateSection, PageProtection: 00:00:06.9993474
11:39:14.9258273 MsMpEng.exe 976 QueryStandardInformationFile E:\mIRC\mirc82882.tm_ SUCCESS AllocationSize: 16,384, EndOfFile: 10,019, NumberOfLinks: 1, DeletePending: False, Directory: False 00:00:06.9993668
11:39:14.9287327 mirc.exe 11000 QueryOpen E:\mIRC\mirc82882.tm_ SUCCESS CreationTime: 22/05/2016 11:39:14, LastAccessTime: 22/05/2016 11:39:14, LastWriteTime: 22/05/2016 11:39:14, ChangeTime: 22/05/2016 11:39:14, AllocationSize: 12,288, EndOfFile: 10,019, FileAttributes: A 00:00:07.0022722
11:39:14.9287971 mirc.exe 11000 QueryDirectory E:\mIRC\mirc82882.tm_ SUCCESS Filter: mirc82882.tm_, 1: mirc82882.tm_ 00:00:07.0023366
11:39:14.9288770 mirc.exe 11000 QueryOpen E:\mIRC\mirc82882.tm_ SUCCESS CreationTime: 22/05/2016 11:39:14, LastAccessTime: 22/05/2016 11:39:14, LastWriteTime: 22/05/2016 11:39:14, ChangeTime: 22/05/2016 11:39:14, AllocationSize: 12,288, EndOfFile: 10,019, FileAttributes: A 00:00:07.0024165
11:39:14.9289222 mirc.exe 11000 QueryOpen E:\mIRC\mirc.ini SUCCESS CreationTime: 12/03/2015 18:57:00, LastAccessTime: 22/05/2016 05:37:21, LastWriteTime: 22/05/2016 05:37:21, ChangeTime: 22/05/2016 05:37:21, AllocationSize: 12,288, EndOfFile: 10,018, FileAttributes: A 00:00:07.0024617
11:39:14.9289681 mirc.exe 11000 QueryOpen E:\mIRC\mirc.ini SUCCESS CreationTime: 12/03/2015 18:57:00, LastAccessTime: 22/05/2016 05:37:21, LastWriteTime: 22/05/2016 05:37:21, ChangeTime: 22/05/2016 05:37:21, AllocationSize: 12,288, EndOfFile: 10,018, FileAttributes: A 00:00:07.0025076
# MoveFileW start
11:39:14.9290127 mirc.exe 11000 CreateFile E:\mIRC\mirc.ini SUCCESS Desired Access: Read Attributes, Delete, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 00:00:07.0025522
11:39:14.9290451 mirc.exe 11000 QueryAttributeTagFile E:\mIRC\mirc.ini SUCCESS Attributes: A, ReparseTag: 0x0 00:00:07.0025846
11:39:14.9290612 mirc.exe 11000 QueryBasicInformationFile E:\mIRC\mirc.ini SUCCESS CreationTime: 12/03/2015 18:57:00, LastAccessTime: 22/05/2016 05:37:21, LastWriteTime: 22/05/2016 05:37:21, ChangeTime: 22/05/2016 05:37:21, FileAttributes: A 00:00:07.0026007
11:39:14.9291271 mirc.exe 11000 SetRenameInformationFile E:\mIRC\mirc.ini SUCCESS ReplaceIfExists: False, FileName: E:\mIRC\mirc180652.tm_ 00:00:07.0026666
11:39:14.9292512 mirc.exe 11000 CloseFile E:\mIRC\mirc180652.tm_ SUCCESS 00:00:07.0027907
# MoveFileW end
# DeleteFileW start
11:39:14.9293209 mirc.exe 11000 CreateFile E:\mIRC\mirc180652.tm_ SUCCESS Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 00:00:07.0028604
11:39:14.9293622 mirc.exe 11000 QueryAttributeTagFile E:\mIRC\mirc180652.tm_ SUCCESS Attributes: A, ReparseTag: 0x0 00:00:07.0029017
11:39:14.9293879 mirc.exe 11000 SetDispositionInformationFile E:\mIRC\mirc180652.tm_ SUCCESS Delete: True 00:00:07.0029274
11:39:14.9294170 mirc.exe 11000 CloseFile E:\mIRC\mirc180652.tm_ SUCCESS 00:00:07.0029565
# DeleteFileW end
# MoveFileW start
11:39:14.9295472 mirc.exe 11000 CreateFile E:\mIRC\mirc82882.tm_ SUCCESS Desired Access: Read Attributes, Delete, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 00:00:07.0030867
11:39:14.9295701 mirc.exe 11000 QueryAttributeTagFile E:\mIRC\mirc82882.tm_ SUCCESS Attributes: A, ReparseTag: 0x0 00:00:07.0031096
11:39:14.9295870 mirc.exe 11000 QueryBasicInformationFile E:\mIRC\mirc82882.tm_ SUCCESS CreationTime: 22/05/2016 11:39:14, LastAccessTime: 22/05/2016 11:39:14, LastWriteTime: 22/05/2016 11:39:14, ChangeTime: 22/05/2016 11:39:14, FileAttributes: A 00:00:07.0031265
11:39:14.9296519 mirc.exe 11000 SetRenameInformationFile E:\mIRC\mirc82882.tm_ SUCCESS ReplaceIfExists: False, FileName: E:\mIRC\mirc.ini 00:00:07.0031914
11:39:14.9297761 mirc.exe 11000 CloseFile E:\mIRC\mirc.ini SUCCESS 00:00:07.0033156
# MoveFileW end
11:39:14.9298752 mirc.exe 11000 QueryDirectory E:\mIRC\mirc.ini SUCCESS Filter: mirc.ini, 1: mirc.ini 00:00:07.0034147
Saturn, QuakeNet staff
|
|
|
|
|
Joined: Dec 2002
Posts: 5,493
Hoopy frood
|
|
Hoopy frood
Joined: Dec 2002
Posts: 5,493 |
Thanks for looking into this, however this behaviour is by design. There are some contexts where a DeleteFile() is queued and may not be processed immediately by Windows, resulting in side-effects if an application tries to access the file soon afterwards, so mIRC specifically forces an immediate move of the original file. This was implemented a long time ago to resolve issues related to DeleteFile().
While I appreciate you looking into this, the only real way to track this down would be to find a way to reproduce it by inducing a BSOD, since that seems to be the specific situation which causes it and which I have not been able to reproduce.
|
|
|
|
|
Joined: Dec 2002
Posts: 3,138
Hoopy frood
|
|
Hoopy frood
Joined: Dec 2002
Posts: 3,138 |
You shouldn't be directly deleting mirc.ini at all, you should be writing a new file and then using ReplaceFile() to move it over to mirc.ini.
There is nothing left to be tracked down. You could easily reproduce this problem yourself simply by inserting an exit(0) into the existing code and looking at the results - the problem is what happens when the code doesn't complete, not why it doesn't complete (whether it's power loss, a BSOD, or something else).
Last edited by Collective; 22/05/16 03:13 PM.
|
|
|
|
|
Joined: Jul 2006
Posts: 4,188
Hoopy frood
|
|
Hoopy frood
Joined: Jul 2006
Posts: 4,188 |
I talked to you about this issue myself on numerous occasion, in private or in here, I'm glad this is brought up again. It's unclear how often/much you use mIRC yourself, but I think you should really trust us in this case. This issue has been happening daily since I started using mIRC, and I only started in 2006, I'm sure it happened before. There is no need for a BSOD to get this to occur.
The typical step for people who report this on scripting channels is to simply turn off the computer with the switch, instead of using the standard way using Windows. Of course just doing this won't make it happen, but it's how users typically experience it.
#mircscripting @ irc.swiftirc.net == the best mIRC help channel
|
|
|
|
|
Joined: Apr 2004
Posts: 871
Hoopy frood
|
|
Hoopy frood
Joined: Apr 2004
Posts: 871 |
Thanks for looking into this, however this behaviour is by design. There are some contexts where a DeleteFile() is queued [..] You're missing the point. mIRC should not be calling DeleteFile() at all here. It should atomically replace the existing file with the new file, and there are API calls to do that. mIRC is currently not using them. mIRC should be using them. If for some reason these API calls fail, mIRC could still fall back to the old, less safe behavior. the only real way to track this down would be to find a way to reproduce it by inducing a BSOD, since that seems to be the specific situation which causes it and which I have not been able to reproduce. No, that is simply not true. Robustness against system crashes starts with robustness against application crashes. The best thing that an application can do to protect itself from problems resulting from a system crash, is to make sure that at all times, the file system contains a state from which the application can recover. In mIRC's case, that means that at the very least there should always be a valid mirc.ini in the file system. This may not be sufficient to prevent all cases of corruption, but it is a necessary condition. Right now, mIRC is playing fast and loose with mirc.ini, which may result in its effective loss.You could ignore the clear problem in front of you and try to figure out exactly what is going on with BSODs, or you could let mIRC do the right thing right now and see if that doesn't already resolve the problem in the majority of the (many, many) current cases. This really shouldn't be a hard decision.
Saturn, QuakeNet staff
|
|
|
|
|
Joined: Jan 2016
Posts: 19
Pikka bird
|
|
Pikka bird
Joined: Jan 2016
Posts: 19 |
This has also happened to me on numerous occasions. And while it's not often, it is still rather annoying.
The last time was when my laptop cut itself off whilst I was sleeping (cutting off, meaning the equivelent of just pulling the plug.)
But, it's obviously not going to be a sure way of reproducing as there's other ways that could cause it.
However it would be nice to see this fixed.
Last edited by Stewie1k94; 22/05/16 03:44 PM.
|
|
|
|
|
Joined: Dec 2002
Posts: 5,493
Hoopy frood
|
|
Hoopy frood
Joined: Dec 2002
Posts: 5,493 |
I am pretty sure I looked into ReplaceFile() many years ago when I reviewed the Windows File APIs but probably decided not to use it because the files must all be on the same volume, which is one of the issues that was resolved with the current implementation after users reported file-saving issues in contexts like network shares, USB drives, and so on. At the time, it was probably also not clear whether ReplaceFile() was an atomic operation. I have found several discussions about it and it sounds like it might be atomic, although it does not say this in the documentation. That said, I will look at it again to see if it makes any difference. One potential issue is that if it fails for some reason, it can leave files in different states that will require a separate call to DeleteFile(), MoveFile(), or both. In any case, it will very likely take several version releases to iron out any issues with using ReplaceFile() as past experience with the File APIs has shown that they can behave unexpectedly (such as queuing a file delete instead of an immediate file delete). Thanks for your feedback everyone.
|
|
|
|
|
Joined: Dec 2002
Posts: 5,493
Hoopy frood
|
|
Hoopy frood
Joined: Dec 2002
Posts: 5,493 |
You shouldn't be directly deleting mirc.ini at all, you should be writing a new file and then using ReplaceFile() to move it over to mirc.ini. mIRC already does what you are suggesting. It writes the output to a temporary file and if that succeeds, it then moves the original file to a separate temporary file, deletes that, and renames the updated file to mirc.ini. The reason for this is that a call to DeleteFile() can pend, which means that Windows will not perform it immediately, which can interfere with subsequent API calls. At least, this was an issue in Windows XP.
|
|
|
|
|
Joined: Dec 2002
Posts: 3,138
Hoopy frood
|
|
Hoopy frood
Joined: Dec 2002
Posts: 3,138 |
What I suggested was to not use DeleteFile. mIRC does use DeleteFile, so it's not doing what I suggested. Your reply suggests that you still do not understand the problem, so I will attempt to explain it another way. Firstly, forget about DeleteFile altogether. This issue would exist even if you weren't using it, because you're renaming mirc.ini to something else, which is as good as deleting for the purpose of this bug. Once you have performed this rename, there is no longer a mirc.ini in the mIRC folder. If execution stops there, either due to a BSOD or a power cut, the file is gone as far as mIRC (and most end-users) are concerned. This is not acceptable. mIRC should not be renaming mirc.ini. mIRC should not be deleting mirc.ini. mIRC should leave mirc.ini alone until it is ready to atomically replace it with a new version. ReplaceFile is recommended by Microsoft for exactly this situation. A brief survey of GitHub suggests that MoveFileEx with MOVEFILE_REPLACE_EXISTING is also a common solution, and it was suggested by Sat above. When you implement one of these options, then and only then will mIRC be doing what I am suggesting.
|
|
|
|
|
Joined: Dec 2002
Posts: 5,493
Hoopy frood
|
|
Hoopy frood
Joined: Dec 2002
Posts: 5,493 |
Very well, thank you for the clarification.
|
|
|
|
|
Joined: Dec 2002
Posts: 5,493
Hoopy frood
|
|
Hoopy frood
Joined: Dec 2002
Posts: 5,493 |
Okay, I have had some time to look into this issue a little more. Some observations: 1) I just noticed that the bug report states that both mirc.ini and servers.ini were empty after a BSOD. However, mIRC can update only one file at a time. If mIRC had been terminated half way through a DeleteFile/MoveFile, it should have affected only one file. 2) I came across the following reports where using ReplaceFile() and MoveFileEx() still resulted in corrupt files after a BSOD or power failure: http://stackoverflow.com/questions/31710431/how-to-reliably-overwrite-a-file-in-windowshttp://stackoverflow.com/questions/6525943/cause-of-corrupted-file-contentsAs the point of switching to ReplaceFile() is to resolve this issue, I decided to do a little more testing. I created a version of mIRC that supports the following modes for replacing files, set by a command line switch when mIRC is run: mode0 Original DeleteFile()/MoveFile(). mode1 ReplaceFile() (with backup file and REPLACEFILE_IGNORE_MERGE_ERRORS | REPLACEFILE_WRITE_THROUGH). mode2 MoveFileEx() (with MOVEFILE_COPY_ALLOWED | MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH). mode3 Alternative to mode0 that renames the original mirc.ini to a temporary file, renames the replacement to mirc.ini, and then deletes the original. I then created a perl script that repeatedly runs mIRC in each mode. When mIRC starts, it repeatedly updates mirc.ini using the mode specified on the command line. The perl script waits for one second and then forcefully terminates mIRC. It then checks to see if mirc.ini still exists and if its size matches a reference file. If mirc.ini is lost or corrupted, it is reset to the reference file. lost: number of times mirc.ini was lost. min: shortest number of runs before mirc.ini was lost. max: longest number of runs before mirc.ini was lost. mean: average number of runs before mirc.ini was lost. When mIRC is set to update mirc.ini 150 times a second, the results after 3500 runs (per mode) were: mode0 lost:328 min:1 max:52 mean:11 loss rate:9.37% mode1 lost:133 min:1 max:93 mean:26 loss rate:3.80% mode2 lost:0 min:0 max:0 mean:0 loss rate:0.00% mode3 lost:182 min:1 max:128 mean:19 loss rate:5.20% When mIRC is set to update mirc.ini 50 times a second, the results after 3500 runs (per mode) were: mode0 lost:120 min:1 max:203 mean:29 loss rate:3.43% mode1 lost:67 min:1 max:196 mean:52 loss rate:1.91% mode2 lost:1 min:1098 max:1098 mean:1098 loss rate:0.03% mode3 lost:66 min:1 max:244 mean:53 loss rate:1.89% When mIRC is set to update mirc.ini 10 times a second, the results after 3500 runs (per mode) were: mode0 lost:30 min:11 max:432 mean:114 loss rate:0.86% mode1 lost:16 min:4 max:701 mean:179 loss rate:0.46% mode2 lost:0 min:0 max:0 mean:0 loss rate:0.00% mode3 lost:15 min:18 max:1061 mean:230 loss rate:0.43% mIRC normally only updates mirc.ini once or twice a second. Notes: 1) The test was performed on Windows 7 with an SSD, so the results are likely to be different on other versions of Windows and on slower, non-SSD drives. 2) The results are unlikely to be applicable to a BSOD or power failure situation, where OS-level caching and drive-level caching are involved, eg. some drives tell the OS that the data has been committed to disk when it is actually still in the drive cache. 3) Mode2 showed almost no file loss. The perl script and implementation in mIRC were checked and looked correct. This appears to be a valid result. The results were the same without MOVEFILE_WRITE_THROUGH for MoveFileEx() and without REPLACEFILE_WRITE_THROUGH for ReplaceFile(), which the documentation says is not supported. The effects of the write-through options might only be apparent on slower, non-SSD drives. 4) When anti-virus software is enabled, ReplaceFile() becomes inceasingly slower than the other modes as the replacement file size increases. With a replacement exe file of 5MB in size, it was ten times slower. Anti-virus software also occasionally locked files and prevented a file replace from completing. The above tests were performed with anti-virus disabled. 5) In both mode0 and mode3, there is a moment when mirc.ini does not exist. However, mode0 performs a delete first, which is more involved due to the DeleteFile() pending issue, and then a rename, while mode3 performs two successive renames, which are much faster, and then a delete. This is probably why mode3 performs at about the same level as ReplaceFile() at lower update rates. Overall, it looks like using MoveFileEx() is the best option. However, as other software developers who use it still see file loss in a BSOD or power failure situation, it is not clear how much it will help in those contexts. I will be releasing a beta with this change for testing soon.
|
|
|
|
|
Joined: Apr 2004
Posts: 871
Hoopy frood
|
|
Hoopy frood
Joined: Apr 2004
Posts: 871 |
Wow. Very nice. Your tests make painfully clear that ReplaceFile() is not at all doing what we thought it would.. 2) The results are unlikely to be applicable to a BSOD or power failure situation, where OS-level caching and drive-level caching are involved, eg. some drives tell the OS that the data has been committed to disk when it is actually still in the drive cache. My guess is still that atomic overwriting renames (ie MoveFileEx()) will already help a lot, even if this still won't be safe in 100% of the BSOD cases. At least on *NIX, the rule is to call fsync() on the new file before rename()'ing it, in order to prevent that the file ends up being renamed before its data have made it to disk (thus resulting in an empty/corrupted file after an untimely system crash). The Windows equivalent of fsync() appears to be FlushFileBuffers(). I don't know enough about how NTFS journaling works to say whether it helps to call FlushFileBuffers() before the CloseHandle() + MoveFileEx(). In any case, FlushFileBuffers() is probably the last and only thing you can add to improve the BSOD situation further from the application side. If it doesn't have too much of a performance penalty, it might be worth adding just in case. Edit: just to make sure, FlushFileBuffers() would always be more of an improvement than MOVEFILE_WRITE_THROUGH, as the concern here is about the consistency of the update, not its durability. That is: it is important that mirc.ini is always in a good shape on disk; it is not so important when the whole mirc.ini update actually makes it to disk. 3) Mode2 showed almost no file loss. I'm surprised and slightly concerned that with MoveFileEx() there was still one loss case at all though. With only the application crashing, there should be no way that the file gets lost. Do you have any more details about what happened there? Was the file actually lost, empty, corrupted..?
Saturn, QuakeNet staff
|
|
|
|
|
Joined: Dec 2002
Posts: 5,493
Hoopy frood
|
|
Hoopy frood
Joined: Dec 2002
Posts: 5,493 |
FlushFileBuffers() would always be more of an improvement than MOVEFILE_WRITE_THROUGH, as the concern here is about the consistency of the update The only issue I am concerned about is that, as mIRC is single-threaded, write-through/flushed updates to slower, non-SSD drives may freeze the interface every time there is an update. That said, it looks like just using MoveFileEx() without write-through will help. In fact, even just using mode3 will help - it was probably the issue with the pending delete that was having the most effect. I'm surprised and slightly concerned that with MoveFileEx() there was still one loss case at all though. With only the application crashing, there should be no way that the file gets lost. Do you have any more details about what happened there? Was the file actually lost, empty, corrupted..? Unfortunately I have no more details about this event. After that specific run, the perl script either did not find an mirc.ini or the mirc.ini size was incorrect. As it is such an outlier, it is possible that something else happened and that this was not due to MoveFileEx().
|
|
|
|
|
Joined: Apr 2004
Posts: 871
Hoopy frood
|
|
Hoopy frood
Joined: Apr 2004
Posts: 871 |
The only issue I am concerned about is that, as mIRC is single-threaded, write-through/flushed updates to slower, non-SSD drives may freeze the interface every time there is an update. That might indeed be a problem. Oh well; if it turns out that MoveFileEx() still does not solve the BSOD problems for many users, it could be a next thing to try. Unfortunately I have no more details about this event. Alright. Thanks!
Saturn, QuakeNet staff
|
|
|
|
|
Joined: Nov 2004
Posts: 842
Hoopy frood
|
|
OP
Hoopy frood
Joined: Nov 2004
Posts: 842 |
I should probably weigh in here and say my mIRC is installed to the mechanical hard drive on this laptop rather than the SSD (I use the SSD as little as possible), and that the settings are within the mIRC folder itself rather than using AppData, 'cause I personally found it annoying to access stuff.
Don't know if this information will have any impact on the tests you conducted, though.
Still, the mode2 result looks very promising.
What do you do at the end of the world? Are you busy? Will you save us?
|
|
|
|
|
|
|
|
|
