If you are logging 128 windows and/or have the buffer set high, you will experience lag issues and potential freezing. Consider what happens if you are trying to write data from even half those windows on a regular basis, or if you had for example a 5000 line buffer, then that is 128 * 5000 (640,000) lines to keep in memory. These things are going to slow down the computer and there's not a lot you can do other than to limit logging or reduce the buffer... 1000 is plenty for most people for a buffer, especially with that many windows open at once. Even less may even be advisable for that many windows.
Obviously, there may be other factors involved, but either of these items is likely to be a major factor for you. And, font linking that many channels may also be a slow down as well, though I'm not sure if that's only used when a line is received, or all the time. If it's only when a line is received, then it's not likely an issue unless you're getting constant lines in all of those windows at once.