You could also try turning off the switchbar and treebar to see if that makes a difference. The switchbar/treebar are probably the most resource intensive features because they are graphics-based and need to be updated repeatedly in different contexts, eg. name coloring based on events, blinking, custom window ordering, etc.
Most graphics-related features, including display text in windows, have been updated over the years to use bitmap caching which, depending on your CPU/GPU, could also be resource intensive.
Other features that can be resource intensive are the "Nick Colors" feature, eg. if you are on many channels, with many users, and you have asked the Nick List feature to color nicks based on idle time, that could mean that tens of thousands of nicks need to have their individual idle times monitored/updated.
While most features have been optimized to use hash tables, in some cases it is not possible to use these, eg. where wildcard/regex matches are used, and so on. A feature like the IAL, which stores/updates details on potentially tens of thousands of nicknames if you have many channels open, could possibly cause a slow down.
As maroon mentioned, you would probably need to experiment with disabling features and joining a smaller number of channels, with fewer users, to see where the slow down starts. That might help narrow it down.