Hello,
Sorry for my bad english, it's not my native language.
I did a very rough way. You can check if there are more nicknames in a channel than the height of the window / vertical spacing.
For example, I'm using the font Verdana, size 13. The vertical spacing is 17 (so if the window height is 170, there are 10 lines).
So just edit the 17 according to your font size.
Code:
alias chscrollbar {
  if ($nick(#,0) >  $calc($window(#).dh / 17)) { return $true }
  else { return $false }
}

I hope you understand what I mean.
~cheers

EDIT: I don't know if there is a native way (or a better way) to do that, I tested this and looks like it works