Use the on part event and check $nick($chan,0) perhaps? You would need a timer though since at the time of the part the nick count is not changed. Something like:
on *:PART:#chan:{
.timerpc $+ $ticks -m 1 100 nlcheck $chan
}
alias -l nlcheck {
if ($nick($1,0) == 1) part $1
}
When a nick parts, a 100ms timer is activated that calls the nlcheck alias and parts the chan if there is only 1 user (itself). Add whatever other channels you want it to do this action on.