I haven't worked with dialogs for a very long time, but perhaps something along the lines of:

Code:
if ($dialog(name).x) { commands }

Also, it may help to know that the mIRC Help file states:

You can also use the $dialog(name/N) identifier to list any open dialogs, where N returns the Nth open dialog. If N is zero, the total number of open dialogs is returned.