It looks like both browsers have Tab settings in their Options dialogs that determine whether links are opened in tabs or new windows. I have not been able to find a way of overriding these settings with DDE, which is the standard method for opening links.

Firefox does support command line options -new-tab and -new-window (not sure which version they started supporting these - they may only work for recent versions of Firefox) but if I used these they would only work for Firefox and not for any other browser.