For DCC's to work a direct connection must be established between the two IRC clients involved (hence "Direct Client-to-Client"). Many firewalls, routers and modems block incoming connections.
By default the onus is on the initiator of the DCC to have a setup which allows incoming connections to reach their client, but with passive DCC this is reversed (the receiver, not the initiator, must have a correct setup).
To know how to fix your end you must first determine how you connect to the internet (e.g. which models of modems/routers you use) and what, if any, firewall/security software you are running.
For modem/router configuration visit portforward.com
For the Windows XP Firewall (pre-SP2) follow this
For the Windows XP Firewall (SP2) follow this