It happens with a select set of people "you know", which means it's likely a small sample size, and not random.
Which means it's more plausible that:
You, or they, are just having connection issues. Perhaps your route to that specific set of hosts is unreliable.
Or, perhaps, one of you is on a wifi network and it's cutting out constantly...