Cannot poison while in party

I am playing a Herald of Agony build using a 100% chance to poison Ball Lightning.

I've been playing this build the entire league so far and am level 73.

I just partied with a friend and found that I am unable to poison enemies while in the same area as them - yet the moment they leave I can poison again. Not poisoning means no summoning the agony crawler.