Plague Swarm respawning infinitely

The little bug minions called plague swarm in act 2 in one case kept spawning. No matter how many i killed. It was rather frustating as it meant i couldnt pick up drops with a controller, and i couldnt TP out either. Just left the game and have to wait for the instance to recycle.