Azmerian Wisps Frequently Spawn Next to Rares

Maybe I'm insanely unlucky, but it feels that 75% of the time I encounter a wisp, it's 5 feet from a rare, so it immediately possesses the rare and the encounter is over. Pretty frustrating.

Oh, and I'm sure you know this, but the music for the encounter constantly bugs out, often looping for the remainder of the map. Delwyn's appearances seem buggy too. He's either not there at all, or he spawns by himself with no wisp, or he talks as if the encounter isn't over when it is.