Liege of the Primordial from Forbidden Flesh/Flame "lost" when switching zones

The title is me already interpretting stuff:
I do play an occultist, using Forbidden Flesh and Flame to get Liege of the Primordial and thus raising Golemcount. When I switch zones, one of the Golems is "desummoned" and needs resummoning, which then works just fine.

e.g.: I have Stone + Chaos golem active and got through a portal of the Mapdevice. In the Map only Stone golem is still alive. I can resummon Chaos Golem.

I also use other means to boost Golemcount, but i was able to reproduce this with only Forbidden Flash/Flame active.