PoE2 - Not enough Spirit, but i have enougfh spirit.

I have had multiple instances today where my Raging Spirits has deactivated and it took me a bit to find it. At random instances, my skeletal warriors are costing more spirit based on the number summoned. I have 3 summoned, so some spirit is used, however when I notice my Raging Spirits are no longer summoning, I reduce all minions to 0 (deactivate Raging Spirits as well) except my warriors, which I reduce to 2 and find that I am missing spirit from my spirit pool. When I reduce my warrior count to 0, the missing spirit is refunded into my spirit pool and I can then summon back all of my minions based on my previous count. This has occurred 3 times today during Act 3.

Edit: When my warriors die and respawn is what is triggering the issue. The summoned warriors now count more against the spirit pool. The initial 2 warriors count as 22 spirit, and 3 warriors count as 44 spirit.