Sigil of Power gains stages with infernal flame from triggered skills

I am using Sigil of Power with Pyromantic Pact allocated, and it is gaining stages when I trigger Cast on Minion Death. It is also getting to 4 stages instantly, even though I am gaining way less than 50% of my infernal flame pool. Manually casting spells does not give any stages.