Spirit Points missing

In the King in the Mists part, I didn't get all the spirit points that the game provides, and this is hindering my build. I've done the quest more than twice, the first time in a duo, and then several other times alone, but it still hasn't coun