Can't spend Atlas passive points

I have four passive points by now that I can't spend. The blinking icon sits there at the bottom of my screen and taunts me. First I thought the issue would maybe solve itself once I picked up the next two points, but instead the problem has doubled. A patch/fix for this would be great as I don't wanna keep putting time into mapping if I can't improve my odds of good loot at all.