NPCs missing in Hideout

The 3 NPCs Alva, Helena and Niko keep disappearing from my hideout.
When I look in my Decorations it says I need to complete their quests.
Sometimes they are available again if I log to another character, just relog onto the same character or change my hideout by talking to Helena in the acts. But they are missing about 90%.
I did the Helena quest on all 4 league characters and Niko/Alva on one.


Additional info if it helps:
I have switched hideout a few times this league, trying out different ones from the community. That's how I even noticed Helena starting to go missing.
I have had hard ping spikes (only in PoE) over the last few days, often going to 20k-100k ms. Highest was more than 300k ms (yes, that is 5 minutes) at which point I had to restart the game. Strangely I do not get disconnected when this happens.


This is how it looks ingame: