Unable to Edit hideout decorations in POE and POE 2

I can enter Edit mode via the button. I open the library of decorations. I can bring items out and place them. However, I cannot remove utility items. I have 4 alternate Map Devices, for example, and I cannot select the one in use and remove/reclaim it by pressing Delete. Nothing happens.
This is behavior in both games.