POE performance issues and GPU crashes

I've been struggling with POE having performance issues for a while now.
Last season I attributed it to my Intel I7 13000K dying, but since switching it out, I still have issues.

Not only does the game struggle to keep up with gameplay, regularly dropping to below 60 FPS even with Dynamic culling and whatnot on,
I also have the game crash regularly, most of the time with errors similar to this one:
MAP: The GPU device instance has been suspended. Use GetDeviceRemovedReason to dertermine the appropriate action. (GetDeviceRemovedReason: The GPU Instance has been suspended. Use GetDeviceRemovedReason to dertermine the appropriate action.)

Not to mention random crashes in town, but whatever.

I've tried fucking with all the graphic settings, but nothing seems to be a permanent fix.
I currently have most things back on "default"
Direct X 12 limits my FPS to like 57 to begin with (?).

My PC specs:
CPU: AMD Ryzen 7900X3D
GPU: Nvidea RTX 4070
Motherboard: Gigabyte B650 Eagle AX
RAM: D532GB 6000-36 Corsair Vengeance

I would love to try anything you can throw at this.