Steam Remote Play not working properly

I have been playing with very good performance on my PC but wanted to try playing with my PC streaming to my steam deck in the living room like I do with many other games (like Metaphor: ReFantazio).

However, as soon as I start the game on remote play the performance gets awful until the game eventually crashes. I tried the performance with other games and they are all working fine, including PoE 1.

Thank you!