network issues on different maps

A lot of times there are some zones with high ping spikes.

For example my limestone hideout is having high ping spikes (500ms at max)
Then when i switch to a map from atlas the ping is different.

I tried to switch servers and it didn't help.

My internet connection is fine, and I'm using direct ethernet connection to the modem.

Honestly, I don't see why hideout or any map where I run alone should have such high pings.