Lost all inventory and xp 7 times today

I am in an area called venom caves in act 3. When I use my "move minion to cursor" skill the game freezes and takes me back to the login screen. When I log back in my game is rolled back. I am either in an outpost or starting over in venom caves. And all my earned xp and inventory are gone.

I forgot to mention I was using a side button on my mouse to tell minions where to go. I haven't crashed since I stopped using the mechanic of telling minions where to go. But it happened every single time yesterday.

I did not press the "direct minions" key in all this time until just now and it crashed my game again. I lost my map and it rolled back any xp or items I got while in that portal session.

This is seriously messed up.

I decided to test it out in another map. It happened again, but it might roll back a few minutes because I kept 2 of the items this time. But the map was lost after the crash from hitting 'direct minions' key. I switched the key a few times. It crashed while on the shoulder button this time.\

edit: Since I don't want to keep losing maps and currency and xp I am not going to hit the direct minion key for a while.