Controller Buttons getting stuck

I have flame dash bound to Square (on a ps4 controller; I believe it's X on an xbox controller), and quite frequently that button is getting 'stuck' in-game. So my flame dash will constantly be used if any charges are available. If I exit the game entirely and restart it, then initially this bug goes away.

This bug will often occur when I am alt-tabbing (ie: upon tabbing back into the game, the button will be stuck), but not always.

I have only noticed this happening with the Square (ps4) button.