Crossbow doesn't shoot after ammo switch sometimes

I've a similar issue with it. Sometimes after weapon switch or attacks the bolt number of the active crossbow skill stays at 0 - and then I need to press the key again for selecting the ammo to reload the bolt.

I've found one of the issues and it's related to weapon swapping:

Here's a way to reproduce (you can use many skills in place of these, just using as an example):

Steps to reproduce:

1) (un)lock crossbow to weapon set 1 and 2
2) assign permafrost bolts to weapon set 1
3) assign explosive grenade to weapon set 2
4) exhuast ammo with permafrost bolts; before the auto reload animation, use explosive grenade
5) try to use permafrost bolts. Because the ammo is exhausted, and you're currently on weapon set 2 (from using grenade), it will not automatically switch back to weapon set 1 and auto reload
6) when manually reloading with 'load permafrost bolts' ability, it will reload twice (and take twice as long)