Vengeance Procs With the Off Hand Weapon When Dual Wielding

All attacks alternate hands while dual wielding unless they explicitly diverge from this standard behaviour. Vengeance does not.
Because of the nature of triggered attacks being extra actions that happen while you're performing other actions, the alternating behaviour, which is based on actions performed, will be essentially arbitrary - it will effectively pick a random hand for each counterattack.