Total Attack Time paradox

Why is it that the skills that have +fixed total attack time do not benefit from buffs, but can still be debuffed? I believe it’s already ridiculous that skills like Rolling Slam already have 1.5s vulnerability and zero damage window you can do nothing about, but being slowed down further is a gut punch.