Is stacking "rarity of items found" worth it in poe2?

It is worth it, yes. It increase the chance do drop rares/uniques and increase the chance to get higher tier currencies.

I saw video in YouTube and made a gear with 500% rarity. The drops are very good but divine are still remaining rare. The gear is based on the known gloves that give anything killed by others is killed by you instead. And playing with team the others protect you while killing monsters.

I have 201% rarity on character and the difference is quite literally night and day.

I would say most important to least important on character stats are,

1) DPS
2) Rarity
3) movement speed
4) resistances/evasion/ES/life

You need to kill things quick, get stuff from those kills, move outa the way and take hits dead last. Otherwise progression will feel like watching evolution in real time. You die more at first but get way more rewards, just remember to pick up goodies before moving on.