Help Improving My Phantasmal Vaal Summon Skeletons Build

I made my first build this league and was looking on some advice on how to take it to the next level.

Link to POB: https://pastebin.com/YR11GbE9

Or check out my character named BowsAndBows
(I Meant to name it BonesAndBows but I had a stroke or something when I was typing)

Note the POB really hates the fact that im using skeleton archers and wont properly calcuate my damage which makes it much harder to optimize the build. It should be around 2 mil shaper dps without the vaal skeletons and around 5 mil when they are up.

The build is built around the alt quality summon skeleton gems that summons skeleton archers instead and scaling their physical damage.
The two specters I have are apes that give frenzy charges to allies.
Overall for being my first build it went pretty well as i managed to kill uber elder with only spending about 15ex on the build.

Despite that the build has some issues that i would like help addressing.

1. Issues clearing juiced maps.
The clear actually pretty good on Rare t16 maps when the enemies don't have insane hp but when the second delirium gets onto the map the skeletons lack the damage to efficiently clear the high hp monsters in a pack. The single target dps of the build isn't bad at all but the problem is when in maps the skeletons can't effectively focus the high hp targets.

My first step would be getting awakened minion damage and some better abyss jewels but im not sure what my next best upgrades to dps would be. (I would go for a lvl 21 gem but a 21 phantasmal vaal summon skeletons basically doesn't exist since no one is double corrupting them).

If I could find some way to get corpse explosions for my minions that would be ideal but im not sure if there even is a way for me to do that.

2. My defenses aren't as great as i would like:
I know I should get a non-scuffed two stone ring that can fit a 3rd +min endurance charges but im not really sure about the best way to scale my block or if i should even be investing into block instead of some other defensive layer.

You got limits, changing things will make another build.
Better you scrap it and focus on something else now.
For example swap ascendency to Elementalist and swap skeleton for Carrion golems.
Also that 4L for immortal call use just one endurance charge... Enduring cry do more for heal then rest of it.

If you like really really wanna do something then make use of necromantic aegis or get that The Iron Mass uniq sword.
Changing things give you more experience in how effective you can get.