Avatar of fire + occulist noteables?

What about maybe "Improving" avatar of fire to include chaos damage? Or would that be unbalanced in some way?

Conversion of dmg dealt only works in one way phys -> lightning -> cold -> fire -> chaos.
So chaos cant be converted to fire.

AoF makes you deal only fire damage.
That is the point. You can stack resistances against reflect, all damage you deal is scaled by the same nodes, element-specific leech works how you'd expect, etc.
It is not meant to work with chaos conversions, that is the drawback it has for the bonus it grants.
It's like asking "Couldn't I keep my life when going CI?".
So, no, it won't happen.
You'll have to use items or supports if you want all fire without AoF.