I think that's a really nice idea.
Also it would decrease the whining about the colour bias of the sockets a bit. XD
Often you can do many things with the sockets you have, even if you have to use quite some orbs for them. However, sometimes you could socket it a bit different and more efficient if only this one socket would be red. Well, only the one socket isn't exactly an option with chromatic orbs.
Such a system could just get rid of this problem.
Also it would allow some new type of perfect gear that you can craft, or waste all your orbs trying to do so.
Imagine a high level Armour;Energy Shield helmet with 4 connected prismatic sockets.
You will never have to use an orb of chromatic on this thing again, no matter which character will use it.
I guess such items would be some of the most valuable in the game, even in a white state.