They are not meant to peel.
In bad cases I have successfully peeled them and replaced them with a car wrap type of protective wrap.
In even worse cases I glued a coca-cola or beercap on top of the XT-cap.
And lastly, as a last resort, I made new Aluminium covers on my lathe.
Pics for inspiration: