I'm about to paint mine too, they're a bit rough and lost the paint. But I can do a better job than my dealership'* body shop at least. They've left some orangepeel on my fenders that I've had to sand out and refinish, left unremovable scuffs in the bumper, etc.
I'd sand the hell out of it depending how rough yours are, and prime and paint it. Just make sure you wet sand between base and clearcoat, and some rubbing compound to finish it up. If you even do an 'ok' job, it probably won't be too noticeable if it isn't perfectly blended...And use the claybar for the other not so bad parts