I drove through a small puddle, and my lights dimmed and the A/C got warmer for a few seconds before going back to normal. No big deal, the belt must have gotten wet and slipped.

I don't know how old this belt is, so I'll replace it. But refresh my memory: if the belt slips when it gets wet, should I be worried about the tensioner too?

No, I've had this happen several times before with my cars. Its just the simple fact of the belt getting wet and it can't grip the metal pulleys.

Most of the big box autopart store sell a product called Drive Belt Dressing and Conditioner.

It eliminates squeaks and prolongs belt life. For use on all types of belts, such as serpentine, flat and v-type belts. Prevents slippage and glazing. Removes oil, grease and glaze. Use every 3,000 miles to prevent drying out and slippage.

Might be a short term solution to your belt slipping when wet.