I've debated doing this for a long time. Do you get a lot of people flashing you because your headlights are too bright? Does this make a big difference?

I know quite a few people have done this, but I don't want to blind the oncoming traffic.

This is typically on purpose if your headlights are factory or dealer aimed. The drivers side headlight is reflected close to the middle and slightly angled down for a reason - to help out opposing traffic. You may just not have noticed it as much with the dimmer lights.