I actually talked to my GF about this, and my roommate confirmed it with his GF:
At first appearance, women are mostly turned off by it. For example, if your on spring break, on the beach with no shirt on, the female'* will most likely go after the guy with minimal hair than the guy with a hairy chest (even if kept cleaned trimmed). However, if the girl does not know that you have a hairy chest (say you met in class/work/whatever the situation where you have your shirt on), and she likes you, they really don't seem to mind.
That being said, it'* really a personal preference. I trim mine, so it doesn't go out of control. My roommate has to use a lawnmower on his chest and back, and even when it is about due for a trim, he GF doesn't mind whether he does or not, same holds true with my GF.
It'* really more of the fact that guy'* are a bit more shallow than female'* when it comes to appearance.
2000 Bonneville SE