Thanks for the help.... I crawled under the car and looked and there are 3 holes in the crossmember for the bolts, but only 2 of them have bolts in them. The one that is empty is the one closest to the center of the car. It doesn't look the mount extends out to that hole, so I don't think it would matter if there was a bolt in there or not. The other two bolts were on there very tight (I couldn't tighten them anymore at all).
While under..look at the big block of rubber again. Does it appear to be pulling away on the top or bottom.?
I'm guessing you are talking about the piece of rubber held down by the pin. I checked both sides of it. The top didn't seem to be scarred and the bottom didn't either. The pin was loose enough though that I think I could have pushed the rubber so it was flipped around the other direction if I wished.
Is there something else with this mount that I can check? What else could cause the rougher shifting?