I had this same problem on a Jeep with drum brakes in the rear. Turns out the rear wheel cylinders were clogged up inside. They weren't leaking, but they were clogged up with junk. I would replace them anyways.
How did I find this out? I found a dealer who had a Hunter B400 brake tester. This thing is really incredible. A very very expensive machine, but well worth the cost of the test. It will tell you how your brakes are working and produce a report for each individual wheel. In my case the front were doing 90% of the braking, with the rears doing 10%. Not good.
On Hunters website you can find a local dealer that has a B400. In my case I had to drive 80 miles to get there, but it was well worth it as I was at a loss as to what to do next.
Here is a link to Hunters B400.
'02 Aurora 3.5, factory NAV retrofit, CD changer retrofit, Philips Xenon lights. '96 SE: Gone but not forgotten :( Oh yeah, and a few other toys.