No, the car does not have to be running, and we're looking for volts. This will show what, if any, voltage is being dropped across the cables. It would be helpful to measure the battery voltage (positive to negative) along with it.