The new alternator may have gone bad. It sounds as if you didn't charge the battery before starting it with the new alternator. A dead battery can kill a new alternator. A battery that is almost dead can still read around 12 volts. It just can't put out the current that is needed. Checking all connections in the charging system is a good idea when replacing the alternator
Edit - check the voltage at the battery posts with the engine running. If the alternator is working, it should be between 14 and 15 volts.