No, the core cover doesn't have to come off. But you may have a problem with the length of the airmix actuator harness. Plan on using the old harness to lengthen the new one to make this procedure easier for reassembly.
Retract and recline the seat fully. You may find it easier to recline upside down underneath to do this. I alternate between that and kneeling in the open door.
There'* another electrical connector above one of the two actuator screws (farthest to the passenger side) that you'll have to blindly remove to gain enough clearance for a short screwdriver bit.
This works very well up in there:
3-piece finger grip socket adapter from Sears.