Sounds like the blower motor is going bad. Pretty easy to change it out. Replacement will cost about $50.
Here is the procedure lifted from this old post for a '95 SE:
Check for bad motor by unplugging connector on motor and setting controls for high speed fan
connect test light across terminals to see if power is getting to the motor
if power is getting to motor, and motor is not working properly (e.g., requires whacking to get it to work), its almost certainly bad. so....
remove strut brace (13 mm x 4 nuts or bolts)
remove engine cover (remove oil fill tube front and unsnap rear)
unclip O2 sensor connector and fuel lines connector from rear bracket (needlenose pliers)
remove rear bracket from rear cylinder head (13 mm x 2 nuts)
remove rubber cooling tube (heat with hair dryer if cold to prevent cracking)
remove bolts attaching motor (7 mm x 5)
remove insulating cover from motor (if your car has one)
gently move fuel lines out of the way and pull motor up and out
carefully remove rubber gasket from old motor; install on new motor
remove foam gasket from electrical connector, install on new motor
reverse to install; adding dielectric grease to electical connector
enjoy hot air if winter or cold air if summer
BTW, following the factory service manual procedure to test for a bad blower motor resulted in a waste of about two hours time. Why start by backprobing the blower relay (which is pretty much inaccessible!) when you can just test for power at the motor connector?