The A/C compressor clutch can make a clicking noise when it engages. If your system is low on refrigerant, that can make the compressor cycle more frequently. To test if this is the problem, unplug the electrical connector to the A/C compressor clutch and see if the sound goes away.
[edit: Sorry, read your post more carefully. If the sound goes away when you turn on the A/C, maybe there is a mechanical problem with the compressor clutch that it is not freewheeling as it should. If you can watch it while someone else starts the car, maybe you will be able to see and hear to confirm the problem.]