It looks like you have TRW on there already
You can definitely do it without removing the CA. That castle nut will come off with a wrench. There isn't much room in there to use a socket but a wrench will work just fine. Then just use a pickle fork tool if you have it (I used an old adjustable wrench) and put it under the knucle and pound it so the ball joint shaft starts to pop out. You'll have to use a long pry bar of some kind to pry it out completely.
I removed the CA since they came off easily and I didn't have anything long enough to pry the CA down to remove the shaft from the hole. It was easier to get it out for me, and then drill out the rivets.
Since you don't have the rivets I would think you could just remove that castle nut, remove the three nuts/bolts on the bottom, and the push the CA down to remove the whole b-joint. Reinstall is easy, just torque the 3 nuts on the bottom to 50ft/lb. I didn't use a torque wrench for the castle nuts since it'* impossible. I just tightened the hell out of it with a wrench and adjusted the tightness to get the cotter pin in.