I bet you will find a lot of builders agree with you. I like the aliphatics (yellow glue) myself for the reasons you state. If I'm in a hurry, I'll use CA, but I think the carpenter's glue gives a tougher structure overall.

Mark