No, I wouldn't say you are crazy not to use an arming switch. I have only recently experimented with one after years and don't really like it. I have throttle kill on my Tx that I use religously. It is checked before the batteries are plugged in and isn't switched until the plane is on the runway. It is flipped again when the wheels stop on landing. I do love a quick release canopy for the convenience of not having a ball driver in my pocket at all times.

I have seen a friends arming switch fail cutting off the main batteries. Luckily, since he was using a Ubec, catastrophe was avoided because of his back up system.