I'm in the same camp, chances of a problem, low but why even risk it?
Besides the tiny risk, it's much easier to have a pile of batteries ready to go and swap out. I keep the battery charger on the bench while I'm working, having a tank sitting there would be a hassle. In my house, I could see a wire kicked, pulling the whole thing with it.
On large turret Tamiyas, you can slip the battery out of the opening with no fuss, others I usually hinge the back and build a flip up support to hold the hull open for access. The battery should always be disconnected when not in use, pulling it out shouldn't be any more difficult.