I'm betting on the switch. Intermittent lose of control can be so many things it's mind boggling. It could be a heat problem relating to the battery, switch or receiver, Vibrations at a specific throttle setting can cause all kinds of stuff. You were inverted so there is the possibilty of the antenna touching metal, a connector working loose when inverted.

Now as far as a 600MaH battery is concerned. Without load testing there is really no good way to tell. The Voltwatch does a decent job even when moving control surfaces but it's not accurate. A standing voltage of 5.0 volts with no load could conceiebly drop below 4.8 when under the stress of flight. Most recievers will drop signal around 4.7 - 4.6 and once that happens momentary to full loss of control is a real possibiltiy. A battery that has never been cycled or just plane old can also drop quickly at a certain voltage.

Personally I never fly below 5.0V on a 4.8 battery no matter what servo's are used. The higher the torque the more the load, the bigger the surface, the more the load.