Turn the plane off BEFORE the transmitter. The servos act crazy like that because the receiver has no signal to read with the transmitter off. You will fry your servos definitely and probably your receiver as well.

P.S: Turning them on, you turn the transmitter on FIRST, then the airplane.

In short: NEVER leave the plane ON with the transmitter OFF.