Sounds like the servo is having problems. One quick way to tell is to hook another servo into the rudder channel and see if it does the same thing. If there are problems with the radio it will do the same thing with another servo plugged into it. Along the same lines, plug the rudder servo into another channel. If it's bad it will do the same thing plugged into another channel.

Ken