I just saw something similar while helping a friend setup a new model.

In his case he had a flaperon mix and the flaps still active and it over drove the servo and caused the "jitter".

Once the flap function was turned off all was normal again.