Does this happen when the servo is on the bench, or only when it's installed in the plane and trying to move the control surface? If it's only when moving the control surface, check the control rods for binding, this can slow the servos down.