If it's moving it is because the servo arm is "torquing" or twisting as it tries to push/pull the pushrod. The pushrod is resisting the "command" from the servo. There is resistance in the control somewhere. A bind. It may not be stuck but stiff.

The other point is How much is it moving? A little movement is normal because of the grommets and stuff.