one thing you can and SHOULD always do. Remove the linkage from the servo arm and remove the servo arm from the servo. Put the pushrod as closely as possible to where it would be if it were hooked up to the servo. Move the pushrod back and forth. If it tries to move out of the plane that you're moving it in, then there is a problem.
Also, if it is difficult to move or "sticky" then there is a problem.
The bottom line is you have to look at the pushrod from the servo's point of view.