It sounds like something is catching when you move it one way. Disconnect the clevises at the rudder end and move the rudder back and forth to see if that is the source. If not, get a helper to move the rudder stick while you hold the two clevises to see if you can feel some binding. If that doesn't work, reattach the clevises and remove the ones on the servo and try moving the rudder with them. Also move the servo back and forth to see if it's not centering correctly. One of these tests might help you find the source of the problem.

Something else that comes to mind is do you have the cables running parallel, or crossed? If they are crossed, they could be catching on each other. If they are parallel, perhaps one is catching where it exits the fuselage. It's ok to cross the cables as long as they are coated, or if you've tilted the servo enough to prevent the cables from rubbing each other.