I think I've got it. (if only I had paid attention in that vector dynamics class I took about 20 years ago). It appears to me, after reading several very informative replys to my question, that I have set myself up in the worst way possible; very short control horn on the control surface and relatively long servo arm.
I already have my order in for replacment, longer control arm for the control surface and when those come in I will be moving the clevis on the servo inboard toward the drive output of the servo. The lugwrench analogy was very insightful as were the drawings.
I'll post the results as soon as I can try out the recommendations