Kevin,

Torque is a measure of rotational force and is the product of a force and a distance (ounce inches) . The amount of force (ounces) you get out of your servo is dependent on the length of the lever arm but the torque doesn't change. Nothing comes for free though. If you use an inner hole you get more force but over less linear distance.

You should try to use the longest servo arm you can with an appropriately sized horn at the control surface to deliver the required amount of throw while using 100% of your servo throw. Less slop and better resolution.