So there are a couple of options; you are right that the structure at the LE of the rudder is often not strong enough (for me) to trust.

1 - use a connector that extends forward from the bolt line (aft of the hinge line) and makes the PIVOT point right on the hinge line. This is the approach I ususally take, because it is easier for me.
2 - use an offset arm on your servo (boomerang shape) that sets the pivot points back aft of the servo axle the same distance as your pivot points are aft of the hinge line on the rudder. Usually it is easier to make or buy the arm, measure the aft offset, and drill your rudder bolt the same distance aft of the hinge line.

The square geometry you want is equal moment arms (off the longitudinal centerline) for the servo arm and the rudder bolt, and equal offset of the line connecting your pivot points (2 servo pivots and 2 rudder pivots) from their respective axis of rotation (servo axle and rudder hinge respectively)