Actually the servo arms are offset to compensate for the fact that you get varying line tension if the servo arm is straight and it's connected to the control surface that is NOT straight (See the attached image, which WOULD give undesirable varying cable tension).

If you could set up the control surface so that it too is straight, then unequal length arms between the servo and the control surface would not matter, because the ratio or distance one side moves is exactly matched by the ratio/distance the other side moves, and cable tension stays constant.

To minimize this varying line tension, you can (1) use an offset servo arm to match the amount of offset at the control surface between the hinge/pivot line and a line drawn between both clevis holes or (2) offset the control surface arms to so the clevis holes are at or near the hinge/pivot point, thereby matching the straight servo arm.

Actually, it isn't absolutely critical that both lines stay absolutely tight. What IS critical is to avoid the cables becoming tighter when the surface is moved from neutral to extreme travel. It's fine (actually a bit desirable) for the cable to loosen slightly at extreme travel. Air pressure from flight loads will keep the loaded (pulling) side tight, and since one side of the surface is under a load, flutter isn't an issue.