Who's this "we"?
Never have balanced a rudder.
If the surface (not limited to rudders) can bind against what is supporting it, then what happens to it can affect the plane.
Every one that's come loose on the planes I fly has faired itself. It's a rare event in any place.
A sufficiently heavy surface.. rudder, elevator, aileron, -can- go into flutter and affect the rest of the plane, but mostly it will fair and exert no influence on the plane.
Surface flutter is frequently created by an overly heavy surface, restrained or not.
.
Witness some early pre-WWI biplanes with the ailerons dangling when the plane is sitting still on the ground.
As speed increases during the takeoff run, the aileron will rise, attempting to fair itself. The control cable, which pulls the surface down only, restrains the upward motion at some designed angle.. then the surface acts as an aileron is expected to.
.
Flying contact combat on the slope, losing the control horn on an aileron merely inactivates that surface. As long as the aileron on the other side was still operational the combat can continue.
.
Losing an elevator horn, or having the pushrod snap, is not a benign situation though. It's as if half the horizontal vanishes, and the plane seeks a new trim position, which is generally some feet underground.
.
This one lost the entire right side horizontal in flight.. the pushrod was on the left half. The plane was landed without incident