One of the reasons you need holding power more so on the rudder is the fact that the rudder is neutrally loaded most of the time. With the ailerons, flaps, and elevators there are aero loads trying to displace the surface under normal fight conditions with the servo resisting that displacement. The rudder however is just trailing in the airstream most of the time and any slop in the linkage, gears, mounting, or deadband will induce flutter well below the flutter speed of the plane. As I am sure all of you know every plane (model and full scale) has a flutter speed above what is usually called out as the VNE by the manufacture.