I use 2-56 on everything except the aileron to aileron link. I keep all unsupported control rods to a bare minimum length to prevent buckling. I see no reason to go 4-40 unless you have a buckling problem (like the aileron link I already mentioned). A 4-40 rod can easily pull a servo completely off its rails (unnecessary overkill in my opinion).

I have close to 100 flights on a CGM Sukhoi with all 2-56 controls. I have hardly any wear, much less a failure. It's your choice though.