During a harrier or elevator forward velocity is very slow if at all, and the ailerons are only used to keep the airframe level. Since the amount of roll couple is proportionate to the amount of side loading, at no velocity there is no side load, however large amounts of rudder are needed to maintain directional control. You really don't want the aileron input when upright at minimum airspeed. And if elevator mix is used it will make it more difficult to hover as every yaw input will induce a pitch change, at least thats what my two remaining brain cells think. :^)