In looking what is happening it almost seems that maybe the linkage is slipping? How is the servo connected to the control rods? I have had issues with the control rod connectors that have a set screw that would seem tight but would slip under load. What I suspect maybe happening is that the connections have enough friction to hold while the lower ailerons are connected but when you add the upper ailerons the hinges are too stiff and the control rods are slipping.

In any case it is either binding or something is slipping. Check your control rod connections and the movement of the ailerons it has to be in there somewhere.