At these sizes, metal gears have more to do than only the torque of the servo. If your 120 oz servo has metal gears, you should be fine. If not, you will have problems, especially on ailerons, where vibration kills the most.

It also depends on the throw of the control surfaces. The less throw, the less torque required...