As a private pilot, and have had (about 7 years ago) about 5 years of RC A/C experience I personally believe that the physical act of flying an RC A/C is more difficult than a 'full scale' aircraft. Having said that, everything else is much more difficult in a full scale:

1) Decision Making
2) Navigation
3) Pre Flight
4) Radio Work
5) Emergency Procedures
...

The list goes on and on . . .

Regards,
Mark F. Sanderson
PP/VFR C172, C15X, Piper Warrior, Tiger (Tin Cans!)