Unless the difference is substantial you are probably worrying unnecessarily - especially if you have a rudder. After all the full size could fly (just) on one.
Where you would be in trouble (again just like full size) is if one actually failed just after take off!

My EE Canberra with cheap EDFs and ESCs has no problem.

And its just 'bank and yank' (no rudder)!