To make it simple, it will be just like training a student on the buddy box.

Pick the guy who has the master control carefully. If he has the right skill set he'll know when you're getting close to the limits of his comfort zone and rescue the aircraft before the situation develops beyond his control.

That is the difference.

Most will agree that in control means the plane is going where intended somewhat smoothly and the next turn or two is thought out ahead of time.

You're right, many people can't fly this way looking at the plane from a third person point of view...they need focused practice before they start getting on the end of a master control for someone else.

When the guy wearing goggles will not know he needs assistance until the situation has developed beyond his ability to recover.

Flight training. Simple. Yes, with some outside references (a fence, trees or other land marks) and some flight training, it is definitely possible to fly FPV unassisted. Make that LOTS of flight training for most people. Now you're flying a plane just like being in the real thing, except that it is harder because you can't feel the flight loads on the airframe.