Yes, look at the B2 bomber.

The vertical is used as a passive method of keeping the nose going where you want it to. Verticals are cheap, easy to install and work with no extra fuel, plumbing or electronics - a pretty good device.

On something like the B2 the use of drag flaps on the wings and an active autopilot keeps the airplane stable or at least it thinks it is stable. You can use jets of air, rockets, etc. to accomplish the same thing. All you have to do is have something that will create a force of the right magnitude and direction to keep the fuselage pointing in the direction that you want it to go.