Yes, you can cover with a paper bag.

To do it, you would wet the paper with a solution of white glue or yellow glue thinned with an equal amount of water. Brush it on and let it soak in, making sure that your paper is wet everywhere before applying it to the wing.

To get a good smooth finish, I use plastic gift wrap from the dollar store (only costs a buck a roll -- obviously), wrapped over the paper... so you'd lay out the gift wrap (a piece a little longer than a wing panel and wide enough to wrap over the wing leading edge and cover top and bottom surfaces with a little extra at the trailing edge. The paper gets laid out on that and wetted all over (not drippy wet, just wet through), then that all gets smoothed onto the wing starting at the leding edge and smoothing back with a credit card edgte to get out all the air. Smooth out the underside first, lay it in the bottom cradle half left from the foam cutting, then smooth out the upper surface the same way. Put the upper cradle half on top and add a bunch of weight spread out evenly. Let it dry overnight. It gets quite stiff and does add strength.

You can decorate it with wall paint if you like. If your plane will use an engine, seal it with a clear paint or sealer.

Depending on just how much strength you need/want, you can also add spars or other reinforcements before doing the paper covering. What kind of plane is this you're working on?