Fiberglass is too brittle, too heavy, and too much trouble. You want the wing of a combat plane to be able to flex a bit to absorb impact. Fiberglass covered foam won't flex, instead the glass cloth will buckle and pull off the foam.

No forumula for winglets, really, just make sure they extend far enough up and down, and go all the way to the TE or slightly beyond. Personally, I'd suggest not thinking of them as part of your rudder area. They will add some, but not that much.

If you really wanted, you could calculate the tail volume added by the winglets. Just calculate it the same way you would for the vertical stab. Find where the average area is, measure the distance back from the 25% MAC point of the wing, and multiply.

But keep in mind that the actual aerodynamics are very complicated, so you're best off just doing cut-and-try and find something you like.