Thanks to everybody for all their input on this. Its been a real eye opener.
I've been searching t'internet for some definitve word on this subject and found a very interesting treatise on this subject on the UK Micromagic class association board of all places:
My interpretation of the theory discussed is that fin, rudder and bows should be as smooth and shiny as possible but the rest of the hull is at least as fast just wet sanded in the direction of the water flow. The amount of the bows which should be shiny smooth reduces as the boat speed increases and also as the surface conditions deteriorate. So, if you sail in light winds on a glassy smooth pond you probably want to polish the first 25-50% (never more) of the hull and sand the back section. As wind speed and chop increase that percentage decreases pretty quickly.
In practice, that seems to support the "Don't polish, sand" camp.
PS It also states that you should NEVER sand a gelcoat!