Time wise my first hand knowledge/experience is very similar to yours and even though in most cases I would agree with the lighter air frame proponent you explain I have also learned that what ever the rule in this hobby there are always going to be exceptions and compromises that might be taken to accomplish a specific goal.
http://f3j.in.ua/ballasts.html
https://www.soaringusa.com/Ballast/
http://www.espritmodel.com/sailplane...-brass-14.aspx
Many many reputable companies have ballast kits available.
Dennis