I guess it wont hurt to sand as long as you don't take off too much.
I did not sand every piece, built with wood glue and have pulled seveal "full-speed-full-elevator and or aileron" with no failures.
Personally I don't think it's important to clean every piece since everythin is interlocking.
Those glue joints, sanded balsa or not, will be stronger than the balsa theyr'e attached to anyway.