Not really. The biggest problem is loss of effectiveness. You have to go back to how planes fly. Air flowing over the wing causes a pressure difference between the two sides of the surface, thus creating lift. When air flows through the gaps it changes this pressure differences which effects the flight of a plane.

I guess over a long time it could possibly cause problems and loosen the hinge up. But I have several planes 8+ years old that don't have the gaps sealed and the control surfaces are still solidly connected. The biggest problem I have observed is that it decreases the performance of the plane.

Ken