There are a few things that can cause this
Are you certain that you CA'd the hinges well? There are a few ways of ensuring this - Drilling a hole in the hinge slot to allow the CA to wick in properly - I also drill a hole half way through the surface to the hinge area and put CA in there as well.

Is there any slop at all in the control linkages or servos used?

How fast is not very fast? Always use really fine props on such models (Eg 4" pitch or less on a 2 stroke).

Any slop at all in the controls will allow flutter to happen (Linkages, Hinges, Servos).