Adrian's,
Sounds to me like you have poorly built ailerons or a bad servo with excessive gear play. check your aileron for stiffness or as a last resort, remove covering and sheet the aileron.

Seal the gaps, even though your hinge line is tight, those are big surfaces to move.

A 12-4 should be perfect for that engine, what kind of rpm's are you getting?