Strange, very strange. I use 5645's on flaps only, and with the speed dailed down to 50%. They do seem to run a little rough, but I wouldn't expect what you experienced either.
Give it another try with any servo's you have that you know are good. You really don't need much torque at all on ailerons.