There is a secondary effect of ailerons, it is yaw.

Also you do need the rudder for things other than the 'p' factor. It is true that you do not use them in a jet as much as you do in a tail dragger, but you still have to use them every time you turn!