Sierra, is your rudder servo mounted in the tail or inside the fuse? If it is inside the fuse, I would wonder if it is the control rod flexing under load.

I had the Venus II with the same engine; the plane should be able to hold knife edge all day and all night. I had the JR DS821 for all servos on a 6 volt power system - that's 88 oz-in of torque. It was enough to fly the plane though I'm not sure what would happen with less power on the rudder. The Futaba S3010 is the standard, hi-torque, ball bearing servo and has 72/90 oz-in depending on battery.

What are you using for a prop?