Could be a number of things....

What type of RX battery are you using?

What is the torque rating on the rudder servo? It may not have enough torque or the servo could be defective. It sounds like the servo is losing power and then getting it back again. It could also be overheating under load.

You could also be getting radio interference. What frequency are you on?

Also, I see you're using a connection rod between the servo and rudder. Is the rod fully supported? It could be bending/bowing under load inside the fuse.

Cheers
Jason.