Ok, I think I see what's happening here.

The outter rod needs to be securely glued (BTW, scuff the surface with sandpaper and use CA. Epoxy doesn't stick to plastic [that's why plastic coffee can lids make good epoxy mixing trays]) If possible, surround the rod by either running it through a hole in a piece of scrap, or just attaching it to a former with a "V" shapped scrap.

BUT!!!!

Do NOT secure the outter rod too close to the servo! As you know, as the servo arm swings back and forth, it also swings somewhat side to side. The outter rod needs to swing with it. See pic.