Find the slop and fix it. Either your servo arm has slop in it, the servo is loose, the hole the rod in the servo arm is too big, the bend in the rod is flexing, the hinges are loose, or the control horn in the surface is loose. Not too many other things can cause it. Move the surface and see where the give is and fix that.

Hope that helps some