Wood dowls will flex and it you only have one control horn for both elevator halfs, you are not getting equal movement on both sides, dual servos should fix that.

Does it do the same thing inverted? If you push? If it does, it sounds like balance. If it does not snap inverted, it should be the servo issue.