Are you using digital servos?. Sometimes they will flutter for a few seconds after you turn them on. But they definitely should not shake while in flight.
Heres how I cured the "play" problem with metal cleviss.
1. Thread on a 4-40 not before putting the clevis on all the way to the end of the threads or at least till it is out of the clevis's way while you are making adjustments..
2. Thread on the clevis and adjust the clevis's till you get the alignemnt of the surface like you want it.
3. Once the clevis final adjistments are made, tighten the nut up against the clevis and all the play will be gone.

Sounds like your pushrods are in the right places on the servo arm and horn. Should be out as far as possible on the servo arm and in as far as possible on the control horn to get maximum throw.