What airplane is it?

It sounds like your CG is too far back.

On some planes, a tendancy to hunt or pitch up or down at random can be caused by the ailerons being drooped, so you may need to reflex the ailerons a little bit. Reflexing means raising both ailerons.