I don’t know where you get your information. I have flown thousands of hours over the last few decades and trained many pilots in the process, and when flying an airplane without an autopilot, there is seldom an occasion where a hand is not holding the stick. The phugoid mode mentioned by mesae causes the airplane to slowly climb and descend in cruise no matter how carefully it is trimmed, even in calm air. In addition, there are up and down drafts nearly all the time that require response in order to maintain a level altitude.

Unless the ailerons and rudder are interconnected, like on the Bananza, Tri-pacer, and some others, the practice of steering with the rudder in flight is just sloppy flying and still requires corrections in pitch to maintain altitude.

The single axis autopilot you describe is useful in tracking nav aids, but the pilot still has to maintain altitude with the stick and it is not a “hands free” device.

When you see that Cessna or Piper flying over, you can bet the pilot is holding the stick.