I tend to agree with you. In my mind it's more of a cause-and-effect thing, rather than just saying "this control does this" and "that control does that".
Elevator deflection does one thing directly: It changes the AOA of the wing, and therefore, the aircraft's attitude in the pitch axis. Any subsequent speed change as a result, is an indirect effect of the pitch change. An altitude change is more of a direct effect.
Same deal with throttle. More or less throttle does one thing: It causes a speed change. The resulting speed change generates more or less lift on the wing (assuming there are no offset thrustline pitching issues), which can, but not always, cause the indirect result of a climb or dive (altitude change). The speed change is a direct effect of the throttle change.

All I know is my planes do what I want 'em to (mostly) when I wiggle the sticks the right way. How they do it is of somewhat less consequence than THAT they do it. It's good for starting arguments with, that much is certain.

Rick