Our controls are set up to mimic the stick in an aircraft.

If you were sitting in the plane and wanted to raise the nose you would pull back on the stick.  Thus the term "pull up".

If you wanted to push the nose down you would push the stick forward.

Likewise the left and right movement of the stick mimics the stick on a full scale aircraft.