Easy things first - does the prop shaft spin freely? A tight shaft is very good at making a motor work too hard and heat up.
A fairly useful rule of thumb I was given was that the prop should be of a smaller diameter than the motor.
Any gearing then makes the motor think it has a smaller prop to drive.