Three things that I'm aware of, other than too much prop, that can overheat an electric drive, are:
- too small of wires between the ESC and motor
- connectors that have "high" resistance
- excessive drag in the driveline installation
Not sure if this is your first electric or not but, what I've seen, many times, is the ESC is located a short distance from the motor. This requires a jumper or harness of no smaller than 12 guage wires, preferably an 8 or 10 gage. Many buy pre-made ones that are of sufficient size but some prefer to make them for a custom fit. In many cases, the wires used are too small and the system overheats. This is also something that happens between battery packs and ESCs.