The motor insulation is one of the factors that determine maximum applied voltage.
Next, the maximum current is a factor.
Assuming a brushless motor, and an ESC that applies full voltage pulses to the motor,
It's possible to have peak currents that are higher than the motor is designed for if the applied voltage is too high.
An 8S lipo is about 32v. add 8 more volts for the additional two cells you mention.
One of the complications is that the HV ESCs can get a bit pricey when you go above 6S.
Props also have an RPM limit that depends on the length and type of the prop.
Finally, an ESC provides pulses at a rate. It can be that the motor RPM has a limit determined by the ESC.