I'm betting it is the battery as mentioned....

that battery is only rated to 12A discharge while it looks like that motor can pull 16A or more....

have you measuered the Amps being pulled??
normal cutoff is 9v for a 3 cell lipo.... but if the battery can't supply the juice, it can drop below that VERY quickly.... then once you shut off, it recovers....

I recommend a Watts Up meter to measure it... great tool...

in the meantime, test it with a bigger battery or different gearing