sorry its tough for us it troubleshoot when we arent physically there. Try a different battery, if the problem goes away, it was the battery, try a different motor, if the problem goes away, it was the motor.... We can throw out all kinds of suggestions, but really all we can tell you is what you already know... there is something wrong with the battery or the motor, or it could be the ESC....
are you sure you charged the battery?