I was just thinking the same thing.
You may have inadvertantly armed the ESC messing with the sticks is why it worked when it did.
Most ESCs require your throttle stick AND throttle trim switch to be in the lowest possible position when you apply power to the ESC.
This is for safety so that your motor will not start when you are plugging your battery in.
If your throttle and trim are all the way down it should only take about 10 to 20 seconds or so for your ESC to arm, most will beep or flash lights or something to let you know what is happening.