I dont really know what else to offer you as far as advice goes. Maybe someone else will chime in on something else I havent experienced yet and give you something else to look at.
One more thing though, since you said you had the Castle ESC. Did you set your throttle end points? If not, search on how to do that and try that. I dont think that would cause your issues, but one more thing that should be looked at.
Im thinking you may be right about it being the motor. Perhaps a magnet has come unglued inside the motor and that is whats causing your issues.