You're correct, a smoking esc is not normal.
Are your components compatible, brushed motor and esc. or brushless? Brushed and brushless are not compatible in any combination.
If the esc and motor are compatible, does the esc have a li-po setting for the number of cells? 11.1 v. is 3-cells; some esc's must be programmed for that.

Some components are faulty out of the box, so this is a possibility also.