Capacity in mAh's at the end of the charge. Voltage is important as well but the capacity is how long the battery will last. Sorry for the confusion. You mentioned that the battery is rated at 4000 mAh's. That means by the end of the charge, assuming that your battery is dead when you go to recharge, you should have 4000 mAh's in the battery. There is an indicator on the 965 that displays how much capacity has been added to the battery. Voltage wise, you should see around 8.4 volts for a fully charged 2 cell lipo.
You are correct. If the voltages are reading correctly then the battery should be alright. Here is a simple test that you can perform on your battery. Put the battery on charge tonight and when the 965 indicates a full charge, write down the voltage, and then disconnect the battery from the 965. Return to the battery tomorrow and check the voltage on the 965 but do not start charging. You should see the same voltage as the night before, which means your battery is ok.
You don't have any way of adjusting the settings on the ESC? Do you know if it has a NiMH mode and a LiPo mode? This is very important because even though a 6 cell NiMH battery is the same voltage as a 2 cell LiPo, the cut off setting for a NiMH is drastically different than that of a LiPo. If you really believe this to be the problem then I would suggest trying your battery on a friends RC car with an ESC that they know is working properly. This way you can eliminate any doubts about the condition of your ESC.