If the charger you are talking about is the Wall type then no the lights will not go out when charged, They only indicate the charger is connected.

The 2000 mah battery you linked the answer has to be Yes and No. First yes you can use the higher capacity pack and that will provide longer flying time at the field if and this is a big if; if you get it fully charged. You can use your wall charger which is intended for batteries of around 700mah to use as an overnight charger. Now if you use that little wall 'wort' it will take three time as long (days) and you eventurally find yourself not getting the battery fully charged and not actually getting the full 2000mah capacity.

So although you can use your wall charger its no you will not eventually get the benefit of the higher capacity unless you purchase a more capable charger intended for such batteries.

Check the label on your wall charger and read the charge output and if its 70mah or less then that is the situation. There are some higher output wall chargers but I do not believe that's what you have, check.

John