No matter what the plane weighs, you want the CG where the manufacturer said it should be (or, better, where an online CG calculator says it should be).

The OP's question isn't really "does changing weight change the CG?," it's "does changing weight change the point where you want the CG to be?" The answer to the first question is "maybe or maybe not, depending on where in the plane the weight changes occurred." The answer to the second question is "no."