Quick and easy(?) way to do it.

As you describe it, remove the servo, assemble the plane and support it at the intended CG. Then use blue tack and stick the servo on to get the CG right. With the plane at hand this is actually the best way to do it.

Second best, make a full size sketch on the wall or blackboard/witheboard to visualize. Then measure, weight and do the math. Andy Lennons book on model aircraft design outline this procedure.