the position of elevator is not a good referrence for balance- when you flip to inverted, the geometry of the linkage plus looseness in connections and expo etc., all combine to require more hand movement than actual surface re positioning.
every thing you say, says - grossly tail heavy or and maybe very high wing loading.
set c/g at 25% of MAC- try again.
and make sure that the elevators move positively and easily in even very small movements.