What type of clunk line do you have. If the engine only sags when inverted, then I'm guessing the clunk line is too stiff to fall easily to the "top" of the tank when inverted. The engine starves for fuel, goes lean and loses rpm.

If your clunk line is a little too long, it may also be hanging up against the back of the tank.

Have you tried inverted when you first take off, ie. when the tank is full?
Dennis-