I cannot think of a way stuck lifters would prevent the cam housing from coming free as they only contact the housing and the cams. The cams are contained within the housing so that shouldn't be a problem. The housing only butts up to the back of the crank case. Does the housing move at all or is it just stuck to the back of the case? If it moves, it "should" fall right off. If it is stuck, try giving it a gentle tap with a small hammer through a block of wood or use a small mallet. It sounds like the gasket between the housing and the case is doing it's job really well!

John