Try drilling a few tiny holes in the tube near the end that will be inside the muff. That way you can push it farther in than just the thickness of the muff material.
JB will work - it is essential to rough up the AL surrounding the hole and around the tube - basically, wherever you want the JB to adhere. Clean with denatured alcohol.
You're going to want a fillet around the seam at least 1/4 inch thick stretching about 1/2 in. up the tube and onto the muff.
It'll work!