No, I don't think you built the box fuselage too weak. It's just a weak design to begin with.

I can't say how much rudder correction to add. Enough to make it go straight? Can you put a nylon washer or two under the mount to tilt the motor to the right? If not, I bet you could sand the front of your mount at an angle to build in the right thrust. If you think it may go too thin, you could always glue a piece of balsa to it before you sand so you're just sanding away the new piece.

You may want to recheck your CG too. If it is a bit too far back, that may be adding to your problem.