You've already been given good advice. Do not worry about going through the side of the wood. That can be patched up and leveled out with balsa saw dust and thin CA glue. Any imperfections beyond that can be leveled with light weight spackle or epoxy mixed with micro balloons to form a paste.

Most of the time I don't bother drilling for the joiner, just ditch out a channel in the elevator halves, stuff the joiner in there and backfill with sawdust and thin CA.