Are you using an electric drill for the hole? I generally do these by hand. I start with a small bit then slowly work my way up to the size I need. It takes a few minutes but I just rotate the bit by hand hogging out a little at a time. This makes it easier to keep the hole aligned and straight. If it ends up a little sloppy, no problem. Just fill the voids up with plenty of epoxy. Hardening the surrounding area with thin CA and/or glass cloth like was mentioned is good insurance too.