Guess I don't understand the down part. Looking at the picture and the part that you have circled the dowel looks like the way to go. Or if you have a hardware store you can get a 1/4" rat tail file. I personally don't like drilling as it always has a tendency to slip (at least mine do) It's called the OOOOPS maneuver.