I use a drill press for all props of any size, especially my bigger props. drill slowly with a sharp bit or a reamer, and keep a windex etc. spray bottle handy and mist the prop as you work on it... that will keep the dust, etc. out of the air and in one local making it easier to clean up. If you use a vacuum make sure no dust is excaping or it will really disperse it in the area. I have a certified HEPA filter on mine but you can try vacuuming up some flour and then shining a flashlight or better yet, a laser pointer in the room with lights out to see if flour got thru the filter.
Good luck and good flying.