Hello,
I have a bunch of metal working tools in my garage so I'm pretty fortunate. I have a floor standing drill press that I have an "x-y" vise mounted to the table. This vise (available at Sears- through the catalog- for about $100.00) allows me to move my work both for and aft and side to side. It turns your drill press into a poor mans mill. I have some round carbide type cutters (I call 'em Burrs but I don't think that's correct) and I move the table up until the cutter and the spinner are at the same height. Then I move the spinner INTO the cutter and quide the spinner using the 2 handles on the vise.
Here's a photo of a typical spinner mocked up in the vise. I would have raised the table up a few more inches but the basic idea is conveyed here. They make smaller vises for smaller drill presses so one wouldn't have to spend a ton of many to get a similar setup.
I second the comment on aluminum and grinding stones--don't do it!!!!!!