Rather than just filing/grinding a flat as "safeTwire" suggested, make it a groove (a flat with two sides. The set screw, with Blue Loctite) is LIGHTLY tightend into the groove. The coller will never fall off, and since you did not hurt the setscrew hex socket, you will always be able to remove the setscrew.