I had this problem too. Mine was not on a dowl but on a ply pin. Apparently there was some slop side to side causing the issue. Made it nice and black. Also cut into it causing even more slop. A simple shim made out of brass fixed the problem

Phil (Mesa, AZ)