Hi Brandon,
If you spin the shaft with your fingers, how long does it take to stop turning? Three to five seconds is about normal, less than this, your preload is to high.
Do you still have the rear o-ring fitted to the shaft tunnel?
If so I would suggest removing it along with your extra spacer, also check that your sleeve & rear bearing slide easily in the shaft tunnel.

Cheers
Darrel