Yes, I tried the ballast at the same time I took the fin off.

I think the answer is like Jan said....a flex shaft. At the angle the prop shaft is now, it just making the bout fly out of the water, instead of pushing it forward.