I need to report a pilot error on the trims.

I was not aware that when you have speed, start or crow set that the trims go into those mixes rather than to your normal trims. So, on those occasions where it seemed that the trims weren't working, I had one of these modes set. I have confirmed this.

Problem solved!