Define "Quite a bit".

Can you wiggle the joiner? If so, how much?

Is the slop from top to bottom, or front to back?

Even though the fit may not be snug, it may still work if the gap isn't too big, in either case, a tiny shim against the lesser side of the pocket will force the joiner towards the Major (Spar) side. That's what's really important.