Have you check how the plane tracks on the ground?

Roll it down a small incline or push it forward and make sure that the tailwheel causes it to track straight.

It sounds like adjusting the tail wheel assembly is all that is needed.