Next time you are the field, look at all the models there. You will see a lot of them that have the same problem you have. Most guys dont even know it.
If it is not way off, & you are not flying precision aerobatics or competition, you may never notice either.