I usually associate a slow drift with excess vibration getting to the gyro.

Possible causes:
Improper foam installation
zip tie safety strap
tight velcro safety strap
gyro physically touching frame (under boom installations)
tight servo wires between gyro and frame zip tie hold down point
excess servo wires bundled up and wedged between gyro and frame