ORIGINAL: white pulsss
The only thing i can narrow it down to is a tiny bit of slop in the elevator, ( not much) checked the ailerons, same bit of play. I highly doubt you could get it out at all.
The slop in the elevator is not the problem.
The tail is pushing down; hence, any slop will become a down deflection of the elevator.
The CG is correctly located, for now.
I also suspect of the rubber bands and the wing incidence, since the problem happens erratically.
Do you have an experienced pilot that can fly it for you?