It may be tougher with the fabric covering because the glue will get into the weave. Good luck.
Gotta love kids......Just think of it as a finishing touch.
Remember, fabric covering can easily be painted, so if you want, paint it.