I never mix fillers with fast set epoxies for that very reason. If you are planning to use fillers, you need to use a slow set epoxy so that the epoxy has the time to be mixed and you still has time to add and mix in the filler. I would recommend using no faster than a 15 or 30 minute epoxy for that purpose. In my case, I routinely use West Systems extra slow hardener with the base resin to give me more working time with the epoxy so that parts can be repositioned as needed before the epoxy starts to harden