I got that early on. I realized that for myself, there were a few factors contributing to my pendulum swings:

1) Over correction. But with a little practice you can get it to stop rather than stop then swing back.
2) Blade tracking - If they're not tracking within a certain threshold, the rig as a whole will want to fly out of Radd's box
3) Trim - get it into a flat hover at about 3feet then see which way it wants to slide - adjust trim to cancel out the slide
4) Center of Gravity - check the CG of the helicopter
- with the heli nose facing 12 oclock, rotate the rotor blades so they're facing 12 and 6, and the flybar paddles facing 9 and 3. Pick up the helicopter by the flybar and see if the skids come off the table at the same time or if the front/back come up first. Adjust your CG accordingly till the skids come off the table at the same time.

Hope that helps some.