You should have a little metal tool that came with your rusty. It is just a peice of metal with a slot cut into it.
Take that tool, put it on the metal ball and pry it out.

Here
http://www.misbehavin-rc.com/pit-lan...all-joints.asp