Have you tried heating the whole assembly up? The centre part may have a different expansion characteristic. Also, you could try using sockets to press it out on a vice. A big socket on the back and a smaller one to push the part through the bearing. If that fails, have a dremel party with it... :-)

Cheers,
Jason.