Yeh, you will have to change them (I suggest you change all at once), otherwise the first time you hit a big bump, the ball cup will just slide off the shaft. When you replace them, be careful how you screw the shafts in, make sure you scerw them in straight, otherwise the same thing will happen soon. Also a good tip for ball cups is to lubricate them (where they fit onto the ball) with graphite dust, it's dry so it doesn't attract dirt, and gives them a nice smooth movement.