I use the rubber if the servo needs to be a little higher in the mount to properly align the linkage. As mentioned above, the vibration damping isn't needed, but does not hurt either.
We use solid mounted servos with outboard bearing supports in fast sailplanes so that the control is precise. When I started building jets I was surprised the same was not needed.