I usually break all the fins off with pliers

Then I use a Die grinder with a carbide cutter and smoth it off

I balance the flywheel using a Dubrio prop balancer and shave more metal off the heavy side with the air powered die grinder.