It's like most any symbol used by an unpopular or reviled ideology-- it comes to represent that ideology whether intended or not. The swastika has been used for over 3000 years in some form or another, usually to represent the sun, power, strength, or good luck. Even the Native Americans used the symbol, and it has also been used by the American military before the Nazis. The American 45th Division had it on their shoulder flash, and the Finnish air force had it on their airplanes up until WWII. All these uses predate the Nazi Party acceptance of it as their official symbol, so although now anytime it is seen, the immediate image is of fascism and WWII atrocities, it had many other, less heinous, connotations before the late 1930's.

A similar situation exists with the Confederate Battle Flag, which conjures up many negative feelings in America today.

I think both are a part of history, and we need to remember our history in order to take a lesson from it and not to repeat the errors of the past; however, we should not dwell on the negative connotations or take personally the modern display of these symbols, unless the behavior of the past is present with them. This is what we need to suppress.