I don't think this is a "language guys" thing, I think it's a mature human being thing. More importantly in this case, we should be able to tell the difference between saying a word in context and using it as an attack.
I agree, I think that we'd be better off if all racial slurs faded from use. But I don't think we can get there by promoting the idea that simply uttering them, regardless of context, is somehow harmful.