I think it's the typical dramatisation of black people's lives that you get from a certain portion of the black commentariat.
To be black is absolutely not to walk through the world and watch people do things you cannot do. That is to have an inferiority complex. People like Coates make a living pretending that there isn't a difference. Actually, do you happen to remember where you saw/heard that quote? I'd like to reference it at some point.
As I've said before, I'd much prefer it if nobody, black or white, used the word to describe people. It is an ugly word. It does have an ugly history. But I think the surest way to achieve that goal is to stop treating it as if a black angel loses their wings every time a white person says it.