I guess you mean well with this, and I actually hate the term "people of colour", but yes, of course there are. Whatever you call them, there are people who, until fairly recently in America, were subject to unthinkable discrimination because of the colour of their skin. We need to be able to recognise that to talk about the discrimination.
But the point is to undo and end that discrimination. Not to define people by it or start applying it in new ways.