Yeah, I hear you. I actually hate the term "person of colour" and I'd never use the term "non-white".
I used POC a few times in this piece partly because I didn't want to exclude people of other colours, and partly because some of the examples I referred to specifically mentioned people of colour.
As I said, I don't like the term, but it's the most easily understood term to get certain concepts across.