What you're describing here is oppression. We already have a word for that; "oppression". It's been rebranded as "systemic racism" and "institutional racism" and "historical racism" and as I've already said, I agree. This is a one-way street.

But the word "racism" describes something else. Or at least describes something else in addition:

"the belief that different races possess distinct characteristics, abilities, or qualities, especially so as to distinguish them as inferior or superior to one another."

If we say that racism is only about systemic oppression, how do we describe somebody calling you a n*****, or refusing to hire you because you’re black, or burning a cross on your lawn? None of these are systemic, but they’re racist, no? If you don't like that definition, that's fine. But that's an issue between you and the dictionary, not you and me.