Yeah absolutely, this is all I'm saying. If you sort a group of people by privilege you'll likely find more white people at the front. And I'm in no way suggesting we shouldn't take that seriously.
But you'll also find more people with rich families and more good-looking people and more intelligent people. No child "earns" these privileges either. But ignoring them means we demonise poor and less academically gifted white kids and undermine the confidence of talented black kids.