People have been saying this forever. They said it about the civil rights act, they said it about segregation, they said it about women's rights, they said it about gay marriage, it's never been true. Conversation works. Education works. Empathy works.
This argument is only ever made by people who don't actually move the needle on these issues. Because it's comforting to use apathy as an excuse for their inaction.
And as somebody who has spent a great deal of time educating himself on sex and gender, let me say that many of the trans people I see arguing online are also hopelessly misinformed about these issues. Of course, they don't realise this. Because everybody who tries to have a converation with them, however politely, gets ignored or blocked or buried under a torrent of abuse.
And by the way, people are still human beings regardles of how much money they have. If your only justification for the cruelty that these people face is "well, they're rich so they don't have any feelings", then you don't have any justification (and of course there isn't any justification).
Why don't you think people should come out "unscathed" for expressing their opinons? Find me one single hateful thing that Chappelle or Gervais or Rowling has ever said about trans people. Just one. Not a joke, or a statement of biological fact that made some people uncomfortable, one comment that advocates for conversion therapy or violence against trans people or anything of the sort.
This is why I criticised your hyperbolic language earlier. Because it allows you to convince yourself that simple disgreement is hatred and warrants vicious abuse in return. I don't believe any of these people hate trans people. But they disagree with some of the ideas regarding "self ID" (in Rowling's case) or the conflation of sex and gender in Chappelle's case (though as he said, he didn't have the knowledge to express this well).
Maybe you think they're wrong. Maybe they are wrong. But I see absolutely zero attempt to have an actual conversation regarding what they're wrong about. Just abuse and accusations of hatred which sadly turn ordinary people against the trans community.