One day I'm going to write something about this. I've developed a strategy of heartily agreeing with ludicrous accusations and then continuing to calmly and compassionately make my point. I've been called transphobic, sexist, anti-Black, anti-White, you name it. So far, it has never failed.
The weapon isn't the accusation itself, it's the fear that the accused has of it. It's the defensiveness that it inspires. It's the way it tends to drag people off topic.
Calling a policy that is demonstrably not bigoted, bigoted, makes you look like a fool as long as the accusation itself isn't given any respect. But you have to stay calm, compassionate, and keep the focus on facts instead of emotion. I realise this isn't always easy to do though.