A privileged vantage isn't necessary. Almost every moral question can be answered by honestly asking oneself, “How would I feel about this if it were happening to me?”
Slavery, segregation, the Holocaust, every single injustice or atrocity in human history, none of them were justified by people saying "I'd be fine with this," they were justified by dehumanising a group of people and then arguing that these sub-humans didn't deserve the same rights or dignity or consideration as the rest of us.
We see exactly the same process at work now. To hear Zionism's defenders tell it, the Palestinians (all 5 million of them) are all violent, fanatical, caliphate-building terrorists. They can't be negotiated with, there's no clear, easily-comprehensible reason why they oppose Israel, no obvious brutality the Palestinians are fighting against, no founding injustice that Israel is paying for, they’re all just motivated by empty antisemitism and Islamic fundamentalism. This is just so ridiculous, so obviously untrue, that it’s hard to see anybody who can't think their way through it on their own as anything but a simpleton or a bigot.
But even applying your standard of consensus, there is, for example, a universal human consensus that we don't want our homes to be stolen and our families slaughtered. I don't think there are any exceptions to this. Therefore, I consider it immoral to do it. And I'm not particularly interested in Israel's or anybody else's excuses for doing it.
I am, of course, fully sympathetic to plight of Jews in 1800s Europe. I completely understand why many of them wanted to leave. But I can't justify, and don't see how anybody can justify, a solution that persecutes a different, wholly innocent group of people. Because, to come full-circle, I don't think anybody would accept that persecution if it were happening to them.
p.s. I‘m aware that Israeli Jews today would make something like this same argument. They don’t want to be killed by Hamas. They don’t want to be driven from Israel and lose their homes. And despite all of the historical and moral context this ignores, I’m not suggesting that they should be (with the exception of the settlers, many of whom should be in jail). But the difference is, Israel is not innocent. Zionism created this problem and is perpetuating it. And as we’ve discussed, all that’s required to solve the bulk of it is that Israel gives up on its expansionist goals and follows international law.
