Why doesn't the Israeli population do more to stop the settlers from antagonizing the Palestinians?

There have been settlers driving people out of their homes, and the Security Barrier gerrymanders to protect settlements (making it hard for Palestinians to travel to work or shop). This contributes to anger among Palestinians, who have committed numerous suicide bombings among random Israeli civilians. And, all young Israelis are subject to the draft, which may mean having to serve in the West Bank. Since the fundamentalists are a minority in Israel, but the broader population has to live with the unpleasant consequences of the settlers' actions, why isn't there more pressure to expel settlers from the West Bank (as was done in Gaza) or punish them when they commit crimes against Palestinians (e.g. shooting livestock, sabotaging irrigation systems, throwing rocks at pers-by, etc.) or withdraw the military's protection?