The poll that gets quoted and the poll that does not
Lately, consumers of English language news are reading that most Israelis now support a hostage deal over continuing military action against Hamas in Gaza. For example, the most widely read English-language Israeli website, Times of Israel, proclaims in its headline:
Two-thirds of Israelis back hostage deal over continuing war in Gaza – poll
and this is being parroted on other news sites and blog posts on numerous platforms around the world.
In their article, they quote figures on a number of questions posed by the Midgam research firm. However, they do not state the number of people sampled, when and how they were sampled, nor the margin of error, information that is essential in a valid poll. At least the Times of Israel is transparent in exposing this point.
Let me digress for a moment
and give you my personal impression upon reading that article and those who repeat the claim that a significant majority of Israelis (2/3, or 66%) prefer the deal over continued military action: first of all, none of the polling firms called me and if they had, I would have been on the side of those who reject a deal. So that is my bias.
Because that is my bias, perhaps, and my impression that most Israelis DO NOT support a ceasefire deal, I do not take the published poll reports without feeling the urge to examine them for validity and reliability, two terms I once learned decades ago when working on theses for both of my masters level degrees at university and can only vaguely define by now. I leave that to those of you who still remember, or perhaps specialize, in statistics. Given my lack of expertise in that field, I can only look at what is available to me online and ask myself if it makes sense.
To fully assess the apparent validity of any poll, I would need to see the raw results or at least the entire questionnaire. With the latter, I could identify bias in questioning, leading questions, missing options in multiple-choice questions that would lead to misleading results, and more. Without seeing the questionnaire, consumers of news reports are left having to trust that the journalists had access to the questionnaires and are not just repeating what some organization wants us to believe. I doubt many journalists or news agencies go into that kind of investigation and, therefore, I am left wondering how much to believe of the published poll results.
I also do not want to claim that my personal impressions — that those demonstrating for a deal at any cost are a minority — is the truth without having access to a valid poll proving that I am wrong. All I can do for now is look at other published polls on the subject and see what comes up.
Another Poll
In May, before action in Rafah, Haaretz reported on a poll of 750 Israelis — 600 Jews and 150 Arabs — conducted by the Israel Democracy Institute and they claimed that “The majority of Israelis [62%] believe that a deal to release the hostages held by Hamas in Gaza should be prioritized over additional military action in Rafah.” To their credit (because not all media reports give us this information), they noted that “the maximum sampling error is 3.65 at a confidence rate of 95 percent.”
The results they published include:
Among Israeli Jews, 56 percent believe that Israel should prioritize a hostage deal and 37 percent prefer military action. The numbers are much starker among Arabs: 88.5 percent believe a hostage deal should be prioritized, and just 6 percent said that a Rafah campaign should come first.
Look at this: They write at the top of the article that the majority of Israelis want a deal over going into Rafah. Then lower down they qualify this — 56% of that “majority” are Jews and 88% Arabs. They just took an average figure for the part of their article that will be read most (not everyone makes it all the way through articles these days). Yes, 56% is a simple majority so it is not wrong, but it is misleading to make a big deal over 56% as a majority, calling it “the” majority.
They go on:
Unsurprisingly, Israeli Jews are deeply divided on the question by political orientation. 92.5 percent of those who identify with the political left and 78 percent who consider themselves part of the center prefer a deal, while a slim majority on the right (55 percent) prioritize action in Rafah.
Here we have an example showing why knowing all the questions is important to understanding the results.
It may sound like huge weight must be attributed to the 92.5% of political leftists who favour a deal; however, what proportion of the population does that represent? In other words, that is 92.5% of what? If leftists were 60% of the population, or even 30%, I would probably say that is a significant percentage of Israelis.
Times of Israel published survey results that showed, in rounded figures, that, as of 2022, 62% of the Jewish population self-identified as right-wing, 24% as center of the political map, and 11% as left-wing (total = 97 and not 100). Without examining in depth that survey, we now can say with a certain degree of confidence that:
- 92.5% of 11% of the Israeli population favour a ceasefire (or 10% of the population)
- 78% of 25% of the Israeli population favour a ceasefire (or 18% of the population)
- 45% of 62% of the Israeli population favour a ceasefire (or 28% of the population)
This adds up to 56% of the population and is consistent with their stated figure for Israeli Jews who favour a deal over continuing warfare. This is not two-thirds of the Jewish population, which is what media consumers will think when they read the headline. Is there a significant difference between two-thirds and 56%? I leave you to decide.
Even the figure of 45% for the Israeli right-wing who favour a ceasefire raises my eyebrows and I would like to know more about that. But that is beyond the scope of this article.
Another interesting poll
At the beginning of June, Maariv published an article in Hebrew comparing attitudes toward a deal over the short span of two weeks.
In the first poll, they found that 43% of the population (Jews? Arabs?) favoured the ongoing warfare over reaching a deal with Hamas (39%). Two weeks later, 47% of the population (sample size of 502) wanted a deal and 39% believed continuing military action would bring the hostages home, an almost complete reversal.
What can account for this? The author of the article tells readers that the earlier poll was taken after the video showing the horrific abduction of the young female IDF spotters and the latter poll followed a report on the deaths of four hostages while they were being held by Hamas. I need not spell it out, but I will: the context in which the poll is taken is salient and should always be reported when publishing survey results.
And on Channel 14?
Channel 14 is vilified by left wingers who see it as a biased right wing extremist outlet. A few days ago, they announced the result of a poll they commissioned. According to this poll, 76% of the public do not trust Egypt to prevent weapons from entering Gaza after a ceasefire were Israel to withdraw from the Philadelphy and Netzerim Corridors as stipulated in the current deal under discussion; 17% trust Egypt and 7% are undecided on this point. It is perhaps remarkable, then, that only 53% of their sample are opposed to the deal, 36% support it and 11% are undecided. This contrasts with the TOI headline that claims that 2/3 of the Israeli population support the deal, whereas, according to this survey, only about 1/3 support it.
The media is not reporting on the Channel 14 poll. While they may ignore it, claiming it is not trustworthy because of their right wing leaning, TOI is clearly left wing and no less biased. So why quote one biased outlet and not the other? Quoting both may be a way to provide balance, if that is of interest to anyone.
Polls are just like some sort of a guess-chance game to keep the credulous people guessing, busy and entertained! Giving a reason or subject to speak about, arguing, criticise, reject etc. The result of a poll depends a lot of the way questions are asked. If for instance people were asked if they want all of our people to come home we would, I think have a 100% of people who would agree with that. The same question with a variant….”at all cost” we will have a whole different result. That being said, my own personal opinion on the matter is that we must first and for all win the war and at all cost and hope most will return. We lost a battle on October 7 and it is imperative we win the war no matter the price or sacrifice.
Asking whether you prefer a deal or continuing the war is meaningless unless you specify the terms of the deal.
There are deals I would take and ones that I wouldn’t (the proposals I’ve heard about recently are awful).
I am sorry we are even considering the possibility of making a deal — we should be dictating terms of their surrender.
Yes, I agree, that’s precisely what’s off about the whole situation. Israel should be dictating terms of their surender but instead Israeli society is being divided over the hostage issue and, well it doesn’t appear to be just Hamas that is manipulating things.
I’ve seen the ToI poll and there’s been an opinion piece by Michael Oren on the subject that I’ve exchanged comments about on the ToI readers pages. I’m no statistician myself but I am highly sceptical and suspicious about the whole thing. I basically ask, why has no pollster asked the basic question, do you believe Hamas can be trusted to abide by any agreement following October 7th? It seems a common belief that Hezbollah is afraid of outright war and yet, intel last year indicated that the original plan was for Hezbollah to attack Israel, not Hamas. So, are Israelis been hoodwinked into a false sense of security regarding the enemy to the north? It’s all very worrying, to put it mildly. I know enough about polls to know that polls are manipulated by the way questions are put, the wording used, etc. Interestingly, the JNS poll says most Israelis oppose terms of hostages-for-ceasefire deal. My own biased view, even though I empathize greatly with the hostage’s families and loved ones is that there is something definitely “off” with the whole cease-fire for hostages deal and Israel is being played.
I don’t think we can say — yet — that Israel is being played. If there is a deal, then we will look at the terms on both sides and see if we were played. But the readership of all these media sites repeated this poll are certainly being played.