Skip to content

Appendix 2 – Methodology

Last updated:1 November 29, 2024

 

Note: this is my attempt to clarify my methodology in evaluating the sources of information I have been using in this document while facing the uncertain reality in Gaza.

 

Overview

As discussed in depth in the Media section of this document, the war in Gaza features purposeful obfuscation, falsification and silencing of information by various actors involved in it. Since the veracity of most information available to the public is not absolute, attempts to make sense of the war such as this document must resolve a tension. An overly naive attitude to the evidence would result in the inclusion of much false content. On the other hand, an overly sceptical attitude to the evidence would result in knowing very little – which in turn serves the purpose of some of the actors in the war. The challenge is to minimize the false information while maximizing the true information. This often boils down to the contrast between what is desirable to what is realistic within the existing constraints. Key within this war is the absence of external reporting – Israel has kept international journalists out of the Gaza Strip for over a year as of writing, significantly reducing the amount of information coming out of Gaza. The clearest voices coming out of Gaza are those of Palestinian journalists, who have been killed – sometimes purposefully so – and intimidated by Israel (I cover both topics in length in the Massacres and Media sections of the document).

My personal approach has been to speak up to the best of my ability and share the truth as I understand it because of the urgency of the matter. I am aware that others may have purposefully or mistakenly misled me so that some of the content I have shared might be imprecise (if you know of any such content please inform me!). Nonetheless, I do not believe we have the privilege to wait for the fog of war to dissipate before taking action. As below, I assume very little and continuously and critically reflect on my sources, drawing on the skills I have been using in my professional work as a historian.

I measure the results of my efforts – this document – by the standard of other institutions who participate in the same discourse. These include official institutions such as the United Nations, as well as reputable NGOs such as Human Rights Watch or Amnesty International. Some of the products of these institutions include a section describing their methodology (such as this one), and some of their products do not contain footnotes that allow one to trace the evidence they used (and reconstruct, to some extent, their methodology). The standards of countries or the media tend to be far lower. While I believe everyone discussing the war (myself included) is biased to some extent, the more reputable sources at least make an effort towards mitigating some of these biases, while countries and the media are often quite obviously biased toward some political position.

Within the current discourse, I believe this document is more robust than most other summaries, reports and analyses I have encountered. It is not perfect – its development can be traced through its earlier versions, which I keep available online – and it is not final. Nonetheless, the additional information that has surfaced over time has reinforced almost all of my earlier estimates. Furthermore, some of this new evidence has demonstrated that some of those earlier estimates have been too conservative.

Below, I lay out how I examine and assess the sources of information I have used in this document, which I divide into primary and secondary sources. I follow up with a few examples of types of evidence and how I evaluate them.

 

Primary sources

By primary sources I refer to several types of evidence:

1.      Evidence recorded or transmitted directly by people who experienced some aspect of the war themselves. This category would include social media posts by Israeli soldiers or testimonies by Gaza civilians.

This is generally the most problematic type of primary source evidence for the purposes of this document as there is sometimes no way to verify the content and even identity of the people speaking. My tendency is to believe people who narrate their experiences in a detailed manner – whether a released Israeli hostage or a civilian in Gaza. The political opinion or affiliation of these sources may bias their accounts but it does not make them inherently unreliable. I consider these accounts more reliable if one of the following two conditions takes place.

  1. The person providing their testimony does it against or at least without concern to the broader interests of their side.
  2. The content shared corresponds to enough other testimonies, reports or stories that I have encountered, which I can cross-reference. The more independent sources say something in a detailed manner, the more I consider the information reliable.

It is always possible to cast doubt on such evidence and claim that it is part of a much broader influence operation meant to convince readers of some falsity. Based on my experience participating in this discourse for over a year, these attacks tend to be attempts to discredit voices of individuals or whole groups for political purposes rather than a pursuit of the truth per se. In those cases that have been exposed as influence operations (such as this2 or this3) – the content is often crude and lacks detail, making such cases easier to identify as false. While crude and undetailed information could be true, I prefer to wait until more information accumulates before incorporating it in my understanding of the war. In those very rare cases where I used it in earlier versions of this document because of the relative absence of evidence early in the war, I qualified my statements accordingly and actively searched for additional information, which I incorporate in more recent versions of the document.

I have been following the war carefully for over a year. Sources that continuously provide information that I can verify elsewhere become more reliable to me over time. If I discover that a source is purposefully lying or substantially wrong, I consider that source unreliable and refrain from using it. Sources that inadvertently make a mistake, discover it and admit they were wrong are exempt from this rule – I believe that such admissions make them more reliable (see point 1 above). I also do not include here minor mistakes such as using outdated material – these are ubiquitous and there is no point in dwelling on them now because they are less important in the present.

 

2.      Facts reported by journalists in the war.

I generally trust fact-based reporting, but I read these critically and attempt to cross-reference them with other sources if possible. This kind of reporting can also be considered a secondary source (see #1 in that section below).

 

3.      Statements, which I often cite because the people who make them hold some official position or have some social influence, or participate in the war in some function.

I use these statements as evidence for the intentions or goals of the individual or the government they represent. These statements are not necessarily true. Because of the temporary nature of much of this material, I often end up referring in my footnotes to a random twitter account that happened to record and upload this particular type of content, which would otherwise be inaccessible to me. In these cases I clarify here that I cite the original content shared rather than the twitter account of the person sharing it, with whom I might vehemently disagree.

I also accept as factual statements in media pieces about some official saying something using a direct quote. I will evaluate the statement of course, but I do not doubt that he or she said it if a journalist says so. Media reports using indirect quotes can be purposefully misleading, for example in the case of the Israeli media’s treatment of the former hostage Noa Argamani (for example here4).

 

4.      Official documents by an institution referring to policy, procedure and the like.

These documents are also not necessarily true, but they are official and thus represent reality or what that institution wants others to believe is reality. I use such documents for their factual claims after carefully evaluating them, and can also use such documents for the purpose or intent of the issuing institution.

 

Secondary sources

1.      Media reports.

Traditional media includes a wide variety of sources of different quality. Much of this media is quite clearly biased toward one or the other side and I employ the rationale I discussed above (under Primary Sources – 1). Detailed investigations that include specific bits of verifiable/falsifiable information are often more reliable. No media outlet is perfect and even the most reputable traditional media can make major errors, as demonstrated in the case of the New York Times’ Screams Without Words story or in its pro-US government / pro-Israel biased representation of the war (both cases are discussed in the Media section).

I also take into consideration the individual journalists responsible for a story or report in my evaluation. Journalists who “got things right” in the past, have access to valuable sources and whose accounts were not refuted become more reliable over time. Journalists who act as de facto mouthpieces of a state or institution (i.e. repeat their narratives while applying little to no criticism) are far less reliable even if they are sometimes used to leak information that I cite.

I refrain as much as possible from using analyses or opinion pieces that appear in the media as I believe that their value for understanding the present is limited and it is better for one to make up their own mind based on evaluating the evidence themselves.

 

2.      NGO reports.

There is no hard rule here, but in general, I consider NGOs that allow access to the sources they used through their footnotes or by quoting them directly with identifiable details as more reliable than those that do not. Within this category one should try to balance this with what is realistic.

I consider NGOs that are more obviously focused on partisan politics on Israel/Palestine/Gaza as more biased and less reliable than others. My impression is that NGOs with wider interests (e.g. human rights in general) are often less biased and more reliable. I end up using NGOs that transmit more information from the ground in Gaza even when their methodology is unclear when this is the highest quality of information I can find. As above, I consider as untrustworthy NGOs that I have discovered purposefully falsify information or lie.

 

***

 

Examples:

•          The images and videos Israeli soldiers upload in which they present their experience in Gaza and in which they act immorally or enjoy such behavior.

This is a primary source. I consider this content generally authentic since the soldiers present information that is politically harmful for their own side. There have been a few cases in which such material was found to be inauthentic, but these are negligible compared to the quantity of information wilfully shared.

 

•          ZAKA accounts of the horrors of Oct. 7.

This was a primary source that drew much attention early in the war. As an NGO, I believed the ZAKA accounts at first, but a few months after the beginning of the war investigative reports revealed that some of the worst atrocities they reported, which also drew the most attention, were fake. Furthermore, ZAKA did not admit that this information was fake. As a result, ZAKA lost its credibility in my eyes.

 

•          Accounts of food prices within Gaza.

Some of the food prices are anecdotally mentioned by individuals in Gaza; other food prices are mentioned in media articles; and some food prices are mentioned in NGO reports. All three independent sources tend to point to similar fluctuations over time, and these fluctuations are correlated with the availability of food within the Gaza Strip (or specific parts of it) as measured by other indicators, such as the number of trucks getting into Gaza. Therefore, I consider these reliable in general.

 

•          Accounts of massacres and deaths in Gaza.

Throughout the war there has been a very large number of videos and images that claim to show the survivors or victims of Israeli attacks. These are often bloodied and sometimes include gory details. They are shared by individuals from Gaza and outside of Gaza, some NGO members and some media outlets such as al-Jazeera. There have been only a few cases in which this kind of material has been claimed to be fake, false or misleading. The massacres and deaths corroborate written and statistical information about the results of Israeli attacks, for example in investigative reports and NGO publications. While one cannot be completely certain that all this material is reliable, there has been enough of it coming from independent sources for me to judge most of it as reliable.

 

***

 

The experience of trying to determine the truth in the process of writing this document has been illuminating for me, often revealing the wide gaps between existing evidence and the lack of reporting on that evidence, or between sometimes wild claims that are accepted as truth and the absence of evidence for those claims. I am aware that some would disagree with my methodology. I appreciate any non-partisan suggestions to make it more robust for the sake of all who would rather unravel the truth, even if that truth is deeply uncomfortable.