When we want to collect information about a group of people, we have the choice between collecting information on all members of the group (a census), or only from a sample of them. In displacement-related contexts, conducting a census is often particularly vulnerable to bias and generally also an expensive task. The second option, called sample-based survey, is usually the preferred method of researchers for the results it yields, the reduced vulnerability to certain biases, and cost-efficiency.
However, finding an adequate sampling approach can be a challenging task given the complexity of many displacement situations. What are the different options, and what do we need to consider when defining the sampling frame, target populations, and other key elements? Our new Sampling Guide for Displacement Situations and Practical Examples, developed in collaboration with our partner Statistics Norway, provides you with practical guidance and real-life examples from profiling exercises JIPS has supported over the past years.
The Sampling Guide complements the JIPS Essential Toolkit (JET) and builds the cornerstone of phase three of a profiling process, the methodology design. The JET offers hands-on tools and guidance to empower our colleagues and partners in the field to jointly produce data and analysis for a shared understanding of displacement situations. Choosing an adequate sampling approach is a critical step in this process:
A properly designed sampling approach enables you to assume, with a given certainty, that the characteristics of the sample hold for the larger population it was selected from (to generalise) and is therefore an essential prerequisite for a sound and rigorous analysis. – Sampling Guide for Displacement Situations.
In an effort to support practitioners in making an informed decision on how to select a sample, JIPS’ team together with colleagues from Statistics Norway put together this two-part guide. Each methodological aspect of sampling is illustrated through a series of practical examples. Throughout this guide, we share our experience and lessons learned with the hope that this will inspire a community of practice to lead and uphold high standards for responsible data processes in displacement contexts.
Choosing the sampling methodology best suited for a given situation is not just about reviewing sampling options (e.g. systematic sample, cluster sample, or two-stage random sample). There are several other key aspects and contextual factors that need to be mapped out and considered. Importantly, clearly establishing and defining the objectives of the study is the first step to informing the type of analysis that researchers are seeking to yield from the sample survey. It implies asking questions such as:
What do I want the survey to be able to answer? Whom do I want the survey to say something about?
In this guide, we tackle key questions that should be asked beforehand and discuss important terminology and technical aspects of different sampling options. Practitioners are walked through the process of finding an adequate sampling approach, and learn about possible challenges and limitations illustrated through practical profiling examples from JIPS’ experience.
For instance, in displacement situations it is rare to find registration lists that are completely accurate. As we deal with an often mobile population group, keeping a list of addresses or up-to-date contact information is a major challenge. Establishing a sample frame in a displacement context thus may require some creative and innovative thinking.
During a profiling exercise in Thessaloniki, Greece, in 2017, profiling partners combined data from UNHCR’s refugee registration data (ProGres) with a list of residents in a refugee camp to construct a sample frame for this target population. However, it was found that the information in the database was outdated. The sampling approach thus had to be adjusted by conducting a census survey combined with snowball sampling.
In the case of a profiling exercise in Kosovo, the original sampling approach had to be re-designed due to the lack of accuracy of the sampling frame provided by the census. The non-probability sampling approach then chosen to identify Albanian IDPs for the sample did not reach the point of saturation, which meant that it was not possible to assess with certainty the representativity of the sample.
The second part of the guide focuses on real-life examples of how sampling was conducted within the limitations of the respective contexts to ensure a “good enough” approach. For this purpose, we reviewed and consolidated sampling practices from six profiling exercises: El Salvador, Greece, Iraq, Kosovo, Somalia and Sudan.
In displacement contexts, the operational reality is one of a variety of constraints that influence the choice of sampling approaches. Building on the different steps outlined in Part I, for each use case we explain the objective(s), target population(s), and plan for establishing the sampling frame. When it comes to the sampling approach itself, we discuss the reasons why we selected a specific approach, what steps were followed, and which challenges and limitations we encountered. It is an honest take on our experience and can help others anticipate better the operational realities and what that means for their study.
We aim for the Sampling Guide for Displacement Situations to be a living document and to add further examples as we gain new insights and experiences from supporting collaborative data collection efforts in the field. We hope to inspire and foster a community of practice which capitalises on transparency and leads the way in terms of responsible data processes.
Here are our main takeaways from this review:
