.png)
We look back on our proof of concept campaign and ahead to our pilot campaign
In our first blog post, we explained why we think family planning is a cause worth prioritizing. Enabling couples to access modern contraceptives may be one of the most cost-effective ways to avert maternal deaths, in addition to providing a host of other benefits.
Often, beliefs and attitudes are the largest obstacle to couples’ accessing modern contraceptives. FEM seeks to promote more informed beliefs and attitudes through educational radio campaigns. We operate in Northern Nigeria, where radio is affordable and widely trusted. We estimate that by 2025, FEM will have helped avert approximately 600 maternal deaths, at a cost of $3,600 per death averted.
In this post, we look back on our first half year, FEM’s proof of concept phase, and ahead to our plans for the remainder of 2021, FEM’s pilot phase (Figure 1). For those interested, we close with some opportunities to support us.
Our proof of concept (Sep. 2020-Feb 2021)
Approach and overall results
The overarching aim of the proof of concept phase was to learn, “is it feasible for us to implement our model?” We believe we passed the feasibility test.
In early February, we conducted a one-week campaign that:
Broadcast ads 140 times on two radio stations
Reached up to 2-3 million listeners
Was remembered by about four out of five radio listeners, according to a small survey we conducted
To successfully implement the proof of concept, we set three key objectives for ourselves: (1) select a promising location to launch, (2) develop implementing partnerships and build a network of supportive stakeholders, and (3) execute a short campaign. We describe how we approached each objective and our results in the section that follows.
Figure 1: FEM's proof of concept and medium term plans
Results by objective
(1) Select a promising location to launch
We selected Kano, Nigeria as our launch point after a three-month, multi-stage analytical process (Figure 2). It was important for us to rigorously evaluate our geographic options. Much as the impact of your charitable dollar can go further in low-income countries, the cost-effectiveness of family planning interventions can vary dramatically across regions and even within countries.
We used cost-effectiveness analysis, weighted factor modeling, crucial considerations, and expert opinion to narrow down a range of potential locations. Ultimately, our country selection came down to Ghana, Nigeria, and Rwanda. We selected Nigeria largely due to its more affordable radio airtime rates, which significantly improved the projected cost-effectiveness of our work. Within Nigeria, Kano stood out across multiple prioritization methods. Key decision factors were Kano’s high rates of maternal mortality, relatively consistent supply of contraceptives, supportive Ministry of Health, and low airtime rates (~$5 for a one-minute ad on a station reaching about 2 million listeners).
Figure 2: Location prioritization methods - country and state
(2) Develop implementing partnerships and build a network of supportive stakeholders
We secured support for FEM from key stakeholders at the federal and state level, and kickstarted implementing partnerships with several organizations. We prioritized establishing partnerships because they are necessary to operate in some cases, and can improve our effectiveness in others.
We formed productive implementing partnerships with:
One local implementing partner, iDevPro-Africa, which is based in Kano. iDevPro-Africa has supported our local stakeholder engagement, prototyping, and monitoring and evaluation. The organization is led by one of the foremost experts in family planning in Nigeria, Dr. Mansur Muhammad Tukur, who shares FEM’s vision and values.
Two radio stations. Freedom Radio, which targets a more urban demographic, and Arewa FM, which targets a more rural one, played our ads during our short campaign. We were able to secure discounted rates from them due to their commitments to corporate social responsibility.
Three radio production firms. We initially assessed firms through referrals and a request for proposals. Three firms then developed prototypes of ads. We then tested the prototypes with target listeners and selected one firm to produce the final versions for the short campaign.
We also worked to build a broad network of supportive stakeholders for our model. We held more than 40 one-on-one meetings with stakeholders and partnered with the Government of Kano State to hold two stakeholder workshops.
We’ve been pleasantly surprised by the positive reception we’ve received from the Nigeria Ministry of Health, Kano State Ministry of Health, Kano State Primary Healthcare Management Board, local NGOs, and religious and community leaders. As an illustration of this support, 43 stakeholders in Kano signed on to a communique saying they “welcome and support” our work, and our co-founders were formally invited to meet both the Emir of Kano (Alhaji Aminu Ado Bayero, Picture 1) and the Emir of Bichi (Alhaji Nasiru Ado Bayero), who expressed their appreciation for FEM’s work.
Picture 1: At the palace with the Emir of Kano
FEM co-founder, Ken (second from left), and iDevPro-Africa country director, Mansur (second from right) with the Emir of Kano, Alhaji Aminu Ado Bayero
(3) Conduct a short campaign
In February, we placed two ads on two radio stations ten times a day for one week. We wanted to trial a short campaign relatively quickly, as we thought it would help us anticipate issues to address in future, longer campaigns.
Figure 3: Components involved in researching, producing, and broadcasting the short campaign
The week-long campaign we conducted was the product of a months-long process of:
Audience research. We reviewed surveys and reports, and spoke to dozens of stakeholders to understand what barriers to contraception to target in Kano. Our audience research helped us to avoid missteps. For example, one of our scripts referenced a husband’s “energy” leading to more babies. Several stakeholders indicated that this innuendo would offend some listeners or distract attention from our message. We replaced it.
Script writing and prototyping. Based on our audience research and other successful campaigns, we worked with our partners to develop seven scripts for ads. We created prototypes of three of the ads and gathered feedback from target audience members through focus groups and interviews. We also gathered feedback from stakeholders. We then selected two ads to be professionally produced.
Production and pre-testing. We conducted another round of focus groups (Picture 2) and interviews with target audience members to get their feedback on the professionally produced versions of the ads.
After the one-week campaign, we conducted a small-scale survey of radio listeners, which indicated that our ads were heard, remembered, and understood. About four out of five of those surveyed recalled hearing our ads. Half of those who remembered our ads were able to convey their core messages about family planning to the surveyor. We consider this to be a strong result after just a week of ads. Equally importantly, we received no negative feedback from surveyors or other stakeholders during or after the short campaign.
Picture 2 & 3: Focus group discussions conducted with target audience members
Reflections
What we did well
Small-scale experimenting. We try to conduct low-risk tests before making major commitments. For example, we had three production firms produce prototypes, and tested them with stakeholders and target audience members before selecting one for our proof of concept. Our research indicated that, at least in this case, the prototypes that most resonated with listeners were those produced by the firm with the most experience on health campaigns - as opposed to firms with extensive experience working in Kano.
Timeboxing. Timeboxing is the practice of setting a maximum duration to implement an activity. We’ve found it helpful to timebox activities where there is no clear endpoint or level to reach. For example, we gave ourselves six weeks to select a country to launch in. Timeboxing forced us to focus on only the most critical factors in the decision. It also meant that we could encounter implementation challenges earlier and reverse our decision earlier, if we had to (we didn’t).
Adapting our operating model. We strive to be a nimble organization that adapts as it learns and gathers evidence. For example, initially, we had planned to hire a Research and Communications Manager to provide on-the-ground support for our pilot in Kano. As we came to better understand the capabilities of iDevPro-Africa, we found that there were many advantages to working with them instead of hiring someone. We’ve benefited from having an entire organization that can provide far more support and expertise than a single individual could.
What we could have done better
Outsourcing legal registration in Nigeria earlier. Initially, we thought we could reduce costs by managing the process of legally registering FEM in Nigeria ourselves. We underestimated the time required to understand the process and complete the necessary requirements. In December, we hired a Nigerian law firm to manage the process for us, which has been well worth the cost in terms of management time saved.
Ensuring quality assurance on documents. We pressure test ideas and strategic and operational plans with several experts, including a FEM Advisory Board. However, until recently, we didn’t have a review system in place to catch proofreading errors. The quality of documents we shared was, in some cases, undermined due to typos and other such defects. We now share important documents with a proofreader for quality assurance.
Our pilot phase (Mar. 2021 - Dec. 2021)
Having answered to our satisfaction the question, “is it feasible for us to implement our model?” we now seek to learn, “how can we implement our model as cost-effectively as possible?” To answer this question, we’ve set out three key objectives to achieve over the course of a nine-month pilot phase:
Further calibrate our intervention approach and content to the local context. We’ll update our approach (e.g., number of ads per day, length of ads, type of shows) and content (e.g., barriers addressed) through desk research, barrier analysis surveying, human centered design, and other quantitative and qualitative research methods.
Conduct a pilot campaign. We’ll integrate findings from our research into a 2-3 month long campaign consisting of ads and hour-long radio shows.
Assess the effectiveness of the pilot. We’ll use a number of methods to assess the pilot’s effectiveness, including a potentially innovative one: creating a control group by disrupting our programming in one location with a short-range signal transmitter.
By the end of the year, we expect to have far more evidence of FEM’s impact (or lack thereof) in Kano. This evidence will inform our decision on rolling out a long-term campaign in Kano and conducting a pilot in other Nigerian states.
How you can help
Supporters can help FEM by:
Providing technical advice. We’re interested in speaking with more academics and other experts on family planning, social and behavior change interventions, and qualitative research.
Volunteering for short- or long-term engagement. We’ve identified numerous opportunities for volunteers from anywhere in the world to support us, especially in desk research and website design. You can express interest in opportunities here and read more about them here.
Contributing funding. We have room for more funding. For $700, individuals may sponsor a week’s worth of advertisement airtime in Kano. Interested donors may contribute on our donation page.
Overall, we’re encouraged by the progress we made through the proof of concept phase and excited about our prospects for the pilot phase. We look forward to hearing from you! Please feel free to reach out to us directly with your feedback, questions, or ideas.
Cross-posted on the Effective Altruism Forum
Comments