Authors: Alison Harrell, Amy Leo, Maxine Secskas, Monica Matts, Rebecca Askin, and Rebecca Herrington
When working in complex systems, there’s only so much we can control, and challenges and disruptions are often the norm. In light of this, Headlight staff performance is not evaluated against our project outputs or outcomes, over which we only have so much control (that performance is learned from and assessed through our organizational monitoring, evaluation, and theory of change testing). Instead, staff performance evaluations look at the ways in which our behavior reflects Headlight’s values, depicted in the graphic on the right: Data-driven, Utilization-Focused, Learning-Oriented, Relationships & Systems-Oriented, and Earnestly Creative & Collaborative. Practical application of these values in the day-to-day ensures we have the enabling environment and behaviors that contribute to quality development work. And, when inevitable challenges arise, our values help us bring our best. Below are snapshots of how we have put each of these values into action through a busy and growth-fueled 2023.
Utilization-Focused: We ensure that our services, resources, tools, and deliverables are designed based on their usefulness to intended users.
Value in Action:
In 2023, USAID/Bosnia and Herzegovina (BiH) conducted a mid-course stocktaking (MCST) for their 2020-2025 Country Development Cooperation Strategy (CDCS). Headlight supports CLA as part of the Mission’s MEL platform, MEASURE II. In this role, we worked with the USAID/BiH Program Office to design an event to help participants make sense of and use a lot of data during the stocktaking exercise. Our support included working with our colleagues to create digestible data placemats and posters for Mission staff to engage with 2+ years’ worth of monitoring and context data.
Headlight also created an event agenda and facilitation process that prompted the Mission staff to discuss and agree on updates to the CDCS Results Framework. We planned two activities to facilitate sensemaking of the performance data and the contextual information: a gallery walk of the posters as a way for staff to review, discuss, and ask questions about the information and breakout groups to discuss issues using a “What? So what? Now what?” format. These small group discussions allowed participants to reflect individually and then discuss the import of the information presented for each Development Objective (DO) and suggest adaptive actions. The participants were active in all of the discursive activities and group work, bringing a variety of ideas and constructive conversation. The different types of activities were successful in engaging different learning styles, and by the end of the event, everyone had participated in some way.
From observing participation during the event and reviewing the post-workshop evaluations, we learned that the MCST sessions were well-received, staff appreciated the opportunity to use and interact with the data, and, as a result, they were able to agree on updates to the Mission’s CDCS Results Framework.
Relationships & System-Oriented: We continuously reflect on relationships among things (systems, interventions, etc.) and persons (collaborators, stakeholders, etc.) to understand and respect the systems in which we work.
Value in Action:
In 2023, Headlight concluded a 2.5-year Development Evaluation (DE) with USAID/Ethiopia’s Disaster Risk Management (DRM) team. The DE team supported the DRM team in identifying prioritized learning questions, implementing Learning Reviews and evidence syntheses, conducting evaluative efforts, and providing ongoing adaptive action support to the DRM Team and its implementing partners (IPs).
Because the SDRM-SI DE was a project-level DE, most of our learning, evidence-generation, and adaptation supports were provided at the project level, but we learned early on that there was interest from DRM’s IPs to tap into the DE support. In order to ensure that DRM IPs were able to access DE services and we were using resources effectively, the DE team brainstormed possible models for support and decided to establish five DE points of contact with 12 DRM implementing partners to assist in ongoing data collection and analysis and support adaptive actions.
In closeout conversations with our IP colleagues, we heard that adapting our service model to the system was a success. We learned that our regular sharing of evidence, adaptive action planning, and facilitation of learning workshops were several of the most valuable supports to IPs. As a result of this support, IPs spoke about using DE deliverables, evidence, and tools to adopt action-oriented recommendations and implement adaptations. For example, one client shared: “These were very useful and I am excited about how much evidence was generated, and the follow-up. In that sense, it allowed us to chart [a path] forward engaging with the government, which we did and now we have got the sustainability strategy [and] also the activities that need to be done before the end of the project.”
Data-Driven: We practice and build capabilities for data-driven decision-making as a catalytic factor in improving the effectiveness and efficiency of development efforts.
Value in Action:
Leading for Change (L4C) is a training focused on teaching leadership skills needed to participate in policy-making processes or policy advocacy. The training is offered via the seven-year (2019 – 2026) Feed the Future Policy Leadership, Interactions, Networks and Knowledge (Policy LINK) project, managed by DAI, which aims to strengthen the capacity of local actors and institutions to lead the agricultural transformation process and contribute effectively and collectively to improved, broad-based food security policy outcomes.
To date, Policy LINK used output indicators and anecdotal stories to talk about the training, but the team in Ethiopia wanted evidence of behavior change outcomes and the Policy LINK Global team was interested in testing the theory of change that underpins the facilitative leadership interventions across buy-in teams. Headlight’s DE team designed and implemented a learning activity to generate evidence about whether participants in the L4C applied new skills and knowledge in their day-to-day work and whether they have observed changes at the institutional or system level. The DE team conducted in-person interviews with participants of the first L4C cohort from several regions in Ethiopia, interviewed the trainers, and facilitated a focus group discussion with the activity team.
The evidence from this study confirms that participants in the Ethiopia Activity’s first L4C training cohort are using a growth mindset in their leadership approach with colleagues and in collaboration with other development partners and successfully engaging and influencing stakeholders in the food policy space. Participants both described a change in attitude or behavior and directly attributed the L4C training as the cause of this change. The Activity team in Ethiopia will use the evidence from the learning activity to adapt the training delivery, add or modify other capacity-building services, and make more explicit connections between the Action Projects (a joint activity that training participants work on together) and their other activity areas such as funding research and supporting policy dialogues.
Earnestly Creative & Collaborative: We share our process, content, and technical knowledge in a way that demonstrates and facilitates “power with” and “empowerment of” our partners and colleagues.
Value in Action:
In South Sudan, Policy LINK used a participatory five-step process to help facilitate communities’ capacity building and autonomy to identify and implement a community-led resilience agenda. In the final year of this work, Policy LINK wanted to share evidence and lessons learned to support other USAID implementing partners working in South Sudan or similar contexts.
As the CLAME thought partner for Policy LINK, Headlight was asked to evaluate the program and identify learnings. Following discussions with the activity team, we agreed to conduct an outcome harvest evaluation. Outcome harvest is an evaluation method that helps identify, verify, substantiate, and make sense of outcomes that may be otherwise unclear or unidentified. It was well-suited to this situation, as it would allow the voices of community members to come through alongside those of government officials and implementing partners, while maintaining evaluation rigor.
The evaluation team conducted more than 130 interviews with participants in South Sudan and, as a result, had a great deal of data to analyze. With a small team and fluctuating deadlines, the evaluation team needed to flex their creative muscles to decide on the best way to analyze all of the data. During a working session together, the team used a Mural to create a visual map of the data excerpts they had coded. With this visual, they were able to understand connections between outcomes, ascertain emerging themes from the data, determine which of those had the most data associated with them, and agree on which themes to prioritize for analysis.
Using the priority themes they identified, the team proceeded with analysis of the interview data, and the evaluation found 13 positive outcomes in connection with Policy LINK’s approach. The identified outcomes included “Community has an improved understanding of shocks, potential contributions to solutions, and experiments with more resilient practices” and “Communities’ mindset shifted to be more self-reliant.” The themes the team identified were also used in organizing the final evaluation report, making it more readable and useful for Policy LINK staff. Policy LINK has shared the evaluation results in several fora, including with USAID, and expects to use the findings in upcoming activity designs.
Learning-Oriented: We integrate learning into everything we do through feedback loops, taking time to reflect, understanding broader contexts, and identifying and acting upon adaptations.
Value in Action:
When staffing our first prime contract – the Learning Adaptation activity with USAID/Ethiopia and Djibouti – Headlight chose to hire Ethiopian staff in key leadership roles with backstops in the home office, no expats on the ground. While we believe firmly in this approach to contribute to better locally-led development, it’s not the norm, and it requires additional time, energy, and care. Communication across time zones can be tricky, and working within different cultural systems, each with its own sets of expectations and norms, necessitates a culture built on trust, openness, and humility. We have to share our perspectives, realize expectations that we may not be sufficiently vocalizing, and navigate establishing a joint organizational culture that meaningfully respects all the contributing cultures.
In order to address these challenges directly, we instituted operational check-ins during our weekly project management meetings. We take 10-20 minutes to reflect on what’s working and not working internally at Headlight, with contractor colleagues, and with Mission staff. Then, we discuss what adaptations we can employ to move toward smoother operations in the next week. This learning approach has helped us to identify common challenges across the team and recognize and appreciate areas of improvement on a continual basis. The sessions have helped us to unite in solutions and draw on our collective strengths as a team.
Comments
no comments found.