Findings, Conclusions, and Recommendations Matrices

September 16, 2020

By: Chelsie Kuhn, MEL Associate, Headlight Consulting Services, LLP

This blog post is the last in a series about qualitative methods.

Up to this point, our series has covered thoughts on qualitative rigor truths, things to keep in mind for implementation, and recommendations to help with qualitative data analysis software and coding setup considerations. To close out the series, we wanted to share one of our other best practices and deliverables to demonstrate how we organize the qualitative process–the Findings, Conclusions, and Recommendations Matrix.

A Findings, Conclusions, and Recommendations (FCR) Matrix is an easy tool to implement when going through the qualitative analysis process after coding your dataset to make reports more accessible for readers. If filled in properly, it should mostly write your report for you, only requiring the report author(s) to expand on a few pieces and integrate illustrative quotes along the way. It can serve as a check on the analysis process–a more experienced analyst/evaluator can look at the matrix quickly and see if the conclusions in the table are logically based on what the findings are saying. The FCR Matrix can also serve as a check to make sure that your recommendations are actionable and proportionate to the evidence.

Step One: Build the Tool

At Headlight, when we want to organize our qualitative analysis process after the primary coding is complete, we start by building out a tool in Excel/Google Sheets to use with all of the columns needed for our data analysis. By setting up a basic framework, the analyst can then focus on filling in the columns and rows accordingly without worry that they are missing something needed for the final report in the process. For those working with multiple analysts or internal reviewers, generating this tool in Google Sheets or another collaborative software from the beginning will be helpful later if/when multiple people need to access the document simultaneously, either to fill in the document or to test the logic flow among findings, conclusions, and recommendations.

Step Two: Focus on Scaffolding the Main Worksheet

When we use an FCR matrix as part of our analysis process, the main overarching sheet is devoted to the compiled matrix with columns for findings, conclusions, and recommendations, and with a row for each triangulated theme from the evaluation, case study, Learning Review, or another data-driven effort. Wait to enter any themes from your effort until you have gone through steps three and four that way you are only incorporating themes that have been triangulated.

Themes are generally a variant on what analysts were originally coding for, so sometimes this mimics the evaluation questions, or sometimes it comes from an emergent theme that showed up during the analysis. In the example above, you can see that the sample data were likely coded for any positive enabling environment factors that contributed to desired results. Oftentimes, these themes help organize the sections in the final report based on the priority of findings and any actionable evidence for the client.

Step Three: Integrate the Analysis Summary Sheet and Triangulate

On a separate Excel sheet or tab, include the code count summary worksheets directly from your qualitative analysis software. This should include trend names/code names and code counts. To this, the analyst will want to add columns for triangulation details (a simple Yes/No dichotomy works here), the number of sources where the code was applied, relevant notes for the trend, and a column of if it should be included in the FCR Matrix. The trend names can be modified from the direct code applied to add more nuance, as these trend names will ideally be used for headings and sub-headings for your report. By pulling the trend names with code counts, analysts can get a sense of how much triangulating they need to work on. Every trend with a code count of at least three ought to be checked for triangulation, meaning the code was applied (and applied properly) across three or more sources. If the trend is triangulated, then the analyst should mark the triangulation details and FCR Matrix inclusion columns as “Yes,” and begin secondary analysis before moving any trends and findings to the overarching Excel sheet.

Step Four: Complete Secondary Analysis

For triangulated themes, we also include separate additional worksheets for secondary analysis where analysts can identify sub-themes, apply secondary codes, and expand upon primary coding of data with as much nuance as much as possible. In our process at Headlight, each triangulated theme goes through secondary coding of the exported excerpts from Dedoose copied into the Excel worksheets directly. We will expand more on this in a future series on Learning Reviews, but for secondary analysis, an analyst goes through excerpt-by-excerpt for each theme to find the highest level of detail that can be incorporated into the findings in the FCR Matrix, and ultimately, the end report for the client. The benefit of being able to share more detailed findings with the client is worth the extra time that it takes to do secondary coding, as it allows for another layer of specification about sub-trends and incorporates an additional check on rigor. Once the triangulation and secondary analysis have been completed, the next step is for the analyst to carry over the triangulated trends and their respective findings to input into the main overarching worksheet created in the beginning.

Step Five: Carry Over the Findings

In the overarching sheet, start to fill in the names of each triangulated trend and their associated findings. By devoting separate columns to each component (findings, conclusions, and recommendations), analysts can ensure that findings are just that–triangulated and validated data without any added interpretation. As you can see in the example above, we typically use “x number of excerpts across y number of sources” language to keep our findings as clear and undiluted as possible. Just what the data state. Often in qualitative analysis, it can be hard to patiently assess data without jumping to conclusions, but additions at this stage are often unfounded assumptions that can detract from the actual findings and can harm the rigor and validity of the analysis. To avoid the blur between findings and conclusions, we recommend that you go through moving over the trends and stating all of the findings first, then once that’s complete, analysts can move to the next step to build-out conclusions.

Step Six: From Findings, Move to Conclusions

With findings covered, the conclusions column creates space for natural evolution and interpretation of the “so what?” behind the findings. This is the space where any interpretation gets added to make sense of the findings in the bigger picture. Conclusions can be built based on one finding for a theme, or they can build upon multiple, similar themes when the data points to interrelated conclusions. For example, if sufficient data show that a positive enabling environment affects one of the other themes, say the timely deployment of resources, you could conclude that it is likely that timely deployment of resources is magnified when communication systems are working (an enabling environment factor). Because this is the component for making sense of the findings, it will be lengthier than the findings column. And keep in mind, the more explaining done in this column about what the findings mean, the easier it will be to translate this analysis work directly into a data-driven report for a client.

Step Seven: Make Recommendations

Once all the conclusions have been incorporated, the recommendations column provides the next step in the analysis chain, answering the “now what?” from the conclusions that have been made. In order to build on things that work or change things that pose challenges, what would you recommend that the client do next? When making recommendations, they need to be framed in ways that make them specific and actionable so that they can be acted upon. 

• A recommendation that’s too broad: Invest more resources in staff
• A recommendation that’s just right: Invest more resources in improving operational enabling environment factors, specifically on capacity-building for staff around project management and business management best practices. If capacity building efforts are paired with establishing and training on standard operating procedures for core communications and operations processes, this can greatly improve efficiencies.

While writing the report, authors may need to expand to give a recommendation more context, but the majority of the recommendations section should come directly from the FCR Matrix to ensure that they are tied to the findings.

Step Eight: Review the FCR Matrix and Check Logic Flows

With the Findings, Conclusions, and Recommendations columns complete for each of the trends, the final step in the process is to go back and review the matrix to ensure accurate capture. Are the findings actually just findings, or are conclusions blurred into them? Are there any recommendations included in the conclusions section where they shouldn’t be? Being sure that each distinct component is only in the column where it belongs is a best practice, and it gets our clients used to seeing the distinctions for their own work. With things in their appropriate columns, take one last look or have a colleague review the logic flowing through each of the themes to make sure everything makes sense. If everything checks out, then you have a completed FCR Matrix ready to use.

Why the rigor is worth the squeeze

Using the FCR Matrix not only serves as a way to organize and scaffold the analysis and interpretation process, but it also provides clear documentation to include with any deliverables for the client that they can refer back to if they have any questions. It is important to note that although this tool helps set up the organization of the process, it entirely depends on the user to keep the content in each of the findings, conclusions, and recommendations discrete and appropriately separated. This can be hard to do as even more-experienced folks in the field can have trouble slowing down enough to separate findings from conclusions. Doing it this way helps prevent us from making assumptions, and once we can get that part down, the matrix sets up a clear, logical flow that makes qualitative work more rigorous, reliable, and actionable.

While this is our last post explicitly in the qualitative series, be on the lookout for more related content in the future as we will continue writing on this topic. If you have liked our blog so far and want to be alerted when a new post is published, subscribe to our email notifications. For other questions, please reach out to us at <info@headlightconsultingservices.com>.