Introduction to Secondary Analysis

April 7, 2022

By: Maxine Secskas, CLAME Associate, Headlight Consulting Services, LLC

This blog post is the third in a 3-part series on components of qualitative methods.

This blog continues the discussion on how we at Headlight conduct Qualitative Analysis from the previous post called Intro to Primary Analysis. This blog post will build on the information from the last post.

SECONDARY ANALYSIS

While primary analysis can provide a detailed overview of your data, secondary analysis is essential to accurately represent stakeholder voices, bring nuance to your findings, and identify actionable learnings. The following guidance focuses on applying secondary analysis using a thematic approach, as this is one of the most common analysis methods and often is applied in substandard ways.

In this phase of analysis, the focus is on re-reading excerpts from codes (also referred to as “themes”) with the highest likelihood for triangulation you identified in Step Two of primary analysis (Intro to Primary Analysis Blog Post) and conducting another round of emergent coding within those thematic excerpts to identify sub-trends, and further explain the “why” and “how” of a particular finding. For instance, knowing that the code for “Cost Savings” is a triangulated outcome for users of digitally enabled solutions is beneficial but not actionable. Secondary analysis within the “Cost Savings” theme may help clarify that this outcome is particular to digitally-enabled utility solutions (e.g., digital water meters), that there are cost savings for both beneficiaries and municipalities, or that cost savings only happen when the roll-out of a digitally enabled solution is accompanied by training.

EXPORTING CODED EXCERPTS FROM DEDOOSE

Here at Headlight, we prefer to use Dedoose software for Qualitative Coding, and this section will review how to export the necessary information from Dedoose. If using different software, you can skip to the next section to learn about our secondary analysis process. While we do all of our coding and much of our primary analysis within Dedoose, for large qualitative datasets, we suggest manually conducting a rigorous secondary analysis in Excel/Google Sheets utilizing exports from your Dedoose dataset.

In order to export your coded excerpts from Dedoose, you need to use the Code Application Chart in the Analyze tab, find the code you want in the top row, and then scroll down to the corresponding code application total count box in the bottom row. Once you click on the total count box, you will open the Chart Selection Reviewer, and then you will click on “Export Excerpts.” Make sure to check the boxes for “Attached Descriptor Info” and “Media Title,” and if you previously determined that you need to check this code for a significant co-occurrence, then also check the box for “Code Applications.” Then click Export. We suggest editing the name of the file to indicate the code you are exporting. From the exported data, you can delete all of the code application columns that are not relevant to the co-occurrence you are checking. Once you have the excerpt information you need, you can copy it into your analysis workbook.

ANALYSIS PROCESS

1. Conduct secondary coding of each identified theme. To start, import the excerpts you have exported into a new worksheet in your analysis workbook (Worksheets E from primary analysis) for each thematic code. For these sub-trends, you do not use your previous codebook but instead create new, emergent codes based on any themes you notice when re-reading the excerpts, which further expand upon or explain the code’s theme. These sub-trends do not need to be too granular in detail but should add a level of nuance to the original code. At Headlight, when doing secondary analysis coding, we assign a name for each secondary analysis code, which we put in the column “Code Category,” and we assign each sub-trend code an alphabetical letter, which we put in the column “Code.” Assigning a letter allows for errors and typos when writing in the code category and also for variations, which happens when reading through large numbers of excerpts while maintaining a clean alphabetical codebook that enables easier sorting for findings write-ups when the secondary coding is complete. For example, one code category could be “savings from cheaper utilities,” and the code could be “A,” and the next category could be “savings from reduced leaks,” and the code could be “B.”

LESSON LEARNED: DO NOT be too specific in your secondary analysis. If you create secondary analysis codes that are too nuanced and context-specific, you will have difficulty triangulating any trends. Maintain a balance between adding nuance to your findings and overemphasizing singular perspectives.

2. Check for triangulation of sub-themes. Once you have conducted the secondary coding for each theme, then you need to check to see if any of your sub-trends are triangulated. You should filter the “Code” column for each alphabetical letter you assigned and determine whether any of the sub-trends are represented in at least three different media sources. That sub-trend will then become part of the write-up of the final findings.
3. Check sub-themes for co-occurrences. Once you have found which sub-trends are triangulated, you will then check each of these trends against any codes or descriptors you might have flagged for co-occurrence in Step Three of the primary analysis blog post. If any sub-trend co-occurs with another code and is still triangulated, then you have identified a significant association between that code’s sub-trend and the co-occurring code or descriptor, which should also be captured and explained in the write up of the finding.
4. Write up your findings. Lastly, in your Analysis Summary Worksheet (Worksheet B), you will write notes about any triangulated sub-trends and co-occurrences. This should be brief and usually includes the number of excerpts and sources which refer to the sub-trend and what the sub-trend says about the original code. For example: “6 excerpts from 3 sources identify increased savings from cheaper utilities as cost savings for beneficiaries. Digitally enabled solutions for energy and water utilities are allowing customers to spend less due to lower fees and more efficient use. This trend has significant co-occurrence with SDG 7 (Affordable and Clean Energy).” Notes should only describe the data without any further interpretation of the “so what?”, which should be saved for the Conclusion write up in the Findings, Conclusions, and Recommendations (FCR) Matrix. These sub-trend notes will be evolved into prose for the Findings section in your FCR matrix, so you may choose to write this analysis in a format that can just be copied and pasted to your Findings. If there are no triangulated sub-trends for a particular code, it will not be carried over to the FCR.
5. Write the Findings, Conclusions, and Recommendations Matrix (see our FCR blog post).

Some things to keep in mind while conducting secondary analysis:

• Themes are what we at Headlight call codes that have gone through secondary analysis and have been triangulated. They are generally a variant of what the analysts were originally coding for, so sometimes they mimic the evaluation questions, or sometimes they come from an emergent theme that showed up during the analysis. These themes help organize the sections in the final report based on the priority of the findings and any actionable evidence.
• For large subsets of excerpts under a single code, we recommend keeping a written/typed list of the secondary analysis codes you are using so you do not have to waste time scrolling through the excerpts to remind yourself of the codes you created.
• If you are analyzing a parent code, start by filtering for and identifying sub-trends for the child codes under that parent code (in the same worksheet E), then analyze any leftover excerpts that were not assigned to any original child codes.

For further information on conducting qualitative analysis, we recommend the Qualitative Data Analysis: Practical Strategies reference book by Bazeley, especially Chapter 13, “Exploring, Seeing, and Investigating Connections in Data.”

If you missed the previous post on primary analysis, check it out!

If you have any questions about qualitative analysis or need help implementing a Learning Review, Headlight would love to support you! We have the breadth and depth of experience to tailor-meet your needs. For more information about our services, please email info@headlightconsultingservices.org. Headlight is a certified woman-owned small business and therefore eligible for sole source procurements. We can be found on the Dynamic Small Business Search or on SAM.gov via our name or DUNS number (081332548).