Building Evaluation Capacity Through Data Jams, Part 3: Readying Extension for the Systematic Analysis of Large Qualitative Datasets

In this third blog post on the University of Wisconsin-Extension Data Jam Initiative, I will focus on four institutional outcomes of this Evaluation Capacity Building Framework.

Screenshot from the University of Wisconsin-Extension Civil Rights Legacy Datasets in MAXQDA.
Screenshot from the University of Wisconsin-Extension Civil Rights Legacy Datasets in MAXQDA.

INSTITUTIONAL OUTCOME 1: Continuous use of Institutionally Collected Data

The Data Jam Initiative provides colleagues with the tools, skills, support and community they need to engage in the analysis of large, often fragmented and hard-to-analyze textual datasets. We are currently conducting a longitudinal study measuring the initiative’s impact on analytic self-confidence and proficiency. At this early stage we observe heightened empowerment in Extension professionals, and we see a steep increase of evaluation, research and internal development projects that utilize the data from our central data collection system.

INSTITUTIONAL OUTCOME 2: Improvement of Institutional Data Quality

An essential element of the Data Jam Initiative is to communicate to colleagues and leadership how data are being used. Institutionally, this validates colleagues’ efforts regarding reporting, and it supports leadership in adjusting data collection foci based on ongoing, interdisciplinary data analysis. This, in turn, helps keeping institutional research, evaluation and communication efforts in alignment with ongoing data collection and storage.

INSTITUTIONAL OUTCOME 3: Building Interdisciplinary Capacity to Quickly Respond to Emerging Analytic Needs

All-Program area Evaluator Data Jam at the University of Wisconsin-Extension, March 2017.
All-Program area Evaluator Data Jam at the University of Wisconsin-Extension, March 2017.

Over time we create a baseline of shared techniques for analysis, and distributed proficiency in utilizing Qualitative Data Analysis software. Consequently, colleagues can tap into shared analytic frameworks when they collaborate on projects. On a larger scale, the institution can quickly and flexibly pull together analysis teams from across the state, knowing that a number of colleagues already share fundamental analytic and technical skills, even if they have never directly worked together. This allows an institution to respond quickly and efficiently to time-sensitive inquiries, and  to analyze more data more quickly, while bringing more perspectives into the process through work in larger ad-hoc analysis teams.

INSTITUTIONAL OUTCOME 4: Retaining Analytic Work through Legacy Datasets

Qualitative Data Analysis Software is designed to allow for detailed procedural documentation during analysis. This allows us to retain the analytic work of our colleagues, and to merge it into a single file. For example, we created a “Civil Rights Legacy Dataset” – a Qualitative Data Analysis Software file that contains all programming narratives containing information on expanding access to underserved or nontraditional audiences, currently from 2014 to 2016. This surmounts to approximately 1000 records, or 4000 pages of textual data. The file is available to anyone in the institution interested in learning about best practices, barriers and programmatic gaps regarding our work with nontraditional and underserved audiences.

The analyses that currently conducted on this dataset by various teams are being merged back into the “Legacy File”. Future analysts can view the work benches of prior analysts and projects, thus allowing them to use prior insights and processes as stepping stones. This enables the institution to conduct meta-analyses, maintain analytic continuity, and to more easily and reliably distribute analytic tasks over time or across multiple analysts. You can find more information on the use of Legacy Datasets in Extension in an upcoming book chapter, published in Silver & Woolf’s textbook on utilizing Qualitative Data Analysis Software.)

Beyond Qualitative Data: A Pathway for Building Digital Learning and Adaptation Skills

The outcomes above are immediate institutional effects the Data Jam Initiative was designed for. But maybe more importantly, we’re creating a base line of proficiency in negotiating between a technical tool and a workflow. Our tools change. Our methodological approaches differ from project to project. Each new project, and each new digital tool requires that we engage in this negotiation process. Every time, we need to figure out how we can best use a tool to facilitate our workflows; this skill is a fundamental asset in institutional professional development, and it transcends the topical area of evaluation.

This means that the Data Jam initiative, as an approach focused on mentorship and making by imbuing a technical tool with concrete, relevant processes, is not limited to qualitative data – it can be a framework for many contexts in which Extension professionals use software to do or build things: Be it visualization tools, digital design and web design, app development, statistics and quantitative research, or big data tools.

The development of the Data Jam Initiative Tool Kit has been supported by an eXtension Fellowship. To access the curriculum, examples, videos and training materials, please visit the UW-Extension Data Jam website: http://fyi.uwex.edu/datajams/