Qualitative methods are intended to explore the socially constructed nature of human relationships by examining the intricacies of social phenomena as individuals look at and experience the world (Given, 2008).
Qualitative methods are intended to explore the socially constructed nature of human relationships by examining the intricacies of social phenomena as individuals look at and experience the world (Given, 2008). Qualitative tools are typically used to explore new emerging phenomena to describe and understand individuals' opinions, sentiments, experiences, behaviors, attitudes, beliefs, emotions, and social contexts. Data is therefore expressed in words and meaning, not in numbers or statistics, as in quantitative methods. According to Potton (2002), qualitative methods are suitable for selecting qualitative approaches for an evaluative inquiry, collecting reliable qualitative data, analyzing, and producing quality qualitative evaluation reports. The effectiveness of qualitative research also lies in its strength to explain non-tangible factors that are deep-rooted (e.g., gender relations and power, social norms, rituals and culture, societal barriers, race and ethnicity). By doing so, lived experience by a given population, and the complexity of their surroundings, can be verified, validated, and generalized into findings for a larger population. In qualitative methods, data is collected through a variety of ways such as open-ended questions and written comments on questionnaires, individual interviews, focus group discussions, logs, journals and diaries, observations, documents, reports, news articles, stories, case studies (Taylor-Powell & Renner, 2003).
Although qualitative methods have merits in generating compelling and anecdotal evidence helpful in shaping opinions and thriving changes, they can also result in vast amounts of data, resulting in complex qualitative data analysis processes (INTRAC, 2017). Seidel (1998) claimed that "the process of doing qualitative data analysis is complex” (p. 2). However, Taylor-Powell and Renner (2003) argued that analysis of qualitative data needs creativity, discipline, and a systematic approach, and there is no best or single way of doing that. They stated that the process for qualitative data analysis depends on three things: (1) the evaluation questions; (2) user information needs; and (3) availability of resources.
In this blog, I share my experience using qualitative methods by focusing on Most Significant Change (MSC) and Working with Codes by adopting content analysis as a basic model for data analysis and interpretation, to hear the voices from the field.
Example: In an evaluation I conducted on a leadership program, I used the MSC technique. MSC is a qualitative method using a story-based approach commonly used in complex social programs or development projects. The evaluator does not need to track social changes beforehand using pre-defined indicators, especially ones that must be counted and measured. MSC harvests the most significant stories from targeted project stakeholders at different levels to collect and select stories of change emanating from the field, which integrates data collection and analysis. It does not aim to assess changes, but rather to establish how changes happened, when, where, and why. This gives an avenue to quantify the observed outcomes with a different lens, other than the conventional evaluation method, without necessarily collecting quantitative data. With MSC, I was able to grasp the changes engendered by the leadership program at the personal development level, transformative changes at the institutional level, and changes observed at the community level through leadership empowerment of program participants.
- Working with Codes:
Working with software such as ATLAS.ti makes qualitative data analysis enjoyable, since themes and data patterns emerge throughout the coding process. Coding is referred to as a way of doing things by simplifying dense data to understand certain words, attribute meanings, and assign relationships by taking text data apart to see what they yield before putting the data back together in a meaningful way (Elliot, 2018). Emergent (inductive) or a priori (deductive) coding is intuitive and informative, to either build thematic analysis with emerging themes to enhance the effectiveness and impact of a given intervention in participants’ livelihoods, or to fashion content analysis to draw interpretation, gain a deeper understanding of phenomena, and infer communication for a given intervention context. To ensure a seamless qualitative data analysis, continued data collection innovation, and content coding form the basis for content analysis is needed to gain a deeper understanding of phenomena and new insights as data analysis unfolds (Lacy et al., 2015). Content analysis in qualitative methods can use either qualitative or quantitative data, and in an inductive or deductive manner.
Example: I evaluated the design and implementation plan of a scholarship program as a palliative measure to cater to the tuition fees of underpaid parents. Working with emergent codes to build inductive content analysis, the findings show that the program was well conceptualized, with a resilient management team confronted with weak funding cycles. Building on emergent codes, results show that the group was working on areas of improvement of scholarship delivery by enhancing the standards of the scholarship, focusing more on quality than quantity of recipients, increasing resource mobilization efforts, and diversifying funding sources for a sustainable program. Better interpretation of emergent codes and inductive content analysis helped to draw unbiased conclusions. The conclusion represented the voice of the program management, as the team worked on ways to enhance scholarship standards for future programming and strengthen scholarship competitiveness to make sure that students are better prepared for the entry test and be granted the scholarship by merit.
Limitations of Qualitative Methods
- While qualitative methods have their merits, it is encouraging to supplement qualitative data with quantitative data to comprehend the breadth and depth of the program evaluation and draw unbiased conclusions.
- Working with codes in qualitative data analysis, though intuitive and informative, generates a vast amount of codes that can be difficult to analyze, and it can be time-consuming if the evaluator does not have a clear data analysis strategy to make informed decision making on the type of data to be generated. Coding must be done with a clear purpose (Elliot, 2018).
- When an evaluator decides to use MSC, it is critical to select a realistic sample size. A small sample size does not provide room for validity and generalizability of evaluation findings. The selection of stories to be evaluated can be subject to bias, posing a high risk to the validity of the stories and changes engendered by an intervention.
References
Elliott, V. (2018). Thinking about the Coding Process in Qualitative Data Analysis. The Qualitative Report 2018. Vol.23. N. 11. How To Article 5, 2850-2861. University of Oxford, UK.
Given, L. M. (2008). The sage encyclopedia of qualitative research methods. Thousand Oaks, CA: Sage Publications.
INTRAC. (2017). Types of Evaluation. Retrieved January 22, 2023, from https://www.intrac.org/
Lacy, S., Watson, B.R., Riffe, D. and Lovejoy, J. (2015). Issues and best practices in content analysis. Journalism and Mass Communication Quarterly. Vol. 92. N.4. p.781-811. DOI: 10.1177/1077699015607338. Thousand Oaks, CA: Sage Publications
Patton. M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage Publications.
Siedel, J.V. (1998). Qualitative Data Analysis. Qualis Research. [Online] Available: www.qualisresearch.com
Taylor-Powell, E., and Renner, M. (2003). Analyzing Qualitative Data. Program Development & Evaluation (G3658-12), 1–12. [Online]. Available: https://www.betterevaluation.org/tools-resources/analysing-qualitative-data