What Makes Collaborative Evaluation Approaches Collaborative? A Q Methodology Study on Program Evaluators’ Perceptions
College of Education and Human Services
Doctor of Education in Educational Leadership (EdD)
Leadership, School Counseling & Sport Management
NACO controlled Corporate Body
University of North Florida. Department of Leadership, School Counseling & Sports Management
Dr. Christopher Janson
Dr. David Hoppey
David Hoppey, Ph.D.
Dr. Katherine M. Kasten
Dr. Richmond D. Wynn
Dr. Elizabeth Gregg
Dr. Diane Yendol-Hoppey
The purpose of this study was to explore program evaluators’ perceptions about what they consider to be the essential practices, elements, or attributes of collaborative evaluation approaches (CEAs). This topic was investigated because of the increasing use of CEAs within the field and the ambiguity about what does and does not constitute a CEA. Several researchers have proposed frameworks that attempt to define the essential elements that distinguish CEAs from other evaluation approaches; however, the literature still lacks consensus on their exact nature. It was through an examination of evaluators’ viewpoints on the subject using Q Methodology that this study attempted to provide the field with empirical evidence clarifying the nature of CEAs.
Thirty-two program evaluators sorted 40 statements describing evaluation practices, elements, and approaches on a continuum from “least essential to CEAs” (-4) to “most essential to CEAs” (+4). These 32 sorts were factor analyzed and rotated. Following these procedures, four factors emerged that represented different perspectives on CEAs. Interpretation of these factors yielded distinct themes. These factor themes were named: (a) Culturally Connect with Stakeholders, (b) Talk to Stakeholders and Trust Will Follow, (c) Teach and Empower Stakeholders, and (d) Facilitate Stakeholder Communication.
The results of this study illustrate a varied assortment of perspectives that value culture, trust building, evaluation capacity building and empowerment, and communication as cornerstones to CEAs. Although this study was intended to be exploratory, the viewpoints that emerged help validate the literature about the purposes and dimensions that define and drive CEAs. Furthermore, this study indicates that while the primary goals of CEAs are relatively static, the means for achieving them will differ.
Weaver, Dax M., "What Makes Collaborative Evaluation Approaches Collaborative? A Q Methodology Study on Program Evaluators’ Perceptions" (2020). UNF Graduate Theses and Dissertations. 953.
Educational Assessment, Evaluation, and Research Commons, Other Social and Behavioral Sciences Commons, Other Sociology Commons, Quantitative, Qualitative, Comparative, and Historical Methodologies Commons, Social Policy Commons