Using Systems Concepts for Evaluation: Debate and Dialogue

Is Critical Systems Practice the best way to evaluate complex interventions? Distinguished systems theorist Professor Michael C. Jackson of the University of Hull asserts that critical systems practice should replace systems concepts as widely used in evaluation (see the reference to his published article below). With over 50 years as an evaluator, I, Michael Quinn Patton, disagree.
This video is my response to Jackson's critique of the use of systems concepts in evaluation. It is a lengthy response because Jackson's critique deserves an in-depth response covering both theory and practice. The video opens with an overview of the issues being debated and then goes into detail about the opposing points of view.
Here is the flow of the video and issues discussed and debated:
00:00:00 Debate overview
00:01:41 Systems concepts vs Critical Systems Practice: The Opposing Perspectives
00:10:51 Critical Systems Practice framework
00:12:50 Three core systems concepts used in evaluation
00:13:42 Jackson's critique
00:15:23 Use of systems concepts in evaluation -- point/counterpoint
00:16:34 Criteria for comparing perspectives
00:17:21 Systems concepts within larger evaluation frameworks
00:22:27 Contingency theory
00:23:28 No best practices
00:24:57 Real world evaluation
00:26:09 Utilization of systems concepts in evaluation -- examples
00:34:38 Philosophical foundations of systems concepts
00:36:21 Theoretical validity of systems concepts
00:46:28 Systems concepts contested
00:48:14 The central challenge summarized
00:48:43 Concept use counterpoints
00:49:53 Diversity of systems perspectives
00:52:11 Sensitizing concepts
00:54:12 Thick concepts
00:55:36 Context and concepts
00:58:50 Min specs
01:01:00 Good enough evaluation. satisficing, heuristics
01:03:13 Battle against simple linear reductionist thinking
01:05:00 Critical systems practices as mechanistic
01:08:40 Evaluating systems transformation in the context of the polycrisis
Reference: Jackson, M.C. (2022) Critical systems practice 4: Check-Evaluating and reflecting on a multimethodological intervention, Systems Research and Behavioral Science · October.
onlinelibrary.wiley.com/doi/1...
SEE recording of the debate: • Debate recording: Patt...
Production note: The editing is rough, and the sound volume varies.
My thanks to Charmagne Campbell-Patton, Pablo Vidueira, and Bob Williams for feedback on earlier versions of this video.

[@helens2821]

Thank you so much for taking the time out to respond to Prof Mike Jackson. This video is so useful and I will be sharing it widely.

[@DNOR-ds1vk]

Michael, I appreciate your critique effort and I like a couple of points on pragmatism. However, I can't weigh this evidence well given a few big misses in the base understanding of CST and CSP (easily addressed by reading more Jackson). Hence, I empathize with your "concern" about more pointed remarks about "evaluation" after such a major career contribution, but your use of terms like "dangerous" and "empty" are discursive distortions that need to be addressed on more equal terms. I am glad to see there is an upcoming debate on this with Jackson on the 27th Feb 23. This will be useful to get both views meeting on more robust Habermasian grounds, offering a more wholesome critique opportunity for all. While interesting, this medium for dialectic is a genuine example of one side of the story.

[@BarbaraSchmidtAbbey]

Thanks very much for sharing this very thorough and considered response to the referenced article by Prof Mike Jackson (which is also worth reading in full), and for bringing these issues out to the public debate - also using this format, rather than limiting it to academic journals - it's great, and ensures a much wider audience! It is a really important discussion, and looking forward to how it unfolds further, and to Prof Jackson's response.

[@pamelakelley7072]

I agree. I think this should be required viewing for all evaluators and anyone else with an interest in evaluation. Thank you MQP!

[@helens2821]

Agreed. I am thankful for the way he chose to share his thoughts- especially using as much plain English as possible. Plus it's not behind a paywall as so many academic works are.

[@lauraformenti4692]

Thank you, I am using very similar concepts in my practice to accompany the transformation of Residential Child Care programs in Italy. The systemic view can have many different interpretations. So many people claim to use a systemic framework, but then, what I see in their practice is linearity, the idea of separated phases (you have first to observe, than plan, then evaluate) and "best practices" is dominant in the field. Responsiveness and reflexivity are my keywords. I am tracking your work, really very useful.

[@cameo03021]

Very helpful! Thank you so much!

[@RickDelmonico]

Embodied; cognition is not just computation. Embedded; cognition relies on what we are connected to. Enacted; is a cogitative relationship between expression and experience. Extended; cognition can be transmuted into alien forms of expression and experience. Emanation: to issue forth out of a source. Emergence: a collective principle of organization. Emotive; energizing feelings and behaviors.

[@RickDelmonico]

The goal for the pseudo intellectuals is to plan the future, to remove as much freedom of choice as possible. To build a rigid framework of predictable outcomes. The problem is imagination. Intellectuals seldom create the future they imagine because the future is emergent and plans are foolish. All plans work at the level of simple and begin to break down as they approach complicated. When we get to the level of complexity, there are no useful plans, only ethical principles and wise conduct. It's not the horizon, it's the direction that matters.