Evaluating outputs and outcomes: A study in transformation

Research output: ThesisDoctoral Thesis

3 Downloads (Pure)

Abstract

How a project is perceived by its stakeholders affects how it is implemented, and how the outcomes of the project are interpreted by the stakeholders influences the impact project can have on those stakeholders. Often this diversity of perspective is considered an impediment to the effectiveness of the project in meeting its goals. Standard project evaluation techniques dependent on linear and conventional methods to assess and present outputs and outcomes from projects fail to consider the complexity in projects. Complexity in a project arises from the involvement of multiple stakeholders from diverse disciplines, backgrounds, and geographies, and having varied perceptions, expectations, and understanding of the project and its aspects.

The overall aim of this PhD is to improve the understanding of evaluation of complex projects by studying the projects from the perspectives of the multiple stakeholders involved in them. The first objective is to explore and understand the approaches to evaluation drawing on perspectives from literature, and observations from the field. The second objective is to understand the perspectives of stakeholders operating at various levels of a complex project on different aspects of the project such as its nature, approach, outputs, and outcomes. The third objective is to relate outcomes at various levels in the project to processes used, as well as associate outputs with outcomes. The fourth objective is to develop an integrated approach to evaluate complex multi- stakeholder projects, which enhances a project’s outcomes and enables learning for the stakeholders involved.

With the aim of improvement in the existing knowledge on evaluating complex projects, the methodological approach is developed from a combination of theories and practices on evaluation. Central themes of the methodology are methodological pluralism, multiple perspectives, systems thinking, and appreciation and learning. To facilitate flexibility in navigating through a variety of theories and perspectives to enable both change and enhancement, the PhD is undertaken and presented as an action research. Three complex projects with stakeholders from diverse backgrounds and disciplines are examined in two stages of this thesis. These projects situated on the Chotanagpur Plateau in India with different intervention areas are, i) an agricultural research for development (AR4D) project, ii) a project to develop the skills of community youth to impart education, and iii) a Corporate Social Responsibility initiative. Data are collected from 82 project participants chosen by purposive sampling in the form of narratives, through semi-structured questionnaires.

Findings from examining multiple perspectives were similar across the three studied projects. Stakeholders interpreted the nature and outcomes of the project uniquely. This study confirmed the existence of diverse stakeholder perspectives that were not captured or acknowledged in the evaluation of the three projects. These perspectives, however, were important for the stakeholders in how they identified with the project, how they functioned in it, and eventually, how it impacted their lives. Moreover, largely, there was no cognisance of this diversity in the stakeholders of the project. In instances where the stakeholders were aware of the multiple views, there was no mechanism for interaction of, or sharing those perspectives. Neither did the project stakeholders learn to acknowledge and work with varied perspectives, nor did they learn from multiple views in the project which were different from theirs. Besides the standard outputs and outcomes from the project, the project stakeholders outlined long-term personal changes. In particular, the learning which they underwent was considered profound and significant. The subtle shifts in learning and development of capabilities in project stakeholders were capabilities that enhance their sense of agency and change their worldviews, which they may further utilise to impact the project, themselves, and others.

In considering these findings and addressing the challenge of incorporating complexity in project evaluation, the thesis develops a framework to evaluate complex projects. The framework is complexity-appreciative which acknowledges, appreciates, and integrates multiple perspectives in the design and evaluation of projects. Evaluation frameworks are always dependent on the contexts in which they are applied, and on those who design and use them, and the kind of boundary judgements they make. Hence, the framework provided in this PhD is not a tool to be used at the end of a project to measure its outcomes; rather, it is a process that must be part of a project from inception as a feedback tool to enhance outcomes. The framework can become a means to create spaces and processes in a project to enable stakeholders to share perspectives, listen to others, understand the diversity in the project, and acknowledge, appreciate and learn from each other’s perspectives as well as each other’s process of learning. Such a space will also allow stakeholders to find their voice and purposes in the project, to help each other do the same, and to further develop those purposes
Original languageEnglish
QualificationDoctor of Philosophy
Awarding Institution
  • Charles Sturt University
Supervisors/Advisors
  • Ramsay, Gavin, Principal Supervisor
  • Hernandez-Jover, Marta, Co-Supervisor
Award date05 Feb 2021
Place of PublicationAustralia
Publisher
Publication statusPublished - 2020

Fingerprint Dive into the research topics of 'Evaluating outputs and outcomes: A study in transformation'. Together they form a unique fingerprint.

Cite this