Introduction to some core dimensions of evaluation in citizen science
In the following video we dive further into the Citizen Science Evaluation Framework that we have developed in 2018 together with some colleagues in the field of citizen science (find the video-transcript here).
The Citizen Science evaluation framework is structured like a matrix and presents two different areas of evaluating a project:
a) The process-based evaluation ("Process & Feasibility" in the figure below) collects formative input about the processes of setting up and implementing a citizen science project and is the basis for an adaptive project design and management.
b) The outcome-based evaluation ("Outcome & Impact" in the figure below) brings summative evidence of a project’s benefits to its participants and their surrounding contexts and shows how much an intervention’s impact contributes to the project’s expected and possibly unintended goals.
Image: The Citizen Science evaluation framework (Kieslinger,
B., Schäfer, T., Heigl, F., Dörler, D., Richter, A. & Bonn, A.
(2018). Evaluating Citizen Science: Towards an open framework. In: Hecker, S. et al. (eds.), Citizen Science – Innovation in Open Science, Society and Policy. London: UCL Press.)
For both, the process- and outcome-based evaluation, the framework presents indicators that investigate three dimensions of participatory scientific processes:
1) science
2) participants and individual actors
3) socio-ecological and economic systems
Since the
framework is comprehensive its application needs tailoring
and contextualising according to the spatial, temporal and socio-economic
demands of the project or program. Criteria in the matrix need to be prioritised and may
receive different weightings depending on project goals.
Please look at the video to learn more about this framework.
Then move to the next section, where we will go into more detail of these dimensions and show practical examples of evaluating the outcomes for science, individual participants and their surrounding communities and systems.
Reference: Kieslinger, B., Schäfer, T., Heigl, F., Dörler, D., Richter, A. & Bonn, A. (2018). Evaluating Citizen Science: Towards an open framework. In: Hecker, S. et al. (eds.), Citizen Science – Innovation in Open Science, Society and Policy. London: UCL Press.